Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Mon 2019-01-21T15:32:27-0800 Paul Hirose hath writ: > On 2019-01-16 1:35, Steve Allen wrote: > > So starting 1970-01-01 DCF77 changed to broadcast Stepped Atomic Time > > (SAT) with second markers that were 1 SI second apart except on > > occasions when they were 1.2 SI seconds apart in order that the time > > markers would stay close to UT2. > I don't know when DCF77 inserted the step adjustments, so I created a small > table sufficient for an example. In Bulletin Horaire series J there are announcements of when and how big jumps should be inserted into SAT. I don't know what meeting prescribed that responsibility to them, but they did it. If the BIH continued to do this then when Germany shifted from old rubber UTC to SAT in 1970 they were almost certainly following the BIH jumps. Starting in 1967 BIH stopped Bulletin Horaire and started Circular D and their Annual Report. The BIH Circulars may be hard to find, but BIH Annual Report is in libraries. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 2019-01-16 1:35, Steve Allen wrote:
So starting 1970-01-01 DCF77 changed to broadcast Stepped Atomic Time
(SAT) with second markers that were 1 SI second apart except on
occasions when they were 1.2 SI seconds apart in order that the time
markers would stay close to UT2.
For fun (?) I wrote a rudimentary C# program to convert an epoch in the
DCF77 scale to proleptic GPS week and second. It uses the JulianDate,
Duration, Utc, and UtcTable classes in my SofaJpl fundamental astronomy
DLL for Windows .NET Framework.
I don't know when DCF77 inserted the step adjustments, so I created a
small table sufficient for an example. Beginning in January the rate
offset becomes zero. There's a step adjustment (almost .2 s) at the end
of the month. To make the example more interesting, I set the epoch
within that step.
Calculations are performed in the TT scale, which my DLL favors since
it's written for astronomy. However, the JulianDate class can represent
TAI or any unstepped time scale. Scales with step adjustments require
the Utc class and a defining table.
This example begins with a date and time in my fake DCF77 time scale and
converts it to TT, then GPS week and seconds, then completes a round
trip via TT to date and time in the original scale as well as UTC in the
rubber second era.
// Simulated DCF77 time scale. First 2 lines from the real UTC table,
// the rest phony. Step adjustment at end of 1970 Jan 31.
string dcf77TableString =
@"1966 JAN 1 =JD 2439126.5 TAI-UTC= 4.3131700 S + (MJD - 39126.) X
0.002592 S
1968 FEB 1 =JD 2439887.5 TAI-UTC= 4.2131700 S + (MJD - 39126.) X
0.002592 S
1970 JAN 1TAI-UTC= 8.820 S + (MJD - 40587.) X 0.0
S
1970 FEB 1TAI-UTC= 8.200 S + (MJD - 40587.) X 0.0
S
1970 MAR 1TAI-UTC= 8.200 S + (MJD - 40587.) X 0.0
S";
// Load the DCF77 table from the string.
UtcTable dcf77Table = new UtcTable(
new System.IO.StringReader(dcf77TableString));
// Load the default UTC table. This is the USNO table.
Utc.LoadTableFromFile(@"C:\Users\Stan\Documents\astro\IERS\leapSecTable.txt");
// Specify desired time display precision (the unit is 1 / hours).
double timePrecision = 3.6e5; // .01 s
// base epoch of GPS time scale (TT)
JulianDate gpsBaseEpochTT = new Utc(1980, 1, 6, 0).TerrestrialTime;
// length of one week and the GPS cycle of 1024 weeks
Duration weekLength = new Duration(7);
Duration cycleLength = new Duration(7 * 1024);
// Set DCF77 epoch to 1970-01-31 23:59:60.05
Utc dcf77 = new Utc(1970, 1, 31, Angle.HmsToDays(23, 59, 60.05),
false, dcf77Table);
// Display DCF77 epoch.
Console.WriteLine("{0} DCF77 epoch",
dcf77.ToTimeFields(timePrecision));
// Convert to TT, display.
JulianDate tt = dcf77.TerrestrialTime;
Console.WriteLine("{0} TT", tt.ToTimeFields(timePrecision));
// elapsed time (possibly negative) since GPS time scale origin
Duration elapsedTime = tt - gpsBaseEpochTT;
// 1024-week cycle number (negative if before 1980)
int cycleNumber = (int)Math.Floor(elapsedTime / cycleLength);
// TT at beginning of applicable 1024-week cycle
JulianDate weekZeroTT = gpsBaseEpochTT + cycleNumber * cycleLength;
// elapsed time from start of cycle
elapsedTime = tt - weekZeroTT;
// GPS week number
int weekNumber = (int)Math.Floor(elapsedTime / weekLength);
// elapsed time from start of week
elapsedTime -= weekNumber * weekLength;
// convert elapsed time to seconds
double seconds = elapsedTime.ToSeconds();
// Show results.
Console.WriteLine("{0:d} = cycle number", cycleNumber);
Console.WriteLine("{0} = GPS week", weekNumber);
Console.WriteLine("{0:f2} seconds", seconds);
// Convert cycle number, week number, and seconds to DCF77 and UTC.
// elapsed time since start of applicable 1024-week cycle
elapsedTime = Duration.FromSeconds(seconds) + weekNumber * weekLength;
// Add the time in the whole cycles since GPS time origin
elapsedTime += cycleNumber * cycleLength;
// Convert to TT.
tt = elapsedTime + gpsBaseEpochTT;
Console.WriteLine("{0} TT", tt.ToTimeFields(timePrecision));
// Convert to epoch in the DCF77 scale.
dcf77 = new Utc(tt, dcf77Table);
Console.WriteLine("{0} DCF77 epoch", dcf77.ToTimeFields(timePrecision));
// Also convert to UTC.
dcf77 = new Utc(tt);
Console.WriteLine("{0} UTC", dcf77.ToTimeFields(timePrecision));
output:
1970-01-31T23:59:60.05 DCF77 epoch
1970-02-01T00:00:40.23 TT
-1 = cycle number
505 = GPS week
604789.05 seconds
1970-02-01T00:00:40.23 TT
1970-01-31T23:59:60.05 DCF77 epoch
1970-01-31T23:59:59.97 UTC
Hand computation confirms the TT and UTC are correct. I sure any GPS
week calculator would reject the date as illegal, so that part I tested
near the present time.
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 2019-01-18 17:11, Michael H Deckers wrote: .. insert a step of 0.2 s in their time signal about every 71 days. when he meant "about every 77 days". Michael Deckers. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Wed 2019-01-16T22:24:46-0800 Paul Hirose hath writ: > On 2019-01-16 1:35, Steve Allen wrote: > > > > Does it know that rubber seconds do not apply to timestamps in central > > Europe made using DCF77 from 1970-01-01 to 1972-01-01? > > All my DLL knows is what's in the UTC file. If the file indicates no rate > offset in the years 1970-71 and some fractional second step adjustments, > that's what you get. An existing file could be edited to match the DCF77 > flavor of UTC. Then construct a UtcTable object from the file: There lies the problem with timestamps, then and now. The folks running DCF77 had convinced themselves that German law did not permit them to broadcast seconds which were not SI seconds, so they stopped broadcasting the rubber second version of UTC which was then the CCIR standard. Instead they convinced themselves that broadcasting "mostly legal" SI seconds was more acceptable under German law. The issues of Bulletin Horaire over the decades show that the existence of any difference between the time signal broadcasts of any service was not acceptable and repeatedly resulted in internatioal meetings with recommendations putting pressure on every transmitter to conform. This was the reason for the creation of UT2. In the 1950s the issues of Bulletin Horaire show that the UK had started correcting their broadcasts, first for what would become called UT1, and then for what would be called UT2. The US Navy broadcasts did likewise, but using a different expression for what would become called UT2. The US NBS WWV broadcasts chose to prefer more steps and fewer frequency offsets than US Navy because the NBS was more interested in standard frequency than in astronomical time. Bulletin Horaire says that by 1955 there were at least 4 different schemes being used to produce broadcasts of time scales akin to what would become called UT2. Bulletin Horaire explains that was the motivation behind the 1955 IAU decision to create UT0, UT1, and UT2. Then folks became uncomfortable with the way that different stations attacked the problem of broadcasting something like UT2. The US Navy method of approximating UT2 using fewer time steps and more changes of frequency offset was good for navigators who would not see their position suddenly shift. The US Navy method was also an irritant, in part to folks who had been driving CCIR recommendations toward making better use of spectrum by reducing spacing between stations using adjacent frequencies, and in part to engineers who had to retune the transmitting gear. The existence of cesium frequency standards made it easy for everyone to see the changes in offsets that had previously been hard to measure, but it also allowed for the agreement on the original form of rubber second UTC where everyone agreed to use the same frequency offsets and time steps. The first version of that agreement was forged over tea in the living room of H.M. Smith, one of the Greenwich engineers associated with UK radio broadcast time signals over the span of his career. After the US and UK had codified their rubber second agreement it was taken to URSI, CCIR, and IAU to get official international status as recommendations. After the CGPM approved the cesium SI second, the German decision that in 1970 they would no longer conform to rubber second UTC led to urgent need to create yet another agreement on a different way of broadcasting time that could be legally tolerated in all countries. There was also dissatisfaction with the changes in frequency and time steps from the LORAN engineers who had to re-tune and re-phase entire chains of transmitters, and the LORAN users who could not navigate during the interval when the chain was retuning and rephasing. So for your software the German decision means that the only way to decode timestamps in central Europe from 1970/1972 is to find the publications where they announced when DCF77 inserted their markers which were spaced by 1.2 SI seconds. The USNO circulars which announced changes in their broadcasts were printed on high-acid paper which has self-destructed while sitting in the few libraries which bothered to store them. I do not even know the name of the German publications which would have announced their time signal steps. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 2019-01-16 1:35, Steve Allen wrote: Does it know that rubber seconds do not apply to timestamps in central Europe made using DCF77 from 1970-01-01 to 1972-01-01? All my DLL knows is what's in the UTC file. If the file indicates no rate offset in the years 1970-71 and some fractional second step adjustments, that's what you get. An existing file could be edited to match the DCF77 flavor of UTC. Then construct a UtcTable object from the file: SofaJpl.UtcTable dcf77Table = new SofaJpl.UtcTable(filename); Construct a Utc object with that table as a parameter. SofaJpl.Utc utc = new SofaJpl.Utc(year, month, day, fracday, dcf77Table); Variable "utc" will convert itself to TT based on the custom table. Then, to complete a round trip, create another Utc object from that TT and the custom table. When formatted as date/time you get the original UTC. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 1/16/19 12:31 AM, Poul-Henning Kamp wrote: In message <20190115205243.gb25...@ucolick.org>, Steve Allen writes: On Tue 2019-01-15T12:34:11-0800 Gary E. Miller hath writ: Yes, and no. time_t is just seconds since an epoch. Which epoch is not well defined. The epoch may well be anything. See "man difftime". That evokes a challenge for all time nuts that I can make based on reading Bulletin Horaire. What is the epoch that was used for TAI? Isn't that the same one Loran-C used ? 1958-01-01 00:00:00 GMT ? You might be thinking of CCSDS Unsegmented Time - in the header of the time message you specify what the epoch is - It's either the 1958 (beginning of "space age") or "user defined" Similar for CCSDS Day Segmented Code which is 16 or 24 bits of days after epoch, milliseconds, and (optional) submilliseconds They also clone the ISO UTC called ASCII Calendar Segmented Time Code - -MM-DDThh:mm:ss.ddZ as well as DOY versions, etc. They manage the leapsecond with a 58 or 61 second minute. It's also clear that CCSDS doesn't like non-continuous, non-monotonic time scales: " f) Standardization on the use of these time code formats for purposes OTHER than identifying an instant of calendar or time in UTC (e.g., unconventional use as a counter or tool for measuring arbitrary intervals) is not recommended. It is felt such a specialized application can best be viewed not as a time code format but rather as an engineering measurement format. Any such application of these time code formats is considered beyond the scope of this Recommended Standard. " https://public.ccsds.org/Pubs/301x0b4e1.pdf "The CCSDS-Recommended epoch is that of 1958 January 1 (TAI) and the recommended time unit is the second, using TAI as reference time scale, for use as a level 1 time code. This time code is not UTC-based and leap-second corrections do not apply." Bit1-3 =Time code identification 001 — 1958 January 1 epoch (Level 1 Time Code) 010 — Agency-defined epoch (Level 2 Time Code) Level 1 in CCSDS means "complete and unambiguous interpretation", Level 2 means "you need some other information" ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 1/15/19 11:36 PM, Warner Losh wrote: > It's a fundamental flaw of UTC. There are other ways to implement > something akin to UTC that implements some form of mean solar time > that's not an observational calendar. If either part of this is consensus, then a proposal to redefine UTC after a certain date is also fundamentally flawed. Redefinition cannot simplify a standard - any standard - it can only make it more complex, e.g.: "before the specified date, the standard behaves one way, after it behaves another way". Timekeeping has many retroactive use cases, including all those intervals people mention. What about instances that cross the redefinition boundary and the two end points have two different meanings? The engineering requirements to support the original definition never go away. The vast majority of timestamps trace back to GPS and other GNSS-derived values, even if GPS is deemed an unclean standard by the lab-coated acolytes of the atomic clocks (https://bit.ly/2MeCcGp). Or TAI itself is already available - indeed, UTC provides TAI as well as a prediction of UT1. Or simply define a new unsegmented TI timescale as was the consensus at the Torino meeting in 2003. Redefining UTC would be a colossal blunder. The time wasted over the past two decades on this fool's errand could have been better spent standing up a new TI infrastructure, or simply writing better documentation. That said, nothing about the current discussion bears on the definition of UTC, only on how best to handle conversions. Rob ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Wed 2019-01-16T00:00:55-0800 Paul Hirose hath writ: > The "rubber second" era is supported. That accounted for about 90% of the > workload to create the UTC implementation! Does it know that rubber seconds do not apply to timestamps in central Europe made using DCF77 from 1970-01-01 to 1972-01-01? After the CGPM redefined the SI second in terms of cesium the German time service bureau determined that they could not use the old scheme of BIH UT2 with its rubber seconds. They had convinced themselves that German law required them to broadcast SI seconds. So starting 1970-01-01 DCF77 changed to broadcast Stepped Atomic Time (SAT) with second markers that were 1 SI second apart except on occasions when they were 1.2 SI seconds apart in order that the time markers would stay close to UT2. This was one of the issues of concern in the CCIR preparatory meetings that led to the decision to have leap seconds. So they were mostly broadcasting legal SI seconds, but too early in history to evoke comparisons with "mostly harmless" and "mostly dead". -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Wed 2019-01-16T09:03:26+ Poul-Henning Kamp hath writ: > In message <20190116085619.ga23...@ucolick.org>, Steve Allen writes: > >The epoch of TAI, and LORAN, and GPS is 1961-01-01T20:00:00 UT2. > >Or maybe 1961-01-00 UT2. > > > >That is when the atomic time scale was set to a specified value. > > But the specific value they set was not zero in 1961-01-01T20:00:00 UT2, > they set it so it would have been zero at 1958-01-01 00:00:00 GMT Because of the reset at 1961 (and other ingredients in the sausage of time) the difference between UT2 and TAI is not zero at 1958-01-01. It is close to zero, but it is not zero. One of the ingredients is that as of 1958-01-01 USNO was intentionally using a longitude that differed by 0.035 time seconds from the globally consistent value. I also have not read closely enough to say whether those offset values in the series H inception of A3 correspond to the values of UT2 before or after the longitudes of all stations changed on 1962-01-01. The global average for that shift was 1.5 ms. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
In message <20190116085619.ga23...@ucolick.org>, Steve Allen writes: >On Wed 2019-01-16T08:31:19+ Poul-Henning Kamp hath writ: >> In message <20190115205243.gb25...@ucolick.org>, Steve Allen writes: >> >That evokes a challenge for all time nuts that I can make based on >> >reading Bulletin Horaire. >> > >> >What is the epoch that was used for TAI? >> >> Isn't that the same one Loran-C used ? 1958-01-01 00:00:00 GMT ? > >Nope. > >The epoch of TAI, and LORAN, and GPS is 1961-01-01T20:00:00 UT2. >Or maybe 1961-01-00 UT2. > >That is when the atomic time scale was set to a specified value. But the specific value they set was not zero in 1961-01-01T20:00:00 UT2, they set it so it would have been zero at 1958-01-01 00:00:00 GMT -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Wed 2019-01-16T08:31:19+ Poul-Henning Kamp hath writ: > In message <20190115205243.gb25...@ucolick.org>, Steve Allen writes: > >That evokes a challenge for all time nuts that I can make based on > >reading Bulletin Horaire. > > > >What is the epoch that was used for TAI? > > Isn't that the same one Loran-C used ? 1958-01-01 00:00:00 GMT ? Nope. The epoch of TAI, and LORAN, and GPS is 1961-01-01T20:00:00 UT2. Or maybe 1961-01-00 UT2. That is when the atomic time scale was set to a specified value. See a few pieces of the way that sausage was made here https://www.ucolick.org/~sla/temporary/taiepoch/ -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
In message <20190115205243.gb25...@ucolick.org>, Steve Allen writes: >On Tue 2019-01-15T12:34:11-0800 Gary E. Miller hath writ: >> Yes, and no. time_t is just seconds since an epoch. Which epoch >> is not well defined. The epoch may well be anything. See "man difftime". > >That evokes a challenge for all time nuts that I can make based on >reading Bulletin Horaire. > >What is the epoch that was used for TAI? Isn't that the same one Loran-C used ? 1958-01-01 00:00:00 GMT ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
In message <20190115123411.44a0c...@spidey.rellim.com>, "Gary E. Miller" writes : >> But the underlying time_t cannot represent leap seconds, can it? > >Yes, and no. time_t is just seconds since an epoch. Which epoch >is not well defined. The epoch may well be anything. See "man difftime". I was lucky enough to share breakfast with Dennis Richie at the USENIX ATC in New Orleans in 1998, and we talked a lot about the history and future of /dev and timekeeping. Originally time_t, which wasn't called that yet, was only 16 bits, and the epoch was whenever you booted the system and because the machines lacked memory management hardware, "rollover was usually not a problem". >time_t dates to the first edition of "Unix Programmer's Manual, November >3, 1971". No timezone or other basis is given for time_t. They just >said "system time". The first leap second was in 1972! time_t never >caught up. >In 1973, just after the third edition, time_t was pegged to GMT. > >According to wikipedia, GMT is mean solar time, not UTC. The seconds >are not the same length, and no leap seconds: Wrong. At the time GMT was what people had always called UTC, and there were no difference between them. >Some long time after that the doc changed from GMT to UTC, but the code >did not. time() returns a time_t which pretends to be UTC, but ignores >leap seconds. From a current "man time": And that was simply how AT telephone network decided to handle leapseconds: There were far too many time keeping devices installed throughout the network for anybody to contemplate implementing leap seconds on them. Most of them didn't keep time that well anyway. I have not yet been able to pinpoint the first UNIX computer which had hardware good enough to tell if leap-seconds were mishandled, my best candidate right now is the Cray Y/MP. The first networking of UNIX computers were over serial lines inside individual labs, and the applications couldn't care less about what time it was anyway. Nobody even dreamed about using computers as authoritative clock for a long time. As far as anybody has been able to make out, the first precision time synchronization over computer networks, where leap seconds could even be relevant, is Dave Mills work on FuzzBall and NTP, in the first half of the 1980-ies on NSFnet - and the FuzzBalls didn't run UNIX and only moved packets. Even in 1988 when the UNIX API was rubberstamped into POSIX by IEEE, nobody but Dave Mills really cared about leap-seconds, and even he didn't care much about it. See for instance his last paragraph of RFC958 from 1985. When leap seconds finally appear on the radar, a freak geophysical happenstance means there are no leap seconds for the precise six years where most stuff is connected the internet. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 2019-01-15 10:50, Tom Van Baak wrote: For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it comes out 0600) Here's a C# code fragment which performs a similar computation. To make the demonstration more interesting I arrange for the augmented UTC to fall in the middle of a leap second. The program calls my SofaJpl fundamental astronomy DLL for the Windows .NET Framework. "Sofa" refers to the IAU SOFA (Standards of Fundamental Astronomy) library, and "Jpl" refers to the Jet Propulsion Laboratory planetary ephemerides. But there's no astronomy in this little example. // Open the leap second table. The '@' prefix on the // string causes the C# compiler to not interpret a backslash as an // escape character. SofaJpl.Utc.LoadTableFromFile(@"C:\Users\Stan\Documents\astro\IERS\leapSecTable.txt"); // Construct a Utc object at the initial epoch. SofaJpl.Utc utc1 = new SofaJpl.Utc(2016, 12, 31, 0, 0, 0); // Construct a JulianDate object at the same epoch (TT scale). SofaJpl.JulianDate tt = utc1.TerrestrialTime; // Construct a Duration object with the amount of time to add. SofaJpl.Duration timeIncrement = SofaJpl.Duration.FromSeconds(86400.5); // Add the time to TT. tt += timeIncrement; // Convert the incremented TT to a new Utc object. SofaJpl.Utc utc2 = new SofaJpl.Utc(tt); // The desired time display precision (the unit is 1 / hours). double timePrecision = 36000.0; // .1 s precision // Display the initial and final UTC epochs. Console.WriteLine(utc1.ToTimeFields(timePrecision)); Console.WriteLine(utc2.ToTimeFields(timePrecision)); Output: 2016-12-31T00:00:00.0 2016-12-31T23:59:60.5 The leap second table follows the USNO format with some extensions. 1) For easy updates by hand, the Julian date field is ignored and may be blank. 2) The final line must have the same TAI-UTC as the previous line, but a later date. This line defines the final epoch of the table. An exception occurs if you exceed it. http://maia.usno.navy.mil/ser7/tai-utc.dat The "rubber second" era is supported. That accounted for about 90% of the workload to create the UTC implementation! JulianDate and Duration objects both represent whole days and fractional days in separate double precision variables. Operators are defined if they make sense. For instance, if you subtract one JulianDate from another the result is a Duration. You can add two Durations but addition of two JulianDates is undefined. For convenience, the Duration class includes the fixed offsets TTMinusGps and TTMinusTai. Time conversions with variable offsets such as TTToTdb() are in the JulianDate class. By design, the Utc class has limited facilities. You can create a Utc object from date and time components or a JulianDate in the TT scale. The inverse operations are available too. But as shown in the example, any time math requires explicit conversion to a JulianDate object in the TT scale, then conversion back to a Utc object if the user needs to see the result in UTC. These conversions are lossless since a Utc object internally stores its value as a JulianDate in TT. (The DLL is intended for astronomy, where TT is more useful than TAI.) By not defining addition or subtraction on Utc objects I avoid ambiguity about the exact meaning of these operations in a stepped time scale. A TimeFields represents epoch as year, month, etc. (It's not called "DateTime" because the Windows .NET Framework already has such a class.) The main purpose of the class is to format JulianDate and Utc objects for output, rounded to the specified precision. For example, if a JulianDate is a few seconds from the end of the year, and precision is 60 (= one minute), the conversion to a TimeFields rounds up to January 1 00:00 in the next year. It's possible to construct named TimeFields objects and extract the components individually for output. But for simplicity my example creates unnamed objects. By default they "stringize" themselves in ISO 8601 format at the specified precision (.1 second in this case). SofaJpl is CLS compliant and therefore can be called by any CLS language, as shown by the C#, C++/CLI, and Visual Basic demonstration programs at the web page. The DLL was created primarily for personal use on my Windows system, but anyone can download it for free. A full installation is almost exactly 1 megabyte. http://home.earthlink.net/~s543t-24dst/SofaJpl_NET/index.html ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue, Jan 15, 2019 at 10:09 PM Gary E. Miller wrote: > Yo Warner! > > On Tue, 15 Jan 2019 17:45:32 -0700 > Warner Losh wrote: > > > > > Except that's not a leap second. struct tm doesn't do leap seconds > > > > really. It kinda sorta does, but it kinda sorts doesn't. > > > > > > Well, it does in a modern Linux kernel. Otherwise NTPD would fail > > > on the leap second, and that only happens every year so. :-) > > > > > > > You are wrong. The kernel doesn't use struct tm. > > I never said the kernel uses struct tm. I said you can query the kernel > for > the time in a struct tm. > What system call is that? A quick grep didn't turn anything up for me. All the calls I'm aware of are library calls that get the system time and then break it up. > > The kernel does this > > by ticking either in TAI or by incrementing a TAI offset. ntpd uses a > > different interface that exposes these things to it so it can > > broadcast the leap second indicators correctly. > > Yup. > > > > > First, when you convert back and forth to time_t, you lose leap > > > > second information. It's impossible to convert back and forth > > > > from at least the UTC time zone. There's some 'right' time zones, > > > > but they aren't strictly speaking POSIX compliant because it uses > > > > the wrong math. Non-conforming, but damned useful and not wrong. > > > > > > Which is why you can't use time_t in UTC, but time_t in TAI > > > can work, as well as struct tm, usually. Lot's of latent bugs... > > > > > > > No. time_t is not TAI or GPS. It is UTC. Adjusted UTC. > > No, time_t is just a way to count seconds. See man difftime() for a > use of time_t that is unrelated to any particular epoch. > This is the biggest mistake people make with POSIX time_t. It's not just a way to count seconds. It is the funky seconds since 1970 thing. By definition. difftime takes two time_t's and returns a double that's the difference between them. This is only a time_t when time_t is defined to be a double (which is allowed, but apart from IBM, nobody has done that), otherwise it is not. Typically, time_t is some int type, so difftime isn't a good example here. Also, the open group standard is silent about what it's supposed to do across a leap second, but since it operates on time_t, it's unclear if a time_t of 1230767 and 1230768002 should be 3 or 4 since those two times straddle 2009 December 31, 23h 59m 60s. I've seen spirited debate on both sides of that issue. > It is the > > number of SI seconds since 1970, minus all the positive leap seconds, > > plus all the negative ones. > > Ciration please? I already cited the Linux doc that says otherwise. > The Linux doc does not define the standard. The POSIX doc does (from the Open Group 2017 edition): 3.150 Epoch The time zero hours, zero minutes, zero seconds, on January 1, 1970 Coordinated Universal Time (UTC). 4.16 Seconds Since the Epoch A value that approximates the number of seconds that have elapsed since the Epoch. A Coordinated Universal Time name (specified in terms of seconds (tm_sec), minutes (tm_min), hours (tm_hour), days since January 1 of the year (tm_yday), and calendar year minus 1900 (tm_year)) is related to a time represented as seconds since the Epoch, according to the expression below. If the year is <1970 or the value is negative, the relationship is undefined. If the year is >=1970 and the value is non-negative, the value is related to a Coordinated Universal Time name according to the C-language expression, where tm_sec, tm_min, tm_hour, tm_yday, and tm_year are all integer types: tm_sec + tm_min*60 + tm_hour*3600 + tm_yday*86400 + (tm_year-70)*31536000 + ((tm_year-69)/4)*86400 - ((tm_year-1)/100)*86400 + ((tm_year+299)/400)*86400 which is equivalent to what I said based on how leap seconds work. I've done a lot with POSIX time_t and what is and isn't stated in the standard. Linux may implement something that's not strictly POSIX compliant, but the root of the problem with time_t is POSIX defines it very particularly and when people deviate from the standard, or make their own up when the standard is silent, they do so is somewhat divergent ways (which is part of the problem as well as leap seconds not being defined in time_t as there's no way to represent them uniquely in a time_t, despite them existing in the non-uniform radix UTC notation). > > > If you squint at it, GPS time (gps weeks, time of week) is just > > > another notation very similar to time. And we know that works. > > > > > > > GPS time has no leap seconds. It's a monotonic count since 1980. > > Correct. > > > > Since future leap seconds are unknown, and unknowable, there is no > > > possible valid way to do this on the time scale of years. > > > > > > > Correct. This is the fundamental flaw of UTC. > > Or a flaw of the universe. For some reason the earth refuses to rotate > uniformly. > It's a fundamental flaw of UTC. There are other ways to implement something akin to UTC
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 1/15/19 9:14 PM, jimlux wrote: > Ground processing only - for user convenience I see somebody else mentioned SPICE, which I assumed you knew all about. >> >> Or more simply, how accurate are the satellite's GPS week and >> millisecond values? > > Time is set by the messages from the OEM-6xx series GPS receiver and > its 1pps - It's GPS time, not UTC. > > if there's a GPS outage, time propagates forward on the onboard > oscillator which isn't great. When GPS resumes, time jumps. Copacetic. Just like holdover specs on rackmount units. >>> We want to synchronize an event on the ground with an event on the >>> spacecraft, so, given that someone on the ground is going to do >>> something on 21 January 2019 at 12:34:56Z, we need to be able to >>> command the spacecraft to do that at the corresponding Week:Second. >>> >>> Right now, they use a website >>> >>> For example: https://www.labsat.co.uk/index.php/en/gps-time-calculator >>> >>> which, by the way, doesn't use leap seconds, so it's off by some 18 >>> seconds >>> >> Do they ignore the 18 seconds? What precision is required for this >> application? > > I think the folks at the labsat website just coded a naive > implementation. > > > For this application, <1 second uncertainty is needed (technically, > one can set events to occur to within 1 microsecond, but that's > counted down from the 1pps using the internal clock, with the > "correct" 1pps done by knowing it's week:second) Ok, so you set a trigger onboard and want to synchronize a state change at the ground station to some tolerance significantly under a second? If those are PC clocks their native imprecision will presumably leave a significant likelihood of ending up in the wrong second whatever the software does, but you should be able to minimize this relative to current practice. Do you have an example use case for what the operators might be doing that requires synchronization? How bad is it to be off by one or more seconds at one end or the other? Is this something like resetting both ends of a telemetry channel to different frequency or bit rate or some such? > And that's exactly how we can deal with it - it's perfectly reasonable > to say "folks, there's a leap second a'coming, run for the hills til > it gets here" Or even, "there might be a leap second, take a coffee break" since these can only happen twice a year. Or lockout such commands for the first and last UTC hour of each month if you want to be really sure without having to keep track of the file. This assumes your NTP installation will handle leap seconds correctly. We rely on Meinberg to get it right for us via both NTP and IRIG. (Same would apply to PTP.) > > For spacecraft flying via JPL ops, we work in TAI (or also, MET - > Mission Elapsed Time or SCLK - spacecraft clock), since all the nav is > done that way.. we also have to deal with relativistic effects and > light time - the whole "what do you do with time" on that scale has > seen 40 years of development. Right. The presumption is that JPL will get it right :-) Mostly we're interested in hearing about your cool systems. >> This conversion is so simple it seems unnecessary to rely on somebody >> else's library. What you might need is to arrange for a local, reliable, >> vetted copy of the leap seconds table. So perhaps folks here could >> comment on their best practices for retrieving same in an operational >> environment? > > or just set up an operational procedure that defines some file or > something under configuration control that has the "GPS time" and the > "Week:seconds" for the same time that is "after the last leap second". I think this is operationally equivalent. There is an operational advantage in leveraging the standard file. ROb ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Yo Warner! On Tue, 15 Jan 2019 17:45:32 -0700 Warner Losh wrote: > > > Except that's not a leap second. struct tm doesn't do leap seconds > > > really. It kinda sorta does, but it kinda sorts doesn't. > > > > Well, it does in a modern Linux kernel. Otherwise NTPD would fail > > on the leap second, and that only happens every year so. :-) > > > > You are wrong. The kernel doesn't use struct tm. I never said the kernel uses struct tm. I said you can query the kernel for the time in a struct tm. > The kernel does this > by ticking either in TAI or by incrementing a TAI offset. ntpd uses a > different interface that exposes these things to it so it can > broadcast the leap second indicators correctly. Yup. > > > First, when you convert back and forth to time_t, you lose leap > > > second information. It's impossible to convert back and forth > > > from at least the UTC time zone. There's some 'right' time zones, > > > but they aren't strictly speaking POSIX compliant because it uses > > > the wrong math. Non-conforming, but damned useful and not wrong. > > > > Which is why you can't use time_t in UTC, but time_t in TAI > > can work, as well as struct tm, usually. Lot's of latent bugs... > > > > No. time_t is not TAI or GPS. It is UTC. Adjusted UTC. No, time_t is just a way to count seconds. See man difftime() for a use of time_t that is unrelated to any particular epoch. > It is the > number of SI seconds since 1970, minus all the positive leap seconds, > plus all the negative ones. Ciration please? I already cited the Linux doc that says otherwise. > > If you squint at it, GPS time (gps weeks, time of week) is just > > another notation very similar to time. And we know that works. > > > > GPS time has no leap seconds. It's a monotonic count since 1980. Correct. > > Since future leap seconds are unknown, and unknowable, there is no > > possible valid way to do this on the time scale of years. > > > > Correct. This is the fundamental flaw of UTC. Or a flaw of the universe. For some reason the earth refuses to rotate uniformly. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can’t measure it, you can’t improve it." - Lord Kelvin pgpO5SYiDWPC3.pgp Description: OpenPGP digital signature ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
JPL has a nice library for computing spacecraft location and pointing, SPICE , with interfaces in Fortran, C, IDL, Matlab, while others have created interfaces in other languages. Relevant here is the extensive well-tested time conversions . It's important to keep the leap second kernel (LSK) updated (as well as SPK, PCK kernel files). Although not mentioned explicitly in SPICE, GPS time lags TAI by 19s so conversion is straight forward. For IDL and IDL users (and some other languages), the Common Data Format (CDF) time conversion routines for CDF_TIME_TT2000 can be used for converting between TT and UTC, and GPS is an easy conversion. Again requires keeping leap second table updated. A Python library is at . ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue 2019-01-15T15:18:00-0800 jimlux hath writ: > It would be much easier for the operators if they could just *enter* the UTC > time, and have the software calculate the GPS Week:millisecond (or vice > versa) The 1970 CCIR decision to begin having leap seconds was made without any technical docuements about how to implement them. At a public meeting after the CCIR decision and prior to 1972 one of the delegates to the ongoing CCIR working group meetings was asked if he could talk about the details "unless that is sub judice". In the context that was a jab from someone who was not happy with the decision for leap seconds. But everyone was going to have to be using the new broadcast time scheme, so it was not funny then to be joking about technical details of the system being secret. That became even less funny over the decades. More disturbing were the contributions to the 1972 CCDS meeting where one of the technical delegates to the CCIR process wrote It is clear that the recommendation of the CCIR concering the UTC system is limited to "standard frequency and time-signal transmissions" in the sense of Study Group 7 of the CCIR. This recommendation does not concern other programs and other methods of broadcasting time, such as radio or television announcments of the hour, hourly services for maritime or air navigation, public clocks, etc. and Again, UTC is a way of providing AT and UT in the same broadcast. The CCIR feels responsible for transmitting AT and UT to users in a reliable fashion and with appropriate technical means. It does not matter to the CCIR that users employ AT, UT, or UTC for their particular problems. The responsibility of the CCIR ends with the transmission of UTC by the specified stations. Another delegate both to the CCIR and that CCDS meeting wrote We should limit ourselves to discussing this issue with people who are well aware of the problems involved. Those are basically the folks who made the CCIR decision saying Many folks don't have to use this, especially us. and Blame the victim if they don't use the right thing. and Let's not talk about this. So the result was that the technical folks did not talk about leap seconds. Nobody was ever tasked with setting up a scheme for transmitting information about leap seconds that was more automated and reliable than the BIH giving letters to the post office to send to various national time service agencies. But before the technical folks washed their hands they arranged for various international assemblies to produce statements approving of the leap seconds. Subsequently the bureaucratic folks took those statements to other national and international bodies and achieved approval for UTC with leap seconds in arenas where the original technical folks had admitted they might not be the best strategy. At this point in history it is unclear who of these technical folks believed that the CCIR decision had limited scope and who were being shrewdly tacit about their ongoing goals for international approval. The only internationally official clue that UTC might not be the best tool was CCIR recommendation 485 https://www.itu.int/rec/R-REC-TF.485-2-199006-W/en which allowed that TAI might be preferable to UTC for some purposes. This rec was withdrawn in 1997. That was just before the "leap seconds must die" process began in earnest in 1999. Ironically at the 14th CCTF meeting in 1999 the head of the BIPM opined that anyone who did not like leap seconds should use TAI as given in rec 485. This betrays that not even the head of BIPM, who define TAI, was aware that the ITU had withdrawn that rec two years earlier. As Captain said in Cool Hand Luke Whut we've got heyah is failya tuh c'municate. https://www.youtube.com/watch?v=V2f-MZ2HRHQ -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 1/15/19 6:34 PM, Rob Seaman wrote: Hi Jim, The use case is this - I have a satellite in orbit which does everything internally in terms of GPS week and millisecond of week. Folks on the ground work in UTC, since that's what the plethora of PCs are using. Just to verify. The satellite does use GPS as its reference clock, not some manual upload (in a similar way as you describe) from ground stations? The routines you're talking about aren't (directly) affecting the state of the clock on the spacecraft? Ground processing only - for user convenience Or more simply, how accurate are the satellite's GPS week and millisecond values? Time is set by the messages from the OEM-6xx series GPS receiver and its 1pps - It's GPS time, not UTC. if there's a GPS outage, time propagates forward on the onboard oscillator which isn't great. When GPS resumes, time jumps. We want to synchronize an event on the ground with an event on the spacecraft, so, given that someone on the ground is going to do something on 21 January 2019 at 12:34:56Z, we need to be able to command the spacecraft to do that at the corresponding Week:Second. Right now, they use a website For example: https://www.labsat.co.uk/index.php/en/gps-time-calculator which, by the way, doesn't use leap seconds, so it's off by some 18 seconds Do they ignore the 18 seconds? What precision is required for this application? I think the folks at the labsat website just coded a naive implementation. For this application, <1 second uncertainty is needed (technically, one can set events to occur to within 1 microsecond, but that's counted down from the 1pps using the internal clock, with the "correct" 1pps done by knowing it's week:second) Sometimes cross checked for consistency by, say, the page at http://leapsecond.com/java/gpsclock.htm It would be much easier for the operators if they could just *enter* the UTC time, and have the software calculate the GPS Week:millisecond (or vice versa) Leap Seconds, should they occur, would be handled by "don't do anything around or across the leap second". Or could you simply forbid activities during a few hours of June 30 / December 31? If the operators are at JPL this would be late afternoon on whatever day of the week. That is, just assume there's always a leap second at each opportunity. As it happens the ops aren't at JPL for this spacecraft. And that's exactly how we can deal with it - it's perfectly reasonable to say "folks, there's a leap second a'coming, run for the hills til it gets here" For spacecraft flying via JPL ops, we work in TAI (or also, MET - Mission Elapsed Time or SCLK - spacecraft clock), since all the nav is done that way.. we also have to deal with relativistic effects and light time - the whole "what do you do with time" on that scale has seen 40 years of development. And, we could rebaseline the conversion i.e. the calculation would be SecondsAfterBaseDate = Seconds(UserEnteredDate) - Seconds(BaseDate) DesiredTime = Seconds(6 Jan 1980) + SecondsAfterBaseDate GPSWeek = Floor(DesiredTime/(86400*7)) GPSMilliseconds = (DesiredTime - GPSWeek*86400*7) * 1000 Which works quite nicely as long as the interval between BaseDate and UserEnteredDate does not include a leap second. This conversion is so simple it seems unnecessary to rely on somebody else's library. What you might need is to arrange for a local, reliable, vetted copy of the leap seconds table. So perhaps folks here could comment on their best practices for retrieving same in an operational environment? or just set up an operational procedure that defines some file or something under configuration control that has the "GPS time" and the "Week:seconds" for the same time that is "after the last leap second". Note that if your use cases include retroactive requirements then even if leap seconds cease you will need to maintain the leap seconds table (perhaps compiled into the source through some date). (For leap second warriors, redefining UTC makes it more complicated, not less, when all use cases are considered.) Not really - for those kinds of use cases there's already issues and solutions with reconciling local observations of clocks against outside time scales. the "don't operate during the leap" and "keep track" work just fine. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Hi Jim, > The use case is this - I have a satellite in orbit which does > everything internally in terms of GPS week and millisecond of week. > Folks on the ground work in UTC, since that's what the plethora of PCs > are using. > Just to verify. The satellite does use GPS as its reference clock, not some manual upload (in a similar way as you describe) from ground stations? The routines you're talking about aren't (directly) affecting the state of the clock on the spacecraft? Or more simply, how accurate are the satellite's GPS week and millisecond values? > We want to synchronize an event on the ground with an event on the > spacecraft, so, given that someone on the ground is going to do > something on 21 January 2019 at 12:34:56Z, we need to be able to > command the spacecraft to do that at the corresponding Week:Second. > > Right now, they use a website > > For example: https://www.labsat.co.uk/index.php/en/gps-time-calculator > > which, by the way, doesn't use leap seconds, so it's off by some 18 > seconds > Do they ignore the 18 seconds? What precision is required for this application? > Sometimes cross checked for consistency by, say, the page at > > http://leapsecond.com/java/gpsclock.htm > > It would be much easier for the operators if they could just *enter* > the UTC time, and have the software calculate the GPS Week:millisecond > (or vice versa) > > Leap Seconds, should they occur, would be handled by "don't do > anything around or across the leap second". > Or could you simply forbid activities during a few hours of June 30 / December 31? If the operators are at JPL this would be late afternoon on whatever day of the week. That is, just assume there's always a leap second at each opportunity. > And, we could rebaseline the conversion > > i.e. the calculation would be > > SecondsAfterBaseDate = Seconds(UserEnteredDate) - Seconds(BaseDate) > > DesiredTime = Seconds(6 Jan 1980) + SecondsAfterBaseDate > > GPSWeek = Floor(DesiredTime/(86400*7)) > > GPSMilliseconds = (DesiredTime - GPSWeek*86400*7) * 1000 > > Which works quite nicely as long as the interval between BaseDate and > UserEnteredDate does not include a leap second. > This conversion is so simple it seems unnecessary to rely on somebody else's library. What you might need is to arrange for a local, reliable, vetted copy of the leap seconds table. So perhaps folks here could comment on their best practices for retrieving same in an operational environment? Note that if your use cases include retroactive requirements then even if leap seconds cease you will need to maintain the leap seconds table (perhaps compiled into the source through some date). (For leap second warriors, redefining UTC makes it more complicated, not less, when all use cases are considered.) On the other hand, if your "plethora of PCs" keep good time (via NTP or SNTP), and you don't care to better than a second (which your current 18s offset suggests might be the case), and there are no after-the-fact use cases, do you need anything other than a procedural rule about avoiding such activities at the end of June and December? That is, put your effort into making your NTP installation as reliable as possible. Rob Seaman, University of Arizona ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue, Jan 15, 2019 at 1:50 PM Gary E. Miller wrote: > Yo Warner! > > On Tue, 15 Jan 2019 12:43:46 -0700 > Warner Losh wrote: > > > Except that's not a leap second. struct tm doesn't do leap seconds > > really. It kinda sorta does, but it kinda sorts doesn't. > > Well, it does in a modern Linux kernel. Otherwise NTPD would fail > on the leap second, and that only happens every year so. :-) > You are wrong. The kernel doesn't use struct tm. The kernel does this by ticking either in TAI or by incrementing a TAI offset. ntpd uses a different interface that exposes these things to it so it can broadcast the leap second indicators correctly. > > First, when you convert back and forth to time_t, you lose leap second > > information. It's impossible to convert back and forth from at least > > the UTC time zone. There's some 'right' time zones, but they aren't > > strictly speaking POSIX compliant because it uses the wrong math. > > Non-conforming, but damned useful and not wrong. > > Which is why you can't use time_t in UTC, but time_t in TAI > can work, as well as struct tm, usually. Lot's of latent bugs... > No. time_t is not TAI or GPS. It is UTC. Adjusted UTC. It is the number of SI seconds since 1970, minus all the positive leap seconds, plus all the negative ones. Now, you can use that type bogusly to hold any count from any epoch with any rules you like, but such use is not POSIX conforming. Many people abuse time_t to store TAI, it's true, but that's not its definition, and you'll wind up confusing things because of the 40 second offset between the two. > If you squint at it, GPS time (gps weeks, time of week) is just another > notation very similar to time. And we know that works. > GPS time has no leap seconds. It's a monotonic count since 1980. > > Second, many of the tm_ fields can create a time that's N 's > > in the future by adding N to that structure and converting. So tm_sec > > == 60 could be a leap second, or it could be the second after tm_sec > > == 59. > > Since future leap seconds are unknown, and unknowable, there is no > possible valid way to do this on the time scale of years. > Correct. This is the fundamental flaw of UTC. > > > But gpsd and ntpd try real hard to be correct. Sadly, with leap > > > smear, there is no consensus on what is 'correct'. > > > > > > > Both leap seconds and leap second smear must die. Others have > > different opinions, some strongly held. Not all the contradictory > > opinions are necessarily wrong. > > Good luck with that. > Yea. Having done timing systems for a decade now a decade ago, I always felt that was the best solution. The BIH guys broke time by trying to fix it. :( Warner ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Yo jimlux! On Tue, 15 Jan 2019 15:18:00 -0800 jimlux wrote: > It would be much easier for the operators if they could just *enter* > the UTC time, and have the software calculate the GPS > Week:millisecond (or vice versa) I've been wanting that CLI tool for a while. I'll add it to the gpsd TODO list. gpsd already has gpsd_gpstime_resolve() to go from GPS weeks and tow to UTC, the reverse would be easy. Then a command line tool that used that would allow for regression testing. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can’t measure it, you can’t improve it." - Lord Kelvin pgpaJzNJL5dc2.pgp Description: OpenPGP digital signature ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On 1/15/19 11:41 AM, Rob Seaman wrote: Isn't this covered (for instance) in section 20.3.3.5.2.4(b) of IS-GPS-200J (https://www.gps.gov/technical/icwg/IS-GPS-200J.pdf)? The various libraries and operating systems listed deal with none of the reference time scales (GPS, UTC, or TAI, for that matter) completely and correctly. There's no obvious connection here with astronomical use cases. We rely on our reference clocks' vendor to have implemented IS-GPS-200J and relevant standards correctly. These devices operate off single board linux cards, but I presume they wrote the critical code themselves to handle leap seconds, but also other precision timekeeping issues not entertained by POSIX and other mass market software. Our clocks handle time scales other than UTC, but since our customer (NASA PDCO) specifies UTC, that's what we deliver. The bread-and-butter leap second discussions on this list seem irrelevant to simply meeting an engineering requirement to adhere to whatever standard. If POSIX, python, excel, whatever don't provide canned tools to convert seamlessly between GPS and UTC, or to handle format conversions from week:ms to sexagesimal, such tools must be implemented outside those languages/libraries. There are a vast number of other science-related data engineering details that POSIX (for example) also does not implement. Indeed - but before leaping[sic] into coding it up, I thought I'd ask around You also asked.. > /I'll probably test it for the cases I'm interested in (Ruby, Python, > Excel, Matlab, Octave), but if someone else has already done it, then > I've got something to cross check against./ I would add MySQL/MariaDB and PostgreSQL, as well as TCL and C/C++ to that list. If JPL were to support libraries implementing some useful subset of timekeeping utilities under a smorgasbord of such languages, members of this list would undoubtedly be delighted to cross-check the results and put them to good use (via GitHub or what have you). --- I don't know that JPL is interested in a generalized library collection. It would be useful.. a sort of "cross platform" collection.. At JPL we tend to just solve the problem at hand, and release just that software. --- That said, it isn't clear what the original use case involves. Something about spacecraft using GPS week numbers and ground systems using UTC? But what do you need to do exactly? And is this onboard or on the ground? --- The use case is this - I have a satellite in orbit which does everything internally in terms of GPS week and millisecond of week. Folks on the ground work in UTC, since that's what the plethora of PCs are using. We want to synchronize an event on the ground with an event on the spacecraft, so, given that someone on the ground is going to do something on 21 January 2019 at 12:34:56Z, we need to be able to command the spacecraft to do that at the corresponding Week:Second. Right now, they use a website For example: https://www.labsat.co.uk/index.php/en/gps-time-calculator which, by the way, doesn't use leap seconds, so it's off by some 18 seconds Sometimes cross checked for consistency by, say, the page at http://leapsecond.com/java/gpsclock.htm It would be much easier for the operators if they could just *enter* the UTC time, and have the software calculate the GPS Week:millisecond (or vice versa) Leap Seconds, should they occur, would be handled by "don't do anything around or across the leap second". And, we could rebaseline the conversion i.e. the calculation would be SecondsAfterBaseDate = Seconds(UserEnteredDate) - Seconds(BaseDate) DesiredTime = Seconds(6 Jan 1980) + SecondsAfterBaseDate GPSWeek = Floor(DesiredTime/(86400*7)) GPSMilliseconds = (DesiredTime - GPSWeek*86400*7) * 1000 Which works quite nicely as long as the interval between BaseDate and UserEnteredDate does not include a leap second. ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue 2019-01-15T10:50:10-0800 Tom Van Baak hath writ: > Nope. All minutes have 60 seconds in Excel. And in Windows. And > in Unix/POSIX. Really any computer system that uses a fixed "epoch" > and a ticking "counter" is ruined by leap seconds. The systems differ > in their epoch's, they all differ in their counters, they can all be > set to UTC, except they all fail when there's a positive or negative > leap second. Nowhere did the folks who were there at the inception of leap seconds say so explicitly, but from reading how things worked over the history of the time services and what the folks paid to provide those services wrote, I am reasonably sure that they also believed that all minutes consisted of 60 seconds, and all days consisted of 86400 seconds. I believe that the concept in their heads was that occasionally there are two calendar days which are not adjacent by one SI second. Or, if the earth rotation were to speed up enough, there might be two calendar days which overlap by one SI second. The notation 23:59:60 is merely a tag for conveniently indicating when a second was inserted between two calendar days. The clock is disconnected from the calendar. Understandably this notion should trigger objections that it is technically barren. At this point I would answer "Yes, and they knew that." They were being paid by their governments to provide the legal and operational time of their nations. Their job had always been to tell the people of their country how to set their clocks. So when a senator or cabinet official asked "Can you tell me what time it is?" they had learned over the course of their careers that the answer was not to wander off into a long technical discussion about how sausage was made, but to smile and say "Yes." Similarly for the 1970 CCIR decision about leap seconds: when facing a chamber full of folks at an international assembly that had the power to dictate how every member nation should set their clocks in the same way that was consistent with the laws of all those nations, the answer was to smile and say "Yes, everyone has agreed that this is the best way to do it." Their job was not to say that the technical folks who made that decision had already decided to disregard that in the systems that they controlled, broadcast navigational signals based on TAI, and make almanacs based on UT (not UTC). -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue 2019-01-15T13:10:54-0800 Gary E. Miller hath writ: > > What is the epoch that was used for TAI? > "TAI in this form was synchronised with Universal Time at the beginning > of 1958," That is conceptually the case, but even as a concept it deserves explanation about how it was that the USNO A.1 atomic time scale which was started with that same epoch differed from the BIH atomic time scale by 0.035 seconds. Furthermore, the epoch of A.1 was 1958-01-01T00:00:00 UT2 whereas the epoch of the BIH atomic time scale was 1958-01-01T20:00:00 UT2, but this does not explain the difference between the two time scales. That 1958 epoch is not the basis for the current incarnation of TAI. Tonight I will scan the pages of Bulletin Horaire with the answer. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Yo Steve! On Tue, 15 Jan 2019 12:52:43 -0800 Steve Allen wrote: > On Tue 2019-01-15T12:34:11-0800 Gary E. Miller hath writ: > > Yes, and no. time_t is just seconds since an epoch. Which epoch > > is not well defined. The epoch may well be anything. See "man > > difftime". > > That evokes a challenge for all time nuts that I can make based on > reading Bulletin Horaire. > > What is the epoch that was used for TAI? According to wikipedia: https://en.wikipedia.org/wiki/International_Atomic_Time "As of 31 December 2016, when another leap second was added,[3] TAI is exactly 37 seconds ahead of UTC." > I expect that many can give an answer, and that nobody can give the > correct answer. "TAI in this form was synchronised with Universal Time at the beginning of 1958," > > The best bet is to ask the kernel for the current TAI time, and work > > with that. Use the leap seconds file to convert TAI to UTC in your > > code. Or just use TAI for everything. > > The trick is to find a source that will set a POSIX system to TAI, Easy: NTPD. > and > then to avoid the gotchas that happen when such a system interacts > with other POSIX systems. So don't let it, except by NTPD. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can’t measure it, you can’t improve it." - Lord Kelvin pgpzDR6oI7G_A.pgp Description: OpenPGP digital signature ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue 2019-01-15T12:34:11-0800 Gary E. Miller hath writ: > Yes, and no. time_t is just seconds since an epoch. Which epoch > is not well defined. The epoch may well be anything. See "man difftime". That evokes a challenge for all time nuts that I can make based on reading Bulletin Horaire. What is the epoch that was used for TAI? I expect that many can give an answer, and that nobody can give the correct answer. > The best bet is to ask the kernel for the current TAI time, and work > with that. Use the leap seconds file to convert TAI to UTC in your > code. Or just use TAI for everything. The trick is to find a source that will set a POSIX system to TAI, and then to avoid the gotchas that happen when such a system interacts with other POSIX systems. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Yo Warner! On Tue, 15 Jan 2019 12:43:46 -0700 Warner Losh wrote: > Except that's not a leap second. struct tm doesn't do leap seconds > really. It kinda sorta does, but it kinda sorts doesn't. Well, it does in a modern Linux kernel. Otherwise NTPD would fail on the leap second, and that only happens every year so. :-) > First, when you convert back and forth to time_t, you lose leap second > information. It's impossible to convert back and forth from at least > the UTC time zone. There's some 'right' time zones, but they aren't > strictly speaking POSIX compliant because it uses the wrong math. > Non-conforming, but damned useful and not wrong. Which is why you can't use time_t in UTC, but time_t in TAI can work, as well as struct tm, usually. Lot's of latent bugs... If you squint at it, GPS time (gps weeks, time of week) is just another notation very similar to time. And we know that works. > Second, many of the tm_ fields can create a time that's N 's > in the future by adding N to that structure and converting. So tm_sec > == 60 could be a leap second, or it could be the second after tm_sec > == 59. Since future leap seconds are unknown, and unknowable, there is no possible valid way to do this on the time scale of years. > > But gpsd and ntpd try real hard to be correct. Sadly, with leap > > smear, there is no consensus on what is 'correct'. > > > > Both leap seconds and leap second smear must die. Others have > different opinions, some strongly held. Not all the contradictory > opinions are necessarily wrong. Good luck with that. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can’t measure it, you can’t improve it." - Lord Kelvin pgpTX8DP2zyyK.pgp Description: OpenPGP digital signature ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue 2019-01-15T11:49:00-0800 Tom Van Baak hath writ: > Still, I bet more astronomers use Python than Excel to point > telescopes. How do you handle the lack of leap second support in > Python? I can't say that anyone uses Python to point a telescope, but it does not matter if they do. See preprint 678 from the 2011 Future of UTC meeting proceedings http://hanksville.org/futureofutc/2011/preprints/index.html In short: 19th century telescopes point by moving them manually. 20th century telescopes are built like battleships and raw pointing is only a bit more accurate than steering a battleship. 21st century telescopes are robotic, but they still have flexure and slop that requires them to update their pointing models regularly, and those slop parameters easily soak up one second. The APF telescope at Lick is robotic. Its pointing system produces a time mismatch alarm whenever there is a leap second and requires a reboot. At the most recent summer leap second we were able to see a star before and after, and there was a jump in the centering. The software chose a different offset in the pointing for that night. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Yo Steve! On Tue, 15 Jan 2019 11:46:02 -0800 Steve Allen wrote: > I would love to find pointers to such strategies and code. In the gpsd code look in the file: gpsd/timebase.c In the NTPsec code look in the files: ntpd/ntp_leapsec.c and ntptime/ntptime.c RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can’t measure it, you can’t improve it." - Lord Kelvin pgpi1hHH2P7J1.pgp Description: OpenPGP digital signature ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Yo Tom! On Tue, 15 Jan 2019 11:40:07 -0800 "Tom Van Baak" wrote: > But the underlying time_t cannot represent leap seconds, can it? Yes, and no. time_t is just seconds since an epoch. Which epoch is not well defined. The epoch may well be anything. See "man difftime". time_t dates to the first edition of "Unix Programmer's Manual, November 3, 1971". No timezone or other basis is given for time_t. They just said "system time". The first leap second was in 1972! time_t never caught up. https://www.bell-labs.com/usr/dmr/www/1stEdman.html In 1973, just after the third edition, time_t was pegged to GMT. According to wikipedia, GMT is mean solar time, not UTC. The seconds are not the same length, and no leap seconds: https://en.wikipedia.org/wiki/Greenwich_Mean_Time Some long time after that the doc changed from GMT to UTC, but the code did not. time() returns a time_t which pretends to be UTC, but ignores leap seconds. From a current "man time": "This value is not the same as the actual number of seconds between the time and the Epoch, because of leap seconds and because system clocks are not required to be synchronized to a standard reference." So if you need "real" UTC, POSIX is no help. And nothing is a help for future dates since the future leapseconds are unknown. The only safe way to compute times (to the second) in the past it to do it yourself using the leapseconds file. > So how does that gmtime hackery work? NTPD, or PTP, or something, tells your kernel what the 'real' current UTC is, including the current leap seconds. gmtime() just returns that. NTPD can get the current and past leap second info from the leapseconds file, from another NTPD, or from a GPS. > Also, presumably Python is based on POSIX? Does it use time_t or > gmtime? Why did Jim's quick Python experiment fail? Out of my area of understanding. But I'll guess that like almost every other time keeping system it is just ignoring leap seconds. Double counting the leap second itself as two 59's in a row. Pretending to be UTC while actually being an approximation of GMT. And, yes, during the leap second, all sorts of implementation dependent, unknown, and unknowable, stuff is happening. The best bet is to ask the kernel for the current TAI time, and work with that. Use the leap seconds file to convert TAI to UTC in your code. Or just use TAI for everything. To add more weirdness, Google, and its NTPD servers, don't use UTC because they use leap smearing. And the upcoming GPS week rollover... RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can’t measure it, you can’t improve it." - Lord Kelvin pgpGNEZTAztr4.pgp Description: OpenPGP digital signature ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue 2019-01-15T12:52:18-0700 Warner Losh hath writ: > This suggests strongly that those models of time are fatally flawed and > should be revisited. I am probably the only person who has ever read all of Bulletin Horaire. https://www.ucolick.org/~sla/leapsecs/bhissues.html I can say that with certainty for the copies in the Lick library because I had to apply xacto knife to separate some of the pages which were not cut at the time of printing and never opened before I did. Over the lifetime of the BIH the staff were prone to be chatty and include transcriptions of the meetings which many times resulted in directions that the BIH must do more work in a different fashion. Those transcriptions point to the dramatis personae at those and other meetings, and to other documents. I can say unequivocally that the people who instituted leap seconds in radio broadcast time signals knew that the idea was fatally flawed. They also knew that they did not have the legal authority *not* to include leap seconds in the radio broadcast time signals, and they were not happy about the situation. They intentionally did not use UTC in their ongoing technical work. They intentionally avoided discussing the technical problems in public, redacted the transcripts of meetings that showed dissension, and arranged forced votes to produce statements indicating that various bodies agreed with the notion of leap seconds. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue, Jan 15, 2019 at 12:41 PM Rob Seaman wrote: > Isn't this covered (for instance) in section 20.3.3.5.2.4(b) of > IS-GPS-200J (https://www.gps.gov/technical/icwg/IS-GPS-200J.pdf)? The > various libraries and operating systems listed deal with none of the > reference time scales (GPS, UTC, or TAI, for that matter) completely and > correctly. > > This suggests strongly that those models of time are fatally flawed and should be revisited. > There's no obvious connection here with astronomical use cases. We rely on > our reference clocks' vendor to have implemented IS-GPS-200J and relevant > standards correctly. These devices operate off single board linux cards, > but I presume they wrote the critical code themselves to handle leap > seconds, but also other precision timekeeping issues not entertained by > POSIX and other mass market software. Our clocks handle time scales other > than UTC, but since our customer (NASA PDCO) specifies UTC, that's what we > deliver. > Spinning rocks are hard, eh? the non-uniform nature of UTC is a big reason it's hard to implement correctly in software (especially when there's code size or speed constraints). It's impossible to implement without tables that must constantly be updated. While this is natural and acceptable in many environments, it presents operational difficulties for others. The ITU standard that tries to keep the spinning rock and atomic time in sync is flawed in that it doesn't adequately accommodate all situations. POSIX has similar sorts of flaws, but the details differ. > The bread-and-butter leap second discussions on this list seem irrelevant > to simply meeting an engineering requirement to adhere to whatever > standard. If POSIX, python, excel, whatever don't provide canned tools to > convert seamlessly between GPS and UTC, or to handle format conversions > from week:ms to sexagesimal, such tools must be implemented outside those > languages/libraries. There are a vast number of other science-related data > engineering details that POSIX (for example) also does not implement. > Yes. Despite the efforts of the POSIX committee to make it easy to implement time, those efforts have resulted in a fatally flawed standard that leads to the "1 second errors are fine, don't bother me" attitude that pervades many adjacent APIs to the POSIX APIs. It was the number one frustration that I had when I had to deal with vendors that didn't get the leap seconds pedantically correct and didn't give a @!##@ that they didn't because there was no financial penalty for not doing so. It was maddening. Warner > Rob Seaman, Lunar and Planetary Laboratory > > -- > On 1/15/19 11:50 AM, Tom Van Baak wrote: > > Jim -- I'm replying via the LEAPSECS list because we love leap second > questions here. > > List -- Jim is a long-time member of time-nuts, works at JPL, and does > wonderful stuff. > > From Jim Lux: > > I'm working with a variety of things which work in UTC or GPS > week/millisecond, so we're doing a lot of conversion back and forth. > (the spacecraft puts out time in week:millisecond, all the ground > systems processing is in UTC) > > The question comes up when converting back and forth, and whether > various libraries handle leap seconds correctly. > For now, I can use a hack of not computing back to 6 Jan 1980, but use > an epoch like 15 Dec 2018 (week 2031: 518,400.000 seconds) and hope > there's no leap second in the offing. > > Yes, that's one way. But that doesn't sound safe or correct to me. > > > For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + > timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it > comes out 0600) > > Similarly, does Excel's time formatting allow for some minutes having an > extra second, or does it just assume all minutes are 60 seconds. > > Nope. All minutes have 60 seconds in Excel. And in Windows. And in > Unix/POSIX. Really any computer system that uses a fixed "epoch" and a > ticking "counter" is ruined by leap seconds. The systems differ in their > epoch's, they all differ in their counters, they can all be set to UTC, > except they all fail when there's a positive or negative leap second. > > > I'll probably test it for the cases I'm interested in (Ruby, Python, > Excel, Matlab, Octave), but if someone else has already done it, then > I've got something to cross check against. > > This is a good question for LEAPSECS. I suspect those packages are well-known > here. > > > (python does NOT know about leap seconds) > > import datetime > > d = datetime.datetime(2016,12,31) > > dt = datetime.timedelta(hours=30) > > d > Out[4]: datetime.datetime(2016, 12, 31, 0, 0) > > dt > Out[5]: datetime.timedelta(1, 21600) > > d+dt > Out[6]: datetime.datetime(2017, 1, 1, 6, 0) > > Right, this is typical for almost any software developed anywhere. Leap > seconds are such a weird exception, almost no commercial software, web > frontend or backend, or mobile phones, or mobile apps deal with them >
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue, Jan 15, 2019 at 11:58 AM Gary E. Miller wrote:
> Yo Tom!
>
> On Tue, 15 Jan 2019 10:50:10 -0800
> "Tom Van Baak" wrote:
>
> > Nope. All minutes have 60 seconds in Excel. And in Windows. And in
> > Unix/POSIX.
>
> Not quite. Check out "man gmtime" for one way that POSIX represents time.
>
Yes and no. time_t doesn't have leap seconds. At all. Ever. So from that
perspective, POSIX can't do leap seconds. time_t is so engrained in the
other interfaces of POSIX, they can't have leap seconds either.
> Specifically:
>
>struct tm {
> [...]
>int tm_sec;/* Seconds (0-60) */
>
> gmtime() is perfectly capable of reporting 61 seconds in a minute.
>
> Failure to account for this causes mnay problems.
>
> What it does not do well is accounting for leap seconds that are in
> the past or future.
>
Except that's not a leap second. struct tm doesn't do leap seconds really.
It kinda sorta does, but it kinda sorts doesn't.
First, when you convert back and forth to time_t, you lose leap second
information. It's impossible to convert back and forth from at least the
UTC time zone. There's some 'right' time zones, but they aren't strictly
speaking POSIX compliant because it uses the wrong math. Non-conforming,
but damned useful and not wrong.
Second, many of the tm_ fields can create a time that's N 's in the
future by adding N to that structure and converting. So tm_sec == 60 could
be a leap second, or it could be the second after tm_sec == 59.
> > Right, this is typical for almost any software developed anywhere.
> > Leap seconds are such a weird exception, almost no commercial
> > software, web frontend or backend, or mobile phones, or mobile apps
> > deal with them correctly.
>
> But gpsd and ntpd try real hard to be correct. Sadly, with leap smear,
> there is no consensus on what is 'correct'.
>
Both leap seconds and leap second smear must die. Others have different
opinions, some strongly held. Not all the contradictory opinions are
necessarily wrong.
Warner
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
> If you set your POSIX system clock at TAI minus 10 seconds then > it keeps track of the number of seconds which have been counted > in the internationally approved radio broadcast time scales. So, wait. In order to use Python for any proper UTC processing you have to configure your PC to run on TAI? That sounds, sort of inconvenient if you're trying to do anything with precise time. It's the same as Excel. Still, I bet more astronomers use Python than Excel to point telescopes. How do you handle the lack of leap second support in Python? /tvb ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue 2019-01-15T12:41:27-0700 Rob Seaman hath writ: > The bread-and-butter leap second discussions on this list seem > irrelevant to simply meeting an engineering requirement to adhere to > whatever standard. If POSIX, python, excel, whatever don't provide > canned tools to convert seamlessly between GPS and UTC, or to handle > format conversions from week:ms to sexagesimal, such tools must be > implemented outside those languages/libraries. There are a vast number > of other science-related data engineering details that POSIX (for > example) also does not implement. IEEE 1588 version 2 (PTP) makes it clear that the intended time scale on the underlying hardware is TAI counted from some epoch. It must be the case that vendors selling such hardware have chosen solution strategies and written code to handle converting between civil time and system time. I would love to find pointers to such strategies and code. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue 2019-01-15T11:40:07-0800 Tom Van Baak hath writ: > Also, presumably Python is based on POSIX? Does it use time_t or > gmtime? Why did Jim's quick Python experiment fail? Guido said that the mainline of Python would not support leap seconds. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Isn't this covered (for instance) in section 20.3.3.5.2.4(b) of IS-GPS-200J (https://www.gps.gov/technical/icwg/IS-GPS-200J.pdf)? The various libraries and operating systems listed deal with none of the reference time scales (GPS, UTC, or TAI, for that matter) completely and correctly. There's no obvious connection here with astronomical use cases. We rely on our reference clocks' vendor to have implemented IS-GPS-200J and relevant standards correctly. These devices operate off single board linux cards, but I presume they wrote the critical code themselves to handle leap seconds, but also other precision timekeeping issues not entertained by POSIX and other mass market software. Our clocks handle time scales other than UTC, but since our customer (NASA PDCO) specifies UTC, that's what we deliver. The bread-and-butter leap second discussions on this list seem irrelevant to simply meeting an engineering requirement to adhere to whatever standard. If POSIX, python, excel, whatever don't provide canned tools to convert seamlessly between GPS and UTC, or to handle format conversions from week:ms to sexagesimal, such tools must be implemented outside those languages/libraries. There are a vast number of other science-related data engineering details that POSIX (for example) also does not implement. Rob Seaman, Lunar and Planetary Laboratory -- On 1/15/19 11:50 AM, Tom Van Baak wrote: > Jim -- I'm replying via the LEAPSECS list because we love leap second > questions here. > > List -- Jim is a long-time member of time-nuts, works at JPL, and does > wonderful stuff. > > From Jim Lux: >> I'm working with a variety of things which work in UTC or GPS >> week/millisecond, so we're doing a lot of conversion back and forth. >> (the spacecraft puts out time in week:millisecond, all the ground >> systems processing is in UTC) >> >> The question comes up when converting back and forth, and whether >> various libraries handle leap seconds correctly. >> For now, I can use a hack of not computing back to 6 Jan 1980, but use >> an epoch like 15 Dec 2018 (week 2031: 518,400.000 seconds) and hope >> there's no leap second in the offing. > Yes, that's one way. But that doesn't sound safe or correct to me. > >> For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + >> timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it >> comes out 0600) >> >> Similarly, does Excel's time formatting allow for some minutes having an >> extra second, or does it just assume all minutes are 60 seconds. > Nope. All minutes have 60 seconds in Excel. And in Windows. And in > Unix/POSIX. Really any computer system that uses a fixed "epoch" and a > ticking "counter" is ruined by leap seconds. The systems differ in their > epoch's, they all differ in their counters, they can all be set to UTC, > except they all fail when there's a positive or negative leap second. > >> I'll probably test it for the cases I'm interested in (Ruby, Python, >> Excel, Matlab, Octave), but if someone else has already done it, then >> I've got something to cross check against. > This is a good question for LEAPSECS. I suspect those packages are well-known > here. > >> (python does NOT know about leap seconds) >> >> import datetime >> >> d = datetime.datetime(2016,12,31) >> >> dt = datetime.timedelta(hours=30) >> >> d >> Out[4]: datetime.datetime(2016, 12, 31, 0, 0) >> >> dt >> Out[5]: datetime.timedelta(1, 21600) >> >> d+dt >> Out[6]: datetime.datetime(2017, 1, 1, 6, 0) > Right, this is typical for almost any software developed anywhere. Leap > seconds are such a weird exception, almost no commercial software, web > frontend or backend, or mobile phones, or mobile apps deal with them > correctly. > > I'm hoping a number of LEAPSECS members can chime in. There are astronomers > here who must be used to solving issues like this, in python or other tools, > so their input would be valuable. > > /tvb > > ___ > LEAPSECS mailing list > LEAPSECS@leapsecond.com > https://pairlist6.pair.net/mailman/listinfo/leapsecs ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
> Yo Tom!
>
> On Tue, 15 Jan 2019 10:50:10 -0800
> "Tom Van Baak" wrote:
>
> > Nope. All minutes have 60 seconds in Excel. And in Windows. And in
> > Unix/POSIX.
>
> Not quite. Check out "man gmtime" for one way that POSIX represents time.
>
> Specifically:
>
>struct tm {
> [...]
>int tm_sec;/* Seconds (0-60) */
>
> gmtime() is perfectly capable of reporting 61 seconds in a minute.
Hi Gary,
Nice to hear from you.
But the underlying time_t cannot represent leap seconds, can it? So how does
that gmtime hackery work?
Also, presumably Python is based on POSIX? Does it use time_t or gmtime? Why
did Jim's quick Python experiment fail?
/tvb
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
On Tue 2019-01-15T10:50:10-0800 Tom Van Baak hath writ: > Jim -- I'm replying via the LEAPSECS list because we love leap second > questions here. > > List -- Jim is a long-time member of time-nuts, works at JPL, and does > wonderful stuff. If you set your POSIX system clock at TAI minus 10 seconds then it keeps track of the number of seconds which have been counted in the internationally approved radio broadcast time scales. If you set your clock that way then you can use the "right" zones of the IANA tz distribution and get conversions between the internal system time and the civil time that has been specified in the international agreements. https://www.ucolick.org/~sla/leapsecs/right+gps.html Note that there are caveats to this scheme. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855 1156 High Street Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 https://www.ucolick.org/~sla/ Hgt +250 m ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Yo Tom!
On Tue, 15 Jan 2019 10:50:10 -0800
"Tom Van Baak" wrote:
> Nope. All minutes have 60 seconds in Excel. And in Windows. And in
> Unix/POSIX.
Not quite. Check out "man gmtime" for one way that POSIX represents time.
Specifically:
struct tm {
[...]
int tm_sec;/* Seconds (0-60) */
gmtime() is perfectly capable of reporting 61 seconds in a minute.
Failure to account for this causes mnay problems.
What it does not do well is accounting for leap seconds that are in
the past or future.
> Right, this is typical for almost any software developed anywhere.
> Leap seconds are such a weird exception, almost no commercial
> software, web frontend or backend, or mobile phones, or mobile apps
> deal with them correctly.
But gpsd and ntpd try real hard to be correct. Sadly, with leap smear,
there is no consensus on what is 'correct'.
RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin
pgpdXnu3Zyhrw.pgp
Description: OpenPGP digital signature
___
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs
Re: [LEAPSECS] leapseconds, converting between GPS time (week, second) and UTC
Jim -- I'm replying via the LEAPSECS list because we love leap second questions here. List -- Jim is a long-time member of time-nuts, works at JPL, and does wonderful stuff. >From Jim Lux: > I'm working with a variety of things which work in UTC or GPS > week/millisecond, so we're doing a lot of conversion back and forth. > (the spacecraft puts out time in week:millisecond, all the ground > systems processing is in UTC) > > The question comes up when converting back and forth, and whether > various libraries handle leap seconds correctly. > For now, I can use a hack of not computing back to 6 Jan 1980, but use > an epoch like 15 Dec 2018 (week 2031: 518,400.000 seconds) and hope > there's no leap second in the offing. Yes, that's one way. But that doesn't sound safe or correct to me. > For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + > timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it > comes out 0600) > > Similarly, does Excel's time formatting allow for some minutes having an > extra second, or does it just assume all minutes are 60 seconds. Nope. All minutes have 60 seconds in Excel. And in Windows. And in Unix/POSIX. Really any computer system that uses a fixed "epoch" and a ticking "counter" is ruined by leap seconds. The systems differ in their epoch's, they all differ in their counters, they can all be set to UTC, except they all fail when there's a positive or negative leap second. > I'll probably test it for the cases I'm interested in (Ruby, Python, > Excel, Matlab, Octave), but if someone else has already done it, then > I've got something to cross check against. This is a good question for LEAPSECS. I suspect those packages are well-known here. > (python does NOT know about leap seconds) > > import datetime > > d = datetime.datetime(2016,12,31) > > dt = datetime.timedelta(hours=30) > > d > Out[4]: datetime.datetime(2016, 12, 31, 0, 0) > > dt > Out[5]: datetime.timedelta(1, 21600) > > d+dt > Out[6]: datetime.datetime(2017, 1, 1, 6, 0) Right, this is typical for almost any software developed anywhere. Leap seconds are such a weird exception, almost no commercial software, web frontend or backend, or mobile phones, or mobile apps deal with them correctly. I'm hoping a number of LEAPSECS members can chime in. There are astronomers here who must be used to solving issues like this, in python or other tools, so their input would be valuable. /tvb ___ LEAPSECS mailing list LEAPSECS@leapsecond.com https://pairlist6.pair.net/mailman/listinfo/leapsecs
