Re: [time-nuts] GPS jumps of -13.7 us?
David, On 02/02/2016 04:23 PM, David J Taylor wrote: Appears to have caused a disruption to the UK digital radio distribution network: http://www.bbc.co.uk/news/technology-35463347 Thanks for that one. It is typical for installations like that to be affected. It's not the only broadcast network affected, I know of several. I doubt much more will be said beyond the press release in any time soon. Hopefully there will be a more detailed report, and if so, I would not expect it to surface until maybe a few months on the earliest, but 6-12 months is maybe a little more realistic. There is ways to make receivers more robust to this and a number of similar data-errors. It will be interesting to see what GPS vendors do to make their receivers more robust. It would be useful if there where a standard set of improvements, as this would allow "checkboxing" that help operators to get the right gear. Cheers, Magnus ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Appears to have caused a disruption to the UK digital radio distribution network: http://www.bbc.co.uk/news/technology-35463347 Cheers, David -- SatSignal Software - Quality software written to your requirements Web: http://www.satsignal.eu Email: david-tay...@blueyonder.co.uk Twitter: @gm8arv ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
NeT MonK wrote: > On Fri, Jan 29, 2016 at 3:29 PM, Chris Caudlewrote: > >> On Fri, January 29, 2016 1:32 am, NeT MonK wrote: >>> As a side effect of those glitch in the GPS matrix, the utc_valid_flag >> was >>> not anymore set in the stream of the primary master clock, just before my >>> secondary starts to become active (loss of primary stream) which leads to >>> linux server ptp slave to readjust of +36 seconds and jump backward -36 >>> seconds as far as the flags was coming back. >> >> The difference between UTC and TAI is 36 seconds. >> >> Are you running ptpd or linuxptp on your Linux servers? >> It sounds like the ptp agent running on your servers interpreted the lack >> of UTC valid flag to mean that the timestamps were now in TAI, so the >> server kernel applied the TAI to UTC offset to the time received via PTP. > > > Dear Chris, > > I run sfptpd which is a fork of linux ptpd daemon adapted to solarflare > network adapter. > I will have to fine tune it in order to be more resilient in such scenario. Hm, the task of the PTP slave software is to discipline the PC's *UTC* time, but this sounds like it even accepts the time from the PTP (grand)master if the master has *not* set utc_valid flag to 1, i.e. the master tells the slave it has no idea about the correct UTC time, but the slave accepts the uncorrected time anyway. Maybe in this case the slave software should rather generate an error and discard the PTP source, instead of accepting and using the pure TAI time, which causes the PC's system time to go wrong. > But i guess it's not very often the gps system as such incident. Of course this is not primarily related to GPS. I'd expect is could under some other confitions that the utc_valid flag is cleared. Please not with GPS and UTC correction it's somewhat similar. The GPS satellites also provide the UTC correction data set, but if you start a GPS receiver in cold boot mode, i.e. without any satellite data already saved in non-volatile memory, the GPS receiver is unable to convert GPS time to UTC, until it has received a UTC parameter set from the satellites. This can take up to 12 minutes since these parameters are sent repeatedly every 12 minuts. Without UTC correction parameters the receiver can be used for navigation, but not for timing. If a timing GPS receiver would declare itself "synchronized" in this state then the time which is output would be GPS time only, not UTC, and a time discipline software which accepted the time from the GPS in this state would also set the PC system time wrong by 17 seconds, since this is the current difference between UTC and GPS system time. Martin ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
On Mon, February 1, 2016 9:49 am, Martin Burnicki wrote: > Hm, the task of the PTP slave software is to discipline the PC's *UTC* > time, Typically, but I don't think there is a requirement to discipline the UTC system time. You could maintain a separate PTP time domain, or run the system clock as TAI (or GPS) and let the system software convert as needed. But of course flexibility offers the opportunity to interpret incorrectly... > but this sounds like it even accepts the time from the PTP > (grand)master if the master has *not* set utc_valid flag to 1, i.e. the > master tells the slave it has no idea about the correct UTC time, but > the slave accepts the uncorrected time anyway. I think it might be a slightly more subtle error. It appears that the master cleared the flag to indicate "this time is no longer correctly and fully synchronized to UTC time," no more and no less. It appears that the slave interpreted the lack of set flag to mean "this time is the correct TAI timestamp," which is obviously not the same thing as "this time is not fully synchronized to UTC." > Maybe in this case the slave software should rather generate an error > and discard the PTP source, instead of accepting and using the pure TAI > time, which causes the PC's system time to go wrong. Yes, some variation of setting an error flag would be more appropriate. It seems that a more reliable method would have a UTC_Valid flag, and a separate field to indicate that the time stamp should be interpreted as UTC, GPS, or TAI, or unsynchronized time (i.e. possibly syntonized but not set from the accepted epoch). In that case the UTC_valid flag should probably really be a "timestamp_valid" flag, or "timesource_synchronized" flag, some kind of indication that the source of the timestamp is thought to meet the standard implied by the source field. -- Chris Caudle ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Cisco as a boundary clock is susceptible to error. Ive seen under conditions of cpu-bound high traffic load (bgp event, high levels of broadcast) the ptp process has a lower interrupt priority on the scheduler and therefore prone to more jitter when it comes to acting as a master. As the Cisco devices do not accept a pps input one is better off using a server which does for greater master clock stability, or going to what device recieves your signal from GPS antennae. Dont expect too much from TAC on this btw. //r1ftrouter > On Jan 29, 2016, at 3:05 PM, NeT MonKwrote: > > On Fri, Jan 29, 2016 at 2:45 PM, Martin Burnicki < > martin.burni...@burnicki.net> wrote: > >> Hello Net Monk, >> ... >> I don't know the details of your PTP setup, but we (at Meinberg) have >> already had another customer some time ago where a Cisco switch acting >> as boundary clock didn't send UTC data reliably when the grandmaster was >> switched. >> >> So eventually you should contact Cisco and see if there is an update >> available for the switch where this is fixed. >> >> Martin >> >> >> > Hello Martin, > > Thank you for your reply and details, this is also what i already stated on > my incident report. > My ptp client applied the utc offset when the flag disappeared which lead > to this jump forward and backward. > > A call is already opened to Cisco and they are coming in coming weeks for > auditing the ptp infra. > > Regards. > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. > ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Le 29 janv. 2016 22:03, "Eric Scace"a écrit : > >I have an idle curiosity as to whether the erroneous UTC correction propagated into any high-speed trading platforms in the financial markets in a what that caused a disruption. I cannot speak for others but on our plateform, order passing tcp connections were closed by market servers when we were jumping backward -36seconds. This happened a hundred of time on the whole platform but didn't impacted much out daily business result. Regards. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Le 27 janv. 2016 18:04, "Martin Burnicki"a écrit : > > Magnus Danielson wrote: > > Hi, > > > > On 01/26/2016 11:46 PM, Martin Burnicki wrote: > >> Paul Boven wrote: > >>> Hi everyone, > >>> > >>> Has anyone else seen GPS time jump by -13.7 usec today? Good morning, I have subscribed to this list following this incident as far as it was the first of the few results of search in Google for three days ago. On my linux platform of hundreds serveurs, I had incident because of this, on servers which only use of ptp. I have two sources grand master clock one I control (a meinberg appliance) and act as passive and the second is just a Cisco relaying the ptp stream from the market. As a side effect of those glitch in the GPS matrix, the utc_valid_flag was not anymore set in the stream of the primary master clock, just before my secondary starts to become active (loss of primary stream) which leads to linux server ptp slave to readjust of +36 seconds and jump backward -36 seconds as far as the flags was coming back. Am I the only one who suffered the loss of the utc_valid_flag in ptp stream ? And what appliance would behave like this ? As far I can tell my secondary meinberg appliance aside a few oscillator recalibration message in the console log behaved very nicely. But I have no view of the primary master source and kind of hardware they use. Thank you in advance for your answer. -- Netmonk ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Hello Net Monk, NeT MonK wrote: > I have subscribed to this list following this incident as far as it was the > first of the few results of search in Google for three days ago. > > On my linux platform of hundreds serveurs, I had incident because of this, > on servers which only use of ptp. > I have two sources grand master clock one I control (a meinberg appliance) > and act as passive and the second is just a Cisco relaying the ptp stream > from the market. > > As a side effect of those glitch in the GPS matrix, the utc_valid_flag was > not anymore set in the stream of the primary master clock, just before my > secondary starts to become active (loss of primary stream) which leads to > linux server ptp slave to readjust of +36 seconds and jump backward -36 > seconds as far as the flags was coming back. > > Am I the only one who suffered the loss of the utc_valid_flag in ptp stream > ? And what appliance would behave like this ? As far I can tell my > secondary meinberg appliance aside a few oscillator recalibration message > in the console log behaved very nicely. > > But I have no view of the primary master source and kind of hardware they > use. The time stamps sent in the PTP messages on the network are usually TAI, not UTC. The PTP announce message contains a UTC offset field telling the current time difference, and a flag indicating if the UTC parameter is valid. Currently the difference between TAI and UTC is 36 seconds, so if the grandmaster sends no valid UTC offset, or utc_valid flag is not set, then a client probably just evaluates the raw TAI time, and thus its time steps by 36 seconds until it receives a PTP announcement message with valid UTC parameters again. I don't know the details of your PTP setup, but we (at Meinberg) have already had another customer some time ago where a Cisco switch acting as boundary clock didn't send UTC data reliably when the grandmaster was switched. So eventually you should contact Cisco and see if there is an update available for the switch where this is fixed. Martin ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
On Fri, January 29, 2016 1:32 am, NeT MonK wrote: > As a side effect of those glitch in the GPS matrix, the utc_valid_flag was > not anymore set in the stream of the primary master clock, just before my > secondary starts to become active (loss of primary stream) which leads to > linux server ptp slave to readjust of +36 seconds and jump backward -36 > seconds as far as the flags was coming back. The difference between UTC and TAI is 36 seconds. Are you running ptpd or linuxptp on your Linux servers? It sounds like the ptp agent running on your servers interpreted the lack of UTC valid flag to mean that the timestamps were now in TAI, so the server kernel applied the TAI to UTC offset to the time received via PTP. -- Chris Caudle ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
On Fri, Jan 29, 2016 at 2:45 PM, Martin Burnicki < martin.burni...@burnicki.net> wrote: > Hello Net Monk, > ... > I don't know the details of your PTP setup, but we (at Meinberg) have > already had another customer some time ago where a Cisco switch acting > as boundary clock didn't send UTC data reliably when the grandmaster was > switched. > > So eventually you should contact Cisco and see if there is an update > available for the switch where this is fixed. > > Martin > > > Hello Martin, Thank you for your reply and details, this is also what i already stated on my incident report. My ptp client applied the utc offset when the flag disappeared which lead to this jump forward and backward. A call is already opened to Cisco and they are coming in coming weeks for auditing the ptp infra. Regards. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
A nice summary: http://www.insidegnss.com/node/4831 /tvb ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
On Fri, Jan 29, 2016 at 3:29 PM, Chris Caudlewrote: > On Fri, January 29, 2016 1:32 am, NeT MonK wrote: > > As a side effect of those glitch in the GPS matrix, the utc_valid_flag > was > > not anymore set in the stream of the primary master clock, just before my > > secondary starts to become active (loss of primary stream) which leads to > > linux server ptp slave to readjust of +36 seconds and jump backward -36 > > seconds as far as the flags was coming back. > > The difference between UTC and TAI is 36 seconds. > > Are you running ptpd or linuxptp on your Linux servers? > It sounds like the ptp agent running on your servers interpreted the lack > of UTC valid flag to mean that the timestamps were now in TAI, so the > server kernel applied the TAI to UTC offset to the time received via PTP. Dear Chris, I run sfptpd which is a fork of linux ptpd daemon adapted to solarflare network adapter. I will have to fine tune it in order to be more resilient in such scenario. But i guess it's not very often the gps system as such incident. Regards ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
I have an idle curiosity as to whether the erroneous UTC correction propagated into any high-speed trading platforms in the financial markets in a what that caused a disruption. signature.asc Description: Message signed with OpenPGP using GPGMail ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
This may be a dumb question, but I am new to using LadyHeather with my Thunderbolt (previously it had been slaved to driving a clock display but that broke, which made me get around to installing a serial card in my linux box)... Are these "events" something I should be able to see in LH? If so what should I be looking for? I see a pair of off-setting events in the graphs (a sharp negative spike in DAC Voltage & PPS, followed later by a sharp positive spike of the same magnitude), but not sure if I am seeing what I want to see, of if this is actually what I am looking for. Off to study the LH Manual some more! 73, Brian KD4UYP On Wed, 2016-01-27 at 01:15 +0100, Magnus Danielson wrote: > Hi, > > On 01/26/2016 11:46 PM, Martin Burnicki wrote: > > Paul Boven wrote: > > > Hi everyone, > > > > > > Has anyone else seen GPS time jump by -13.7 usec today? > > > I just heard from several geographically quite distributed radio > > > observatories that they have seen their GPS receiver(s) jump > > > compared to > > > their in-house standards. > > > > We were able to track this down today at Meinberg. > > > > The problem was that some satellites were sending invalid UTC > > correction > > parameters. The UTC correction parameter set not only contains > > current > > leap second information but also coefficients (A0, A1, WNt, tot) > > for a > > polynomial used to compensate the fractional difference between GPS > > time > > and UTC(USNO). > > > > Normally A0 (the constant offset) is very small, close to 0. WNt > > and tot > > give the week number (truncated to 8 bits) and second-of-week for > > which > > A0 is valid, i.e. the reference time for the time correction > > parameters. > > WNt is currently about 89. > > > > Today the faulty satellites sent about 13.7 microseconds for A0, > > and WNt > > as well as tot were both 0. So when the GPS receiver updated its > > UTC > > correction parameters from a faulty satellite the UTC correction > > jumped > > from close to 0 to about 13.7 microseconds, which let the UTC time > > step, > > and when the GPS receiver received the UTC parameter set from a > > healthy > > satellite the UTC time stepped back. > > > > We have recorded a few of the faulty subframe words. If someone is > > interested I can provide more detailed information. However, I'm > > currently out of the office and don't have the information here > > right now. > > You know I want more details. ;-) > > I did see the report you guys sent to the USCG, good work there and > good > report. > > I also some another report showing issues with other SVNs. > > Cheers, > Magnus > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to > https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
brian wrote: > This may be a dumb question, but I am new to using LadyHeather with my > Thunderbolt (previously it had been slaved to driving a clock display > but that broke, which made me get around to installing a serial card in > my linux box)... > > Are these "events" something I should be able to see in LH? If so what > should I be looking for? I see a pair of off-setting events in the > graphs (a sharp negative spike in DAC Voltage & PPS, followed later by > a sharp positive spike of the same magnitude), but not sure if I am > seeing what I want to see, of if this is actually what I am looking > for. It depends on what you mean by "events". The problem should not affect normal GPS time, so navigation should not have been affected since only the conversion from GPS time to UTC was messed up. If you use the PPS output which normally corresponds to the changeover of the UTC second then the PPS slope should have stepped by 13.7 us or not, depending on if and when the GPS receiver accepted the faulty UTC correction data set from on of the satellites which broadcast it. If you use the 1 PPS signal to discipline an oscillator thenit depends on how the oscillator control loop deals with such time step, whether it silently follows the step, or if it generates some kind of alarm due to this unusual step. In this case the problem was visible when you looked at the current UTC correction parameters decoded from the satellites' nav messages. For example, using a Meinberg GPS180PEX card with its Linux driver you can see the following status: martin@pc-martin:~> mbgstatus -vvv /dev/mbgclock0 mbgstatus v3.4.99 copyright Meinberg 2001-2015 GPS180PEX 029511026220 (FW 2.04, ASIC 8.06) at port 0xE000, irq 16 Normal Operation, 9 sats in view, 9 sats used Osc type: TCXO, DAC cal: -550, fine: +666 Date/time: Th, 2016-01-28 08:19:53.05 UTC, st: 0x14 Local HR time: 2016-01-28 08:19:53.0548183 UTC, st: 0x0014 Status info: Input signal available Status info: Time is synchronized Status info: Receiver position has been verified Last sync: Th, 2016-01-28 08:19:00.00 UTC, st: 0x14 Receiver Position: x: 3885686m y: 631143m z: 5001726m lat: +51.9824 lon: +9.2258 alt: 167m latitude: N 51 deg 58 min 56.47 sec longitude: E 9 deg 13 min 33.05 sec CSUM: 1615, valid: 0001 t0t: 1881|503808.000, A0: -9.31323e-10 A1: 3.55271e-15 WNlsf: 1851, DN: 3, offs: 17/17 UTC offset parameter: 17s, no leap second announced. where the line starting with "t0t" prints the UTC correction data which was faulty in this case. While the problem occurred I ran this command periodically and grepped the t0t line only, so at the "recovery" time you cold see this: t0t: 1792|0.000, A0: -1.3696e-05 A1: 1.24345e-14 t0t: 1792|0.000, A0: -1.3696e-05 A1: 1.24345e-14 t0t: 1792|0.000, A0: -1.3696e-05 A1: 1.24345e-14 t0t: 1792|0.000, A0: -1.3696e-05 A1: 1.24345e-14 t0t: 1881|319488.000, A0: 0 A1: 1.24345e-14 t0t: 1881|319488.000, A0: 0 A1: 1.24345e-14 t0t: 1881|319488.000, A0: 0 A1: 1.24345e-14 t0t: 1881|319488.000, A0: 0 A1: 1.24345e-14 t0t: 1881|319488.000, A0: 0 A1: 1.24345e-14 Please note the week number here is the fully expanded week number: 1792 == 0x700, i.e. the 8 bit truncated value is 0, which is wrong 1881 == 0x759, i.e. the 8 bit truncated value is 0x59 == 89 correctly Unfortunately I don't know if the thunderbold can output this kind of data, and if LH can display it. I've not played with those, yet. Martin ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Paul Boven schrieb: > Hi everyone, > > Has anyone else seen GPS time jump by -13.7 usec today? > I just heard from several geographically quite distributed radio > observatories that they have seen their GPS receiver(s) jump compared to > their in-house standards. We have now received an official statement from USCG: -Original Message- > From: Hamilton, Stephen R CIV > Sent: Wednesday, January 27, 2016 8:13 PM > To: 'cg...@cgls.uscg.mil' > Subject: FW: Official Press Release - GPS Ground System Anomaly > > All CGSIC: > > Air Force Official Press Release - GPS Ground System Anomaly > >On 26 January at 12:49 a.m. MST, the 2nd Space Operations Squadron at the > 50th Space Wing, Schriever Air Force Base, Colo., verified users were > experiencing GPS timing issues. Further investigation revealed an issue in > the Global Positioning System ground software which only affected the time > on legacy L-band signals. This change occurred when the oldest vehicle, SVN > 23, was removed from the constellation. While the core navigation systems > were working normally, the coordinated universal time timing signal was off > by 13 microseconds which exceeded the design specifications. The issue was > resolved at 6:10 a.m. MST, however global users may have experienced GPS > timing issues for several hours. U.S. Strategic Command's Commercial > Integration Cell, operating out of the Joint Space Operations Center, > effectively served as the portal to determine the scope of commercial user > impacts. Additionally, the Joint Space Operations Center at Vandenberg AFB > has not received any reports of issues with GPS-aided munitions, and has > determined that the timing error is not attributable to any type of outside > interference such as jamming or spoofing. Operator procedures were modified > to preclude a repeat of this issue until the ground system software is > corrected, and the 50th Space Wing will conduct an Operational Review Board > to review procedures and impacts on users. Commercial and Civil users who > experienced impacts can contact the U.S. Coast Guard Navigation Center at > (703) 313-5900. > V/R > Rick Hamilton > CGSIC Executive Secretariat > GPS Information Analysis Team Lead > U.S. Coast Guard Navigation Center > 703-313-5930 Martin ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Magnus Danielson wrote: > It's interesting to see how consistent these errors are. > On the other hand, it is interesting to see how it varies even for the > same PRN. Look at how PRN 09 varies between +0.002 us and -13.696 us. Hm, I think basically it doesn't vary. We fortunately just saw the point in time where it was corrected. During the problem all affected satellites sent the same, wrong data set, and obviously a new proper data set was uploaded to or activated on these satellites, so the stopped sending faulty data and switched to sending correct data *once*. Martin ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us? (TBolt data)
I have a hack patch in my copy of the TBolt driver for ntpd. 01-21T21:42:15 ntpd[7559]: TBolt 1 Other: -2.79397e-09 -8.88178e-15 01-22T17:21:08 ntpd[7559]: TBolt 1 Other: -4.65661e-09 -1.06581e-14 01-23T15:13:38 ntpd[7559]: TBolt 1 Other: -6.51926e-09 -1.15463e-14 01-24T17:08:37 ntpd[7559]: TBolt 1 Other: -9.31323e-10 5.32907e-15 01-25T16:16:07 ntpd[7559]: TBolt 1 Other: -1.3696e-05 1.24345e-14 01-26T18:18:38 ntpd[7559]: TBolt 1 Other: -2.79397e-09 -8.88178e-16 Times are local, PST, 8 hours west of UTC. It's supposed to print them out any time they change. The data comes from packet type 0x58, subtype 5. I haven't checked to see when those get sent. -- These are my opinions. I hate spam. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Hi Martin, On 01/27/2016 05:49 PM, Martin Burnicki wrote: Magnus Danielson wrote: Hi, You know I want more details. ;-) I did see the report you guys sent to the USCG, good work there and good report. Thanks. After we had received the first reports about GPS receiver showing a 13 us time offset one of my colleagues who maintains the firmware of our GPS receivers started to investigate. Looking at the data sets decoded from the satellites he saw that temporarily the UTC correction parameter jumped from one or 2 nanoseconds to more than 13 microseconds. So he added some debug code which let the GPS receiver sent the satellite number plus the raw subframe words 7 and 8 as hex numbers to a monitor program which logged them to a file. Several GPS receivers have support to output the raw data, and this is one very good reason. Being able to pull the interpreted data, the covariance matrix etc. and other state like raw pseudo-ranges and doppler data all helps. Being able to output RINEX is always very nice for many reasons. If I only had the time... Then we looked at the raw data, decoded the bits manually and found that some satellites were suspicious UTC correction data with a valid checksum. Here are some subframe words 7 and 8 captured starting around 2016-01-26 12:00 UTC from subframe 4 page 18, and the decoded parameters according to Figure 20-1, sheet 8 of IS-GPS-200H (printed page 83, PDF page 96) http://www.gps.gov/technical/icwg/IS-GPS-200H.pdf svsfw7 sfw8wnt|tot a0 bitsa0[us] 09 0x31B3 0x23800017 --> 00|00: 0xC68E -13.696 * 07 0x3FEA 0x3FD3967B --> 89|319488: 0x -0.001 02 0x3FD5 0x3FD39644 --> 89|319488: 0x -0.001 06 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 23 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 30 0x 0x00139664 --> 89|319488: 0x +0.000 05 0x003F 0x0013965B --> 89|319488: 0x +0.000 16 0x 0x00139664 --> 89|319488: 0x +0.000 26 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 07 0x3FEA 0x3FD3967B --> 89|319488: 0x -0.001 09 0x31B3 0x23800017 --> 00|00: 0xC68E -13.696 * 02 0x3FD5 0x3FD39644 --> 89|319488: 0x -0.001 30 0x 0x00139664 --> 89|319488: 0x +0.000 05 0x003F 0x0013965B --> 89|319488: 0x +0.000 06 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 23 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 16 0x 0x00139664 --> 89|319488: 0x +0.000 07 0x3FEA 0x3FD3967B --> 89|319488: 0x -0.001 09 0x 0x0098D642 --> 89|405504: 0x0002 +0.002 30 0x 0x00139664 --> 89|319488: 0x +0.000 02 0x3FD5 0x3FD39644 --> 89|319488: 0x -0.001 05 0x003F 0x0013965B --> 89|319488: 0x +0.000 23 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 06 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 16 0x 0x00139664 --> 89|319488: 0x +0.000 07 0x3FEA 0x3FD3967B --> 89|319488: 0x -0.001 09 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 30 0x 0x00139664 --> 89|319488: 0x +0.000 05 0x003F 0x0013965B --> 89|319488: 0x +0.000 02 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 06 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 23 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 16 0x 0x00139664 --> 89|319488: 0x +0.000 It's interesting to see how consistent these errors are. On the other hand, it is interesting to see how it varies even for the same PRN. Look at how PRN 09 varies between +0.002 us and -13.696 us. You see that SVs 09, 06, 23, and 26 sent a tot time 0 and a correction parameter A0 of -13.696 microseconds, while the A0 parameter received from healthy satellites was only 1 or 2 nanoseconds, as usual, and the week number wnt matches the current week number, truncated to 8 bit. Eventually there were even more satellites sending faulty data, but these were the ones we could track at our location. You can also see that in the last lines the A0 value sent by SVs 6, 9, and 23 was "normal" again, so we happily just captured a few of the last wrong data sets. All data captured after this was OK again. Please find the full set of debug output in the attached text file. Unfortunately there was no time stamp in the debug output, but we know the satellites repeat the same page 4 once every 12 minutes. Many thanks for that wealth of data. Cheers, Magnus ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Hi Martin, On 2016-01-27 17:49:52, Martin Burnicki wrote: We were able to track this down today at Meinberg. The problem was that some satellites were sending invalid UTC correction parameters. The UTC correction parameter set not only contains current leap second information but also coefficients (A0, A1, WNt, tot) for a polynomial used to compensate the fractional difference between GPS time and UTC(USNO). Today the faulty satellites sent about 13.7 microseconds for A0, and WNt as well as tot were both 0. So when the GPS receiver updated its UTC correction parameters from a faulty satellite the UTC correction jumped from close to 0 to about 13.7 microseconds, which let the UTC time step, and when the GPS receiver received the UTC parameter set from a healthy satellite the UTC time stepped back. Thanks for posting this, it's great that you managed to actually record those frames. This explanation fits the observed behaviour (steps of around 13us all over the world) much better than a malfunction of SVN 23. Furthermore, SVN 23 had already been decommissioned prior to these events. Regards, Paul Boven. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Magnus Danielson wrote: > Hi, > > On 01/26/2016 11:46 PM, Martin Burnicki wrote: >> Paul Boven wrote: >>> Hi everyone, >>> >>> Has anyone else seen GPS time jump by -13.7 usec today? >>> I just heard from several geographically quite distributed radio >>> observatories that they have seen their GPS receiver(s) jump compared to >>> their in-house standards. >> >> We were able to track this down today at Meinberg. >> >> The problem was that some satellites were sending invalid UTC correction >> parameters. The UTC correction parameter set not only contains current >> leap second information but also coefficients (A0, A1, WNt, tot) for a >> polynomial used to compensate the fractional difference between GPS time >> and UTC(USNO). >> >> Normally A0 (the constant offset) is very small, close to 0. WNt and tot >> give the week number (truncated to 8 bits) and second-of-week for which >> A0 is valid, i.e. the reference time for the time correction parameters. >> WNt is currently about 89. >> >> Today the faulty satellites sent about 13.7 microseconds for A0, and WNt >> as well as tot were both 0. So when the GPS receiver updated its UTC >> correction parameters from a faulty satellite the UTC correction jumped >> from close to 0 to about 13.7 microseconds, which let the UTC time step, >> and when the GPS receiver received the UTC parameter set from a healthy >> satellite the UTC time stepped back. >> >> We have recorded a few of the faulty subframe words. If someone is >> interested I can provide more detailed information. However, I'm >> currently out of the office and don't have the information here right >> now. > > You know I want more details. ;-) > > I did see the report you guys sent to the USCG, good work there and good > report. Thanks. After we had received the first reports about GPS receiver showing a 13 us time offset one of my colleagues who maintains the firmware of our GPS receivers started to investigate. Looking at the data sets decoded from the satellites he saw that temporarily the UTC correction parameter jumped from one or 2 nanoseconds to more than 13 microseconds. So he added some debug code which let the GPS receiver sent the satellite number plus the raw subframe words 7 and 8 as hex numbers to a monitor program which logged them to a file. Then we looked at the raw data, decoded the bits manually and found that some satellites were suspicious UTC correction data with a valid checksum. Here are some subframe words 7 and 8 captured starting around 2016-01-26 12:00 UTC from subframe 4 page 18, and the decoded parameters according to Figure 20-1, sheet 8 of IS-GPS-200H (printed page 83, PDF page 96) http://www.gps.gov/technical/icwg/IS-GPS-200H.pdf svsfw7 sfw8wnt|tot a0 bitsa0[us] 09 0x31B3 0x23800017 --> 00|00: 0xC68E -13.696 * 07 0x3FEA 0x3FD3967B --> 89|319488: 0x -0.001 02 0x3FD5 0x3FD39644 --> 89|319488: 0x -0.001 06 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 23 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 30 0x 0x00139664 --> 89|319488: 0x +0.000 05 0x003F 0x0013965B --> 89|319488: 0x +0.000 16 0x 0x00139664 --> 89|319488: 0x +0.000 26 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 07 0x3FEA 0x3FD3967B --> 89|319488: 0x -0.001 09 0x31B3 0x23800017 --> 00|00: 0xC68E -13.696 * 02 0x3FD5 0x3FD39644 --> 89|319488: 0x -0.001 30 0x 0x00139664 --> 89|319488: 0x +0.000 05 0x003F 0x0013965B --> 89|319488: 0x +0.000 06 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 23 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 16 0x 0x00139664 --> 89|319488: 0x +0.000 07 0x3FEA 0x3FD3967B --> 89|319488: 0x -0.001 09 0x 0x0098D642 --> 89|405504: 0x0002 +0.002 30 0x 0x00139664 --> 89|319488: 0x +0.000 02 0x3FD5 0x3FD39644 --> 89|319488: 0x -0.001 05 0x003F 0x0013965B --> 89|319488: 0x +0.000 23 0x318C 0x23800028 --> 00|00: 0xC68E -13.696 * 06 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 16 0x 0x00139664 --> 89|319488: 0x +0.000 07 0x3FEA 0x3FD3967B --> 89|319488: 0x -0.001 09 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 30 0x 0x00139664 --> 89|319488: 0x +0.000 05 0x003F 0x0013965B --> 89|319488: 0x +0.000 02 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 06 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 23 0x003F 0x0098D67D --> 89|405504: 0x0002 +0.002 16 0x 0x00139664 --> 89|319488: 0x +0.000 You see that SVs 09, 06, 23, and 26 sent a tot time 0 and a correction parameter A0 of -13.696 microseconds, while the A0 parameter received from healthy satellites was only 1 or 2 nanoseconds, as usual, and the week
[time-nuts] GPS jumps of -13.7 us?
Hi everyone, Has anyone else seen GPS time jump by -13.7 usec today? I just heard from several geographically quite distributed radio observatories that they have seen their GPS receiver(s) jump compared to their in-house standards. Regards, Paul Boven. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Indeed, multiple Z3801A GPSCON pages show an impulse step for by the amount you describe. I would describe the graphs as showing a positive TI step followed by negative TI step. My reading of the graphs shows that it is consistent with the 13.7 usec you describe. Am attempting to attach some EFC/TI graphs. Everyone seems to configure their scales a little differently. Tim N3QE On Tue, Jan 26, 2016 at 10:12 AM, Paul Bovenwrote: > Hi everyone, > > Has anyone else seen GPS time jump by -13.7 usec today? > I just heard from several geographically quite distributed radio > observatories that they have seen their GPS receiver(s) jump compared to > their in-house standards. > > Regards, Paul Boven. > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to > https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. > ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
On Tue, 26 Jan 2016 16:12:41 +0100 Paul Bovenwrote: > Has anyone else seen GPS time jump by -13.7 usec today? > I just heard from several geographically quite distributed radio > observatories that they have seen their GPS receiver(s) jump compared to > their in-house standards. Do you have references to those reports? The only one i've seen is from the Finnish Alto University Observatory: https://blogs.aalto.fi/metsahovi/gps-kellonajan-outo-hyppays-huomattiin-metsahovissa/#.VqecHW-Vtpg Attila Kinali -- Reading can seriously damage your ignorance. -- unknown ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Paul, I've received inquires about the 13 us jump since 8 PM last night (PST). Possibly related: http://www.navcen.uscg.gov/?Do=gpsShowNanu=2016008 Check your logs to see if your receiver is using SVN23/PRN32. /tvb - Original Message - From: "Paul Boven" <p.bo...@xs4all.nl> To: "Discussion of precise time and frequency measurement" <time-nuts@febo.com> Sent: Tuesday, January 26, 2016 7:12 AM Subject: [time-nuts] GPS jumps of -13.7 us? > Hi everyone, > > Has anyone else seen GPS time jump by -13.7 usec today? > I just heard from several geographically quite distributed radio > observatories that they have seen their GPS receiver(s) jump compared to > their in-house standards. > > Regards, Paul Boven. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
In message <56a78ce9.20...@xs4all.nl>, Paul Boven writes: >Has anyone else seen GPS time jump by -13.7 usec today? >I just heard from several geographically quite distributed radio >observatories that they have seen their GPS receiver(s) jump compared to >their in-house standards. Here is a quick plot: http://phk.freebsd.dk/misc/20160126_gps.svg The two receivers see the same (number of) satelites, but the jumps happen at widely different times. Magnitude around 14 microseconds seems confirmed. These are both Oncore timing receivers. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
The GPSCon "International Z3801A web plot" page is still active: http://www.realhamradio.com/GPS_websites_list.htm Most of the entries are stale but here's a few that show the recent glitch nicely: http://www.shaunmerrigan.info/timeandfreq/gpscon/gpsstat.htm http://www.shaunmerrigan.info/timeandfreq/gpscon1/gpsstat.htm http://www.shaunmerrigan.info/timeandfreq/gpscon2/gpsstat.htm /tvb - Original Message - From: "Poul-Henning Kamp" <p...@phk.freebsd.dk> To: "Discussion of precise time and frequency measurement" <time-nuts@febo.com>; "Paul Boven" <p.bo...@xs4all.nl> Sent: Tuesday, January 26, 2016 8:32 AM Subject: Re: [time-nuts] GPS jumps of -13.7 us? > > In message <56a78ce9.20...@xs4all.nl>, Paul Boven writes: > >>Has anyone else seen GPS time jump by -13.7 usec today? >>I just heard from several geographically quite distributed radio >>observatories that they have seen their GPS receiver(s) jump compared to >>their in-house standards. > > Here is a quick plot: > > http://phk.freebsd.dk/misc/20160126_gps.svg > > The two receivers see the same (number of) satelites, but the jumps > happen at widely different times. > > Magnitude around 14 microseconds seems confirmed. > > These are both Oncore timing receivers. > > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > p...@freebsd.org | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
At 09:12 AM 1/26/2016, Paul Boven wrote: Has anyone else seen GPS time jump by -13.7 usec today? My tbolt locked ntp server saw an unusual spike of about that magnitude just after 00:00 1-26-2016 UTC. (The bump at 4 seconds is from the daily backup run.) -- newell N5TNL ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Here's what my Z3801A saw using Z38XX. 00:00 on the graph = 06:00 UTC. My location is ~N50, W104. I don't know what SVNs were used. I'm not logging that data. Ed On 2016-01-26 11:00 AM, time-nuts-requ...@febo.com wrote: Message: 3 Date: Tue, 26 Jan 2016 16:12:41 +0100 From: Paul Boven <p.bo...@xs4all.nl> To: Discussion of precise time and frequency measurement <time-nuts@febo.com> Subject: [time-nuts] GPS jumps of -13.7 us? Message-ID: <56a78ce9.20...@xs4all.nl> Content-Type: text/plain; charset=utf-8; format=flowed Hi everyone, Has anyone else seen GPS time jump by -13.7 usec today? I just heard from several geographically quite distributed radio observatories that they have seen their GPS receiver(s) jump compared to their in-house standards. Regards, Paul Boven. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Paul Boven wrote: > Hi everyone, > > Has anyone else seen GPS time jump by -13.7 usec today? > I just heard from several geographically quite distributed radio > observatories that they have seen their GPS receiver(s) jump compared to > their in-house standards. We were able to track this down today at Meinberg. The problem was that some satellites were sending invalid UTC correction parameters. The UTC correction parameter set not only contains current leap second information but also coefficients (A0, A1, WNt, tot) for a polynomial used to compensate the fractional difference between GPS time and UTC(USNO). Normally A0 (the constant offset) is very small, close to 0. WNt and tot give the week number (truncated to 8 bits) and second-of-week for which A0 is valid, i.e. the reference time for the time correction parameters. WNt is currently about 89. Today the faulty satellites sent about 13.7 microseconds for A0, and WNt as well as tot were both 0. So when the GPS receiver updated its UTC correction parameters from a faulty satellite the UTC correction jumped from close to 0 to about 13.7 microseconds, which let the UTC time step, and when the GPS receiver received the UTC parameter set from a healthy satellite the UTC time stepped back. We have recorded a few of the faulty subframe words. If someone is interested I can provide more detailed information. However, I'm currently out of the office and don't have the information here right now. Martin ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Hi, On 01/26/2016 11:46 PM, Martin Burnicki wrote: Paul Boven wrote: Hi everyone, Has anyone else seen GPS time jump by -13.7 usec today? I just heard from several geographically quite distributed radio observatories that they have seen their GPS receiver(s) jump compared to their in-house standards. We were able to track this down today at Meinberg. The problem was that some satellites were sending invalid UTC correction parameters. The UTC correction parameter set not only contains current leap second information but also coefficients (A0, A1, WNt, tot) for a polynomial used to compensate the fractional difference between GPS time and UTC(USNO). Normally A0 (the constant offset) is very small, close to 0. WNt and tot give the week number (truncated to 8 bits) and second-of-week for which A0 is valid, i.e. the reference time for the time correction parameters. WNt is currently about 89. Today the faulty satellites sent about 13.7 microseconds for A0, and WNt as well as tot were both 0. So when the GPS receiver updated its UTC correction parameters from a faulty satellite the UTC correction jumped from close to 0 to about 13.7 microseconds, which let the UTC time step, and when the GPS receiver received the UTC parameter set from a healthy satellite the UTC time stepped back. We have recorded a few of the faulty subframe words. If someone is interested I can provide more detailed information. However, I'm currently out of the office and don't have the information here right now. You know I want more details. ;-) I did see the report you guys sent to the USCG, good work there and good report. I also some another report showing issues with other SVNs. Cheers, Magnus ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] GPS jumps of -13.7 us?
Slashdot got it. http://it.slashdot.org/story/16/01/26/1735223/discrepancy-detected-in-gps-time On 01/26/2016 12:20 PM, Scott Newell wrote: > At 09:12 AM 1/26/2016, Paul Boven wrote: > >> Has anyone else seen GPS time jump by -13.7 usec today? > > My tbolt locked ntp server saw an unusual spike of about that > magnitude just after 00:00 1-26-2016 UTC. (The bump at 4 seconds > is from the daily backup run.) > > > > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.