Re: [ntp:questions] A Christmas puzzler - intermittent offset oscillations with a PPS source
On 21/12/2012 19:51, Rick Jones wrote: [] Does the gpsd do anything every 5/3 minutes? Or put another way, can you find a similar periodicity in the CPU utilization? If it does do something interesting at that frequency and it involves a system call, while the act of tracing would perturb things, you might find it in a (timestamped) system call trace (strace) of the gpsd. Perhaps the luck of process scheduling and the gpsd or some other daemon holds-off the ntpd? (Raspberry Pi's are single-core systems right?) Does the ntpd run at a higher (realtime?) priority than the gpsd? Might there be any other background dameons consuming more CPU on the one system than the other? rick jones Thanks for your input, Rick. There's nothing obvious on the CPU plot which correlates with the NTP offset plot, so I'm unsure whether they are related. As an experiment, I've now removed NTP's dependency on gpsd (although it didn't appear to be selected as a source, and was /not/ the prefer source), and I will see whether the oscillation re-occurs. If I still see the problem, I will restart the system without gpsd enabled. The systems are nominally identical, except that the oscillating system has the serial data fed via GPIO pins, and the stable system is using USB for the serial I/O, but that serial data isn't being used anywhere (NTP relies on the LAN servers for its coarse time, and experimentally it's now doing that on /both/ systems). -- Cheers, David Web: http://www.satsignal.eu ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Clock Offset
Mischanko, Edward T wrote: Thank you for your reply. This certainly explains the differences, *** but this would seem to raise another question. Any correction *** in frequency would have to come from the most current average *** of offsets, not the last 8? Otherwise, how can NTP keep up *** with the changes in a high jitter situation? If the offset is changing quickly, it reduces the poll interval, so that the 8 samples span a smaller amount of time. Also, the definition of best is the ones with the lowest round trip delay. If the measurement is changing very rapidly it is likely to be due to variations in that delay, and the one have the lowest delay are likely to be the most accurate ones. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Win7: ntpd adjusting time backwards
On 12/11/2012 12:49 PM, David Taylor wrote: Sorry: catching up. What happens if the link to the Internet is rather asymmetrical? For example, here I am stuck with 30 Mb/s down, but only 3 Mb/s up. The actual bitrate is not so important. True: it determines the time a packet spends on the wire. But more important is (or can be) the amount of time a packet spends in various queues before actually being sent. This time varies with instantaneous network load, and with the size of the queue. Google for bufferbloat, and apologies if everyone here already knows all of this. Having said that: there can indeed be asymmetrical transmission delays that are linked to the technology being used. My VDSL2 modem tells me that the downstream delay is 14.1ms and the upstream delay is 4.4ms. The ratio of these numbers is not equal to the ratio of the downstream and upstream bitrates (which are 16544 kbit/s and 2056 kbit/s respectively). So note also that the downstream delay is greater than the upstream delay, although the downstream bitrate is higher. HTH, Jan ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Win7: ntpd adjusting time backwards
On 22/12/2012 14:36, Jan Ceuleers wrote: On 12/11/2012 12:49 PM, David Taylor wrote: Sorry: catching up. What happens if the link to the Internet is rather asymmetrical? For example, here I am stuck with 30 Mb/s down, but only 3 Mb/s up. The actual bitrate is not so important. True: it determines the time a packet spends on the wire. But more important is (or can be) the amount of time a packet spends in various queues before actually being sent. This time varies with instantaneous network load, and with the size of the queue. Google for bufferbloat, and apologies if everyone here already knows all of this. Having said that: there can indeed be asymmetrical transmission delays that are linked to the technology being used. My VDSL2 modem tells me that the downstream delay is 14.1ms and the upstream delay is 4.4ms. The ratio of these numbers is not equal to the ratio of the downstream and upstream bitrates (which are 16544 kbit/s and 2056 kbit/s respectively). So note also that the downstream delay is greater than the upstream delay, although the downstream bitrate is higher. HTH, Jan Jan, yes, I appreciate that it's not the rate as such, I was really trying to show how asymmetrical (10:1) the connection provided by my ISP was. If they are trying to pack in as many customers as possible, who knows what the actual backbone might be like? This is a cable modem connection, by the way, and not ADSL. I wonder how I could get the delay figures for my own modem? The only data I can extract is: Latency: ~20 milliseconds, RTP jitter - up: 4 milliseconds, down: ~0.12 milliseconds. Fascinating to see you have less delay on the slower upstream! -- Cheers, David Web: http://www.satsignal.eu ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Win7: ntpd adjusting time backwards
Jan Ceuleers jan.ceule...@computer.org wrote: Having said that: there can indeed be asymmetrical transmission delays that are linked to the technology being used. My VDSL2 modem tells me that the downstream delay is 14.1ms and the upstream delay is 4.4ms. The ratio of these numbers is not equal to the ratio of the downstream and upstream bitrates (which are 16544 kbit/s and 2056 kbit/s respectively). So note also that the downstream delay is greater than the upstream delay, although the downstream bitrate is higher. This is caused by interleaving. Packets are not sent back to back sequentially, but they are split into smaller chunks that are interleaved serially. This means a single packet is spread out over time, meaning that a hit of interference takes out a smaller part of each packet. And that again means that the error correction coding that is added to each packet has more chance of recovering the packet. (it is better to hit 8 packets each with a small amount of interference than to hit a single packet with a big blow) This improves the performance for streaming, but it increases the delay because the receiving modem has to wait longer for the packet to be complete. As the downstream with its higher bitrate is more affected by interference, more interleaving is usually applied to it as well. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Win7: ntpd adjusting time backwards
David Taylor wrote: On 22/12/2012 14:36, Jan Ceuleers wrote: On 12/11/2012 12:49 PM, David Taylor wrote: Sorry: catching up. What happens if the link to the Internet is rather asymmetrical? For example, here I am stuck with 30 Mb/s down, but only 3 Mb/s up. The actual bitrate is not so important. True: it determines the time a packet spends on the wire. But more important is (or can be) the amount of time a packet spends in various queues before actually being sent. This time varies with instantaneous network load, and with the size of the queue. Google for bufferbloat, and apologies if everyone here already knows all of this. Having said that: there can indeed be asymmetrical transmission delays that are linked to the technology being used. My VDSL2 modem tells me that the downstream delay is 14.1ms and the upstream delay is 4.4ms. The ratio of these numbers is not equal to the ratio of the downstream and upstream bitrates (which are 16544 kbit/s and 2056 kbit/s respectively). So note also that the downstream delay is greater than the upstream delay, although the downstream bitrate is higher. HTH, Jan Jan, yes, I appreciate that it's not the rate as such, I was really trying to show how asymmetrical (10:1) the connection provided by my ISP was. If they are trying to pack in as many customers as possible, who knows what the actual backbone might be like? This is a cable modem connection, by the way, and not ADSL. I wonder how I could get the delay figures for my own modem? The only data I can extract is: Latency: ~20 milliseconds, RTP jitter - up: 4 milliseconds, down: ~0.12 milliseconds. Fascinating to see you have less delay on the slower upstream! My own usage is more like 15:1 down:up so my 13:1.2 Mbps down:up data rates seem to fit quite well. When on 2 Mbps I had problems with very high latency on uploads and implemented altq traffic shaping which increased my maximum upload speed from around 150 kbps to around 250 kbps. There is also higher priority for icmp, dns and ntp. David ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Win7: ntpd adjusting time backwards
On 22/12/2012 17:49, David Lord wrote: [] My own usage is more like 15:1 down:up so my 13:1.2 Mbps down:up data rates seem to fit quite well. When on 2 Mbps I had problems with very high latency on uploads and implemented altq traffic shaping which increased my maximum upload speed from around 150 kbps to around 250 kbps. There is also higher priority for icmp, dns and ntp. David I don't have the figures to hand, but for some protocols (FTP?) there are acknowledgement packets sent back every so often, and I think that 10:1 is somewhere near the limit. With a greater up:down ratio the limiting factor in downstream speed is actually the upstream bandwidth! Something like that, anyway. It matters less what your usage is. I just happened to find a page which tells me that in the last 9 days I have downloaded 15 GB and uploaded 6 GB. -- Cheers, David Web: http://www.satsignal.eu ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
[ntp:questions] FW: Clock Offset
-Original Message- From: Mischanko, Edward T Sent: Friday, December 21, 2012 9:14 PM Subject: RE: [ntp:questions] Clock Offset -Original Message- From: questions- bounces+edward.mischanko=arcelormittal@lists.ntp.org [mailto:questions- bounces+edward.mischanko=arcelormittal@lists.ntp.org] On Behalf Of Terje Mathisen Sent: Thursday, December 20, 2012 2:42 AM To: questions@lists.ntp.org Subject: Re: [ntp:questions] Clock Offset Mischanko, Edward T wrote: No local clock is in use!! I was really questioning how the averaging algorithm could come up with a negative offset when none of the individual offsets were negative? That's (sort of) easy! The local clock offset is the best estimate of the (exponentially afair) averaging algorithm from the individual iterations of the control loop, while the ntpq -p gives you the last measurement from each server/clock source. I.e. they measure different things. Terje -- - Terje.Mathisen at tmsw.no almost all programming can be viewed as an exercise in caching ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions [Mischanko, Edward T] Terje, I do not doubt you, but if they are measuring different things to represent the same thing, then there is a formula off somewhere? Ed ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] FW: Clock Offset
On 2012-12-22, Mischanko, Edward T edward.mischa...@arcelormittal.com wrote: -Original Message- From: Mischanko, Edward T Sent: Friday, December 21, 2012 9:14 PM Subject: RE: [ntp:questions] Clock Offset -Original Message- From: questions- bounces+edward.mischanko=arcelormittal@lists.ntp.org [mailto:questions- bounces+edward.mischanko=arcelormittal@lists.ntp.org] On Behalf Of Terje Mathisen Sent: Thursday, December 20, 2012 2:42 AM To: questions@lists.ntp.org Subject: Re: [ntp:questions] Clock Offset Mischanko, Edward T wrote: No local clock is in use!! I was really questioning how the averaging algorithm could come up with a negative offset when none of the individual offsets were negative? That's (sort of) easy! The local clock offset is the best estimate of the (exponentially afair) averaging algorithm from the individual iterations of the control loop, while the ntpq -p gives you the last measurement from each server/clock source. I.e. they measure different things. Terje -- - Terje.Mathisen at tmsw.no almost all programming can be viewed as an exercise in caching ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions [Mischanko, Edward T] Terje, I do not doubt you, but if they are measuring different things to represent the same thing, then there is a formula off somewhere? No, they do not represent the same thing. They are different things. One is the instantaneously measured offset between the local clock and the servers. The other is the best estimate ntpd has for the difference between the local clock and true UTC time. Ed ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] FW: Clock Offset
unruh wrote: On 2012-12-22, Mischanko, Edward T edward.mischa...@arcelormittal.com wrote: I do not doubt you, but if they are measuring different things to represent the same thing, then there is a formula off somewhere? No, they do not represent the same thing. They are different things. One is the instantaneously measured offset between the local clock and the servers. The other is the best estimate ntpd has for the difference between the local clock and true UTC time. I don't believe he is now comparing the overall estimate with the most recent individual measurements. I think he is comparing the best estimate for an individual source, with the ntpq peers, most recent offset for that source. The formula for an individual filtered offset is something like: (z ^ n) * Offset where z is the Z-Transform operator and n is an natural number = 7, such that (z ^ n) * estimated-offset-error is minimal. Of course, it is not described that way in the actual code, and I don't think it is described that way in the documentation. I believe the error estimate includes the round trip delay, and the stratum, but it is that sort of thing that is subject to change. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] FW: Clock Offset
-Original Message- From: questions- bounces+edward.mischanko=arcelormittal@lists.ntp.org [mailto:questions- bounces+edward.mischanko=arcelormittal@lists.ntp.org] On Behalf Of David Woolley Sent: Saturday, December 22, 2012 6:14 PM To: questions@lists.ntp.org Subject: Re: [ntp:questions] FW: Clock Offset unruh wrote: On 2012-12-22, Mischanko, Edward T edward.mischa...@arcelormittal.com wrote: I do not doubt you, but if they are measuring different things to represent the same thing, then there is a formula off somewhere? No, they do not represent the same thing. They are different things. One is the instantaneously measured offset between the local clock and the servers. The other is the best estimate ntpd has for the difference between the local clock and true UTC time. I don't believe he is now comparing the overall estimate with the most recent individual measurements. I think he is comparing the best estimate for an individual source, with the ntpq peers, most recent offset for that source. The formula for an individual filtered offset is something like: (z ^ n) * Offset where z is the Z-Transform operator and n is an natural number = 7, such that (z ^ n) * estimated-offset-error is minimal. Of course, it is not described that way in the actual code, and I don't think it is described that way in the documentation. I believe the error estimate includes the round trip delay, and the stratum, but it is that sort of thing that is subject to change. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions [Mischanko, Edward T] David, Thank you for your time in explaining this to me. Unfortunately, my mathematics education stopped at intermediate algebra; it appears that the formula is in calculus? Anyway, I feel like I have beaten this horse to death. I can accept the fact that they are different and yet notice their similarities. I'm not sure Unruh completely understands my point? Regards, Ed ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
