Re: Proposal - drop the GPSD JSON driver
Yo Eric! On Sat, 22 Oct 2016 09:21:59 -0400 "Eric S. Raymond" wrote: > Hal Murray : > > >> I assume that using a pipe or socket rather than SHM would fix > > >> that. > > > > > Probably, but then we run unto buffering jitter again. > > > > Are we on the same wavelength yet? Have we agreed that latency is > > not critical? If so, why is jitter important? > > It is possible that I am confused. Now we agree. :-) > What ntpd gets from a clock source is a series of pairs asserting > "at system time X I believe it was UTC time Y". Yes. > Are you telling me there is no value in minimizing the time from X to > when the sample triggers a correction, and the variation in that time? Yes. ntpd sits on the data, by default, for 32 seconds average, and 64 seconds max. The big ntpd loop only goes around every second. A few micro seconds is not releavnt. Long term tests on GPSD-JSON confirm this. > Basic servocontrol theory tells me that control lag is the enemy of > precision and tends to produce whiplash effects. Does that not apply > here? With all the averaging, filtering and polling, the transit time from delta generation to ntpd input is way less then 3 orders of magnitude less than the ntpd control loop cycle (one second). Then when ntpd averages the deltas over 64 seconds all immmediacy is lost. But don't believe me, or my long term GPSD-experiments, set up your own. Not hard. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp1PmhgBrp9Z.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Hal! On Sat, 22 Oct 2016 03:15:01 -0700 Hal Murray wrote: > >> Word-length mismatch between two programs built under the same OS > >> never happens, or close enough to never that I don't care. > > Uh, no. remember when intel OS went from 32 bit to 64-bit? It was > > a huge issue with ntpd. RasPi is about to have the same > > problems. > > What sort of problem do you expect? Uh, the one I described in the previous email. Binary C structures will not match. Pain ensues, just like last time. > I'd expect troubles if you tried to run a 32 bit gpsd or 32 bit ntpd > on a 64 bit system (using 32 bit libraries) while the other one was > 64 bit and the shared struct used int or long rather than int32_t and > friends. Ah, so we agree, yes, those configurations will be problematic. We live through that in the intel 32 -> 64 bit treansition, we are about to live through it again on the ARM. > I just checked ntpd/refclock_shm.c Yeah, ugly, ugly, ugly. > >> As Hal points out, if you version-number the struct properly you > >> can add fields at the end ad libitum. > > IFF you do it right, AND chose a transport that can handle varying > > lengths. Traditional SHM can not do that. The current NTP SHM can > > not do that. > > The current SHM setup was extended to support nanoseconds without > breaking old code. (and it doesn't even have a version number slot) Yeah, but a ton if extensions not done because it would break old code. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp_59OU7SE08.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Hal! On Sat, 22 Oct 2016 02:31:01 -0700 Hal Murray wrote: > >> I assume that using a pipe or socket rather than SHM would fix > >> that. > > > Probably, but then we run unto buffering jitter again. > > Are we on the same wavelength yet? Have we agreed that latency is > not critical? If so, why is jitter important? Latency is critical, jitter is important, but the GPSD-JSON driver does not add any. As you said ealier, the deltas are computed in the kernel (for KPPS) or in gpsd (for PPS). The gpsd sends that to ntpd. ntpd does not even look at that interval more often than the defaullt 64s poll interval. So a few extra fractions of a second encoding, transmitting, and decoding, is swamped in the 64s poll interval. Even with the now possible 1 second poll it in insignificant. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgps2hBLrouuN.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
> What ntpd gets from a clock source is a series of pairs asserting "at system > time X I believe it was UTC time Y". Right. > Are you telling me there is no value in minimizing the time from X to when > the sample triggers a correction, and the variation in that time? > Basic servocontrol theory tells me that control lag is the enemy of > precision and tends to produce whiplash effects. Does that not apply here? Not "no value", but it depends on the time constant of the PLL filter. Delays of a small fraction of the time constant won't be problem. In the normal case, the data is put into a buffer and isn't processed until several/many seconds later. So except for the last sample, any delay and/or jitter under a second doesn't matter. Things will get more interesting if you try to run with minpoll of 0. My current thinking is that we should either use a cleaned up SHM or the same struct over a socket. The socket will fit in with the big select and has a reasonable chance of working well with minpoll of 0. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Hal Murray : > >> I assume that using a pipe or socket rather than SHM would fix that. > > > Probably, but then we run unto buffering jitter again. > > Are we on the same wavelength yet? Have we agreed that latency is not > critical? If so, why is jitter important? It is possible that I am confused. What ntpd gets from a clock source is a series of pairs asserting "at system time X I believe it was UTC time Y". Are you telling me there is no value in minimizing the time from X to when the sample triggers a correction, and the variation in that time? Basic servocontrol theory tells me that control lag is the enemy of precision and tends to produce whiplash effects. Does that not apply here? -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
>> I assume that using a pipe or socket rather than SHM would fix that. > Probably, but then we run unto buffering jitter again. Are we on the same wavelength yet? Have we agreed that latency is not critical? If so, why is jitter important? -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
>> Word-length mismatch between two programs built under the same OS >> never happens, or close enough to never that I don't care. > Uh, no. remember when intel OS went from 32 bit to 64-bit? It was a huge > issue with ntpd. RasPi is about to have the same problems. What sort of problem do you expect? I'd expect troubles if you tried to run a 32 bit gpsd or 32 bit ntpd on a 64 bit system (using 32 bit libraries) while the other one was 64 bit and the shared struct used int or long rather than int32_t and friends. I just checked ntpd/refclock_shm.c It's got int's, no longs. So does gpsd. We should make things explicit for the great SHM cleanup. It's also got time_t. That will break the 32/64 bit combination on Fedora. It's 4 bytes on 32 bit systems and 8 bytes on 64 bit systems. >> As Hal points out, if you version-number the struct properly you can >> add fields at the end ad libitum. > IFF you do it right, AND chose a transport that can handle varying lengths. > Traditional SHM can not do that. The current NTP SHM can not do that. The current SHM setup was extended to support nanoseconds without breaking old code. (and it doesn't even have a version number slot) -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Got it, thanks! On Fri, Oct 21, 2016, 5:20 PM Gary E. Miller wrote: > Yo Mark! > > On Fri, 21 Oct 2016 21:00:24 + > Mark Atwood wrote: > > > We should remove the GPSD JSON driver from NTPsec. > > I spend 40 minutes last night trying to get the GPSD SHM driver running > on a RasPi. I did not get it finished. The experience reminds me that > the #48 driver makes all sorts of non-standard assumptions about how > gpsd is configured. A configuration that is contorted and non-obvious > to implement. > > I don't even want to begin to discuss the new bag of worms. So, yes, > I now agree, kill it off. > > Just one thing to remember about the experience, #48 proves that using > JSON adds no uncertainty to the time measurements and yields identical > results when fed to ntpd. Please do not pull the driver until Eric > sees, and agrees, with that result. > > RGDS > GARY > --- > Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 > g...@rellim.com Tel:+1 541 382 8588 > ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Mark! On Fri, 21 Oct 2016 21:00:24 + Mark Atwood wrote: > We should remove the GPSD JSON driver from NTPsec. I spend 40 minutes last night trying to get the GPSD SHM driver running on a RasPi. I did not get it finished. The experience reminds me that the #48 driver makes all sorts of non-standard assumptions about how gpsd is configured. A configuration that is contorted and non-obvious to implement. I don't even want to begin to discuss the new bag of worms. So, yes, I now agree, kill it off. Just one thing to remember about the experience, #48 proves that using JSON adds no uncertainty to the time measurements and yields identical results when fed to ntpd. Please do not pull the driver until Eric sees, and agrees, with that result. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpQJdnTa2XUw.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
fallenpega...@gmail.com said: > We should remove the GPSD JSON driver from NTPsec. > It is an attractive nuisance. The SHM driver, possibly with improvements, > is a better interface between NTPsec and GPSD. It's useful for collecting data on non-SHM approaches. How about we defer dropping it until we have something better to replace it and/or a cleaned up SHM that we all like? -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Hi! I've been following this discussion. We should remove the GPSD JSON driver from NTPsec. It is an attractive nuisance. The SHM driver, possibly with improvements, is a better interface between NTPsec and GPSD. ..m On Thu, Oct 20, 2016 at 2:53 PM Eric S. Raymond wrote: > Hal Murray : > > > > e...@thyrsus.com said: > > >> Yes, I've been thinking about mechanisms for this. They're all rather > > >> irrelevant. > > > Argh. I meant "inelegant". > > > > I assume that using a pipe or socket rather than SHM would fix that. > > Probably, but then we run unto buffering jitter again. > > You're right, we need to build some test jigs and measure this stuff. > > I'll make a publishable white paper out of it. > -- > http://www.catb.org/~esr/";>Eric S. Raymond > ___ > devel mailing list > devel@ntpsec.org > http://lists.ntpsec.org/mailman/listinfo/devel > ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Hal Murray : > > e...@thyrsus.com said: > >> Yes, I've been thinking about mechanisms for this. They're all rather > >> irrelevant. > > Argh. I meant "inelegant". > > I assume that using a pipe or socket rather than SHM would fix that. Probably, but then we run unto buffering jitter again. You're right, we need to build some test jigs and measure this stuff. I'll make a publishable white paper out of it. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Hal! On Thu, 20 Oct 2016 14:19:05 -0700 Hal Murray wrote: > e...@thyrsus.com said: > >> The SHM interface is deltas. gpsd figures out the offset and > >> sends that over to ntpd. How gpsd figures out that offset has > >> nothing to do with SHM or JSON or anything else involved with > >> getting the data to ntpd. The > offset > >> won't change much in a short time so modest latency doesn't > >> matter. > > > Your last sentence is one of the assumptions we're now > > questioning. I didn't question it before either, which is why the > > above hack is not implemented. > > We can test it with the current code. Just setup ntpd to use both > SHM and JSON. Been there, done that, when Pearly wrote the GPSD JSON driver. Except for all the bugs in his driver, there is zero perfomance difference. And that makes logical sense, the deltas are from KPPS, and they are unused by ntpd for up to (default) 64 seconds), then averaged. So even a few seconds of delay is irrelevant. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp7jsu3xJGBq.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Eric! On Thu, 20 Oct 2016 16:52:06 -0400 "Eric S. Raymond" wrote: > > The SHM interface is deltas. gpsd figures out the offset and sends > > that over to ntpd. How gpsd figures out that offset has nothing to > > do with SHM or JSON or anything else involved with getting the data > > to ntpd. The offset won't change much in a short time so modest > > latency doesn't matter. > > Your last sentence is one of the assumptions we're now questioning. > I didn't question it before either, which is why the above hack is > not implemented. Ony you are questioning it. Deltas have been around in ntpd for decades. Then they get averaged over minpoll, which default to 64 seconds! The jitter you are looking for is not there. > As you say, we need to try several methods and measure. Try all you want, but the deltas come from KPPS, and the 64 second averaging is in the core ntpd. How they hook up is, by design, not a factor in performance. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpDnl0mgCScT.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Eric! On Thu, 20 Oct 2016 16:11:41 -0400 "Eric S. Raymond" wrote: > Gary E. Miller : > > > What sort of problems are there with the current one? > > > > Do we really need to go over this yet again? These have been > > covered for years. > > > > First, the structure needs to be polled, sometimes leading to long > > delayed and even missed samples. > > This will be the case with any shared-memory driver that doesn't have > a wait semaphore. But we try to stay POSIX, and the POSIX SHM has them, so a non-issue. > Which means that a wait semaphore is a good idea, And POSIX thought so too. > but is orthogonal to the struct-vs.-JSON-question to put in the box. Yes, we do tend to get all fuzzy. Sort of like the presidential debates? > You seem to have changed the subject from "the existing SHM sriver is > bad" to "any shared-memory driver would be bad". Do you believe that? Of course not. Which is why I keep saying I favor JSON in POSIX SHM. And been saying it for at least a month. > > Second, a big one, the C structure is of loosely defined size and > > shape. Even on the same host you need to use the same compiler and > > word length for client and server. For example, you can't compile > > the client as 32 bit and the server as 64-bit. > > No sale. You don't need to use the same compiler, just two compilers > with self-aligned padding. Fair enough, sadly a common problem, so still a reason against. > If this weren't a safe assumption, NTP > Classic would have been changed to remove it decades ago. Uh, look at the (missing) commit history. You will see that interface has been patched, over and over again. And I can not beleive you hold up NTP Classic as a model to emulate! > (Note to self: Revise "The Lost Art of C Structure Packing" to reflect > the NTP field evidence that this has not been an issue since > approximately forever.) Or do a listserver and commit searchs and see you are wrong. You yourself added memory barriers, which only work on some OS. > Word-length mismatch between two programs built under the same OS > never happens, or close enough to never that I don't care. Uh, no. remember when intel OS went from 32 bit to 64-bit? It was a huge issue with ntpd. RasPi is about to have the same problems. HIstory shows us it keeps happening, history repeats itsef. > Not > willing to eat more jitter to armor against that extremely remote > contingency. What jitter? DO I really need to keep repeating mmyself. There is NO added jitter in the exiting ntpd/chrony schemes. There IS packet loss and polling overhead. ntpd sits on the numbers for 64 seconds by default before looking at them! Checkout the code for minpoll and maxpoll, you'll see. All the jitter is frozen in place before gpsd gives the numerbs to ntpd. > Have we on GPSD ever had an actual bug report due to these causes? I > know the answer, and it's a big "no". Since gpsd is not big on the bug tracker thing, you are correct, narrowly. But, loook in the git history, and you will see many patches. Look on the list server archives for the complaints. > > Third, you pretty much need to reboot the server when you change > > the struct size as deleting the old SHM segment while running is > > problematic. > > Not true, we have a recipe for handling this case without rebooting in > the GPSD Time Service HOWTO. Yeah, like how many people get that far? If we can'd do it automagically, then we should not do it! POSIX SHM can, if we work with it. > > Fourth, teaching people to debug with ipcs is a PITA. > > True, but pretty minor stuff. Cases where you have to do that kind of > debugging are rare. But often enough cause users to give up. This is exaclty why you had to write ntpshmmon, because it is hard. > > Fifth, security is hopeless. Client and server both need to run as > > root to be somewhat safe and only 2 SHM slots can do that. After > > that the SHMs are wide open to anyone on the server to mess with. > > I think you're being over-dramatic here. Yes, there is a theoretical > hole here but no evidence that it's ever been exploited. "Hopeless" > would be if there were a history of such attacks. OK, can we agree on big gaping hole? Do we really need exploits in the wild to fix a potential vulnerability? Especially when chrony does not have the same vulnnerability? > Is this a contingent problem with the way permissions are set > by the exiting driver? If so, we can fix it in a new one. POSIX SHM can do it right, the question was what are the known problems with the existing driver. > > Sixth, so many of these problems are so hard to grasp that people > > get amnesia about them after on eday. > > > > Seventh... well. I could go on but I need more espresso, and the > > following should be more than enough to show the current SHM needs > > killing off. Sadly that will take a decade, after the next good > > solution is implemented. > > You seem to have mixed together at
Re: Proposal - drop the GPSD JSON driver
e...@thyrsus.com said: >> Yes, I've been thinking about mechanisms for this. They're all rather >> irrelevant. > Argh. I meant "inelegant". I assume that using a pipe or socket rather than SHM would fix that. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
e...@thyrsus.com said: >> The SHM interface is deltas. gpsd figures out the offset and sends that >> over to ntpd. How gpsd figures out that offset has nothing to do with SHM >> or JSON or anything else involved with getting the data to ntpd. The offset >> won't change much in a short time so modest latency doesn't matter. > Your last sentence is one of the assumptions we're now questioning. I > didn't question it before either, which is why the above hack is not > implemented. We can test it with the current code. Just setup ntpd to use both SHM and JSON. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Eric S. Raymond : > Hal Murray : > > sem_wait doesn't work with select. You would have to do something like > > setup > > another thread and have it send a byte down a pipe. (The DNS lookup path > > does that. It may be on end-of-thread rather than data-ready.) > > Yes, I've been thinking about mechanisms for this. They're all rather > irrelevant. Argh. I meant "inelegant". -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Hal Murray : > sem_wait doesn't work with select. You would have to do something like setup > another thread and have it send a byte down a pipe. (The DNS lookup path > does that. It may be on end-of-thread rather than data-ready.) Yes, I've been thinking about mechanisms for this. They're all rather irrelevant. > > There's an easy way to take GPSD's heavy computation work out of the budget > > that we haven't used yet. You take a nanosecond-precision timestamp on the > > first read of data from the device. You take a second timestamp just before > > you ship. You bump the fix time by the difference between the second and > > first timestamps. > > I can't figure out what you are trying to do. > > The SHM interface is deltas. gpsd figures out the offset and sends that over > to ntpd. How gpsd figures out that offset has nothing to do with SHM or JSON > or anything else involved with getting the data to ntpd. The offset won't > change much in a short time so modest latency doesn't matter. Your last sentence is one of the assumptions we're now questioning. I didn't question it before either, which is why the above hack is not implemented. As you say, we need to try several methods and measure. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
e...@thyrsus.com said: > Hm. That's an argument for something with a wait or ready signal. Not an > argument for JSON in SHM. Might mean we need an auxiliary semaphore to do > sem_wait on; not a problem. I was never interested in JSON in SHM. sem_wait doesn't work with select. You would have to do something like setup another thread and have it send a byte down a pipe. (The DNS lookup path does that. It may be on end-of-thread rather than data-ready.) > There's an easy way to take GPSD's heavy computation work out of the budget > that we haven't used yet. You take a nanosecond-precision timestamp on the > first read of data from the device. You take a second timestamp just before > you ship. You bump the fix time by the difference between the second and > first timestamps. I can't figure out what you are trying to do. The SHM interface is deltas. gpsd figures out the offset and sends that over to ntpd. How gpsd figures out that offset has nothing to do with SHM or JSON or anything else involved with getting the data to ntpd. The offset won't change much in a short time so modest latency doesn't matter. I'd expect the latency to get interesting if you are doing something like minpoll=0. With longer polling the "latency" between collecting the data and actually using it is already over a second except for the last sample. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Gary E. Miller : > > What sort of problems are there with the current one? > > Do we really need to go over this yet again? These have been covered > for years. > > First, the structure needs to be polled, sometimes leading to long > delayed and even missed samples. This will be the case with any shared-memory driver that doesn't have a wait semaphore. Which means that a wait semaphore is a good idea, but is orthogonal to the struct-vs.-JSON-question to put in the box. You seem to have changed the subject from "the existing SHM sriver is bad" to "any shared-memory driver would be bad". Do you believe that? > Second, a big one, the C structure is of loosely defined size and shape. > Even on the same host you need to use the same compiler and word length > for client and server. For example, you can't compile the client as 32 > bit and the server as 64-bit. No sale. You don't need to use the same compiler, just two compilers with self-aligned padding. If this weren't a safe assumption, NTP Classic would have been changed to remove it decades ago. (Note to self: Revise "The Lost Art of C Structure Packing" to reflect the NTP field evidence that this has not been an issue since approximately forever.) Word-length mismatch between two programs built under the same OS never happens, or close enough to never that I don't care. Not willing to eat more jitter to armor against that extremely remote contingency. Have we on GPSD ever had an actual bug report due to these causes? I know the answer, and it's a big "no". > Third, you pretty much need to reboot the server when you change the struct > size as deleting the old SHM segment while running is problematic. Not true, we have a recipe for handling this case without rebooting in the GPSD Time Service HOWTO. > Fourth, teaching people to debug with ipcs is a PITA. True, but pretty minor stuff. Cases where you have to do that kind of debugging are rare. > Fifth, security is hopeless. Client and server both need to run as > root to be somewhat safe and only 2 SHM slots can do that. After that > the SHMs are wide open to anyone on the server to mess with. I think you're being over-dramatic here. Yes, there is a theoretical hole here but no evidence that it's ever been exploited. "Hopeless" would be if there were a history of such attacks. Is this a contingent problem with the way permissions are set by the exiting driver? If so, we can fix it in a new one. > Sixth, so many of these problems are so hard to grasp that people > get amnesia about them after on eday. > > Seventh... well. I could go on but I need more espresso, and the following > should be more than enough to show the current SHM needs killing off. > Sadly that will take a decade, after the next good solution is > implemented. You seem to have mixed together at least three different categories of objections. One is to the existing SHM driver. A second is to any driver based on a shared-memory drop with structure passing. A third is to any shared-memory driver at all. It would be helpful if you separated these more cleanly. -- http://www.catb.org/~esr/";>Eric S. Raymond signature.asc Description: PGP signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Eric! On Thu, 20 Oct 2016 07:11:20 -0400 "Eric S. Raymond" wrote: > Gary E. Miller : > > Oh, please not! That is one of the big problems in the existing > > driver. Way too compiler, CPU and OS dependent. Plus not > > extensible. The exsting C struct has been nothing but problems. > > Maybe you don't buy into JSON, but the C struct is a non-starter. > > Way too compiler, CPU and OS dependent? > > Dependent on lots of things, but important variables (endianness, > word length, CPU, OS) are coupled on either side because the sending > and receiving processes are running in the same memory space on the > same hardware. You would think so, but years of expericne with gpsd and ntpd show otherwise. This problem rears its ugly head several times a year. > Plus not extensible? > > As Hal points out, if you version-number the struct properly you can > add fields at the end ad libitum. IFF you do it right, AND chose a transport that can handle varying lengths. Traditional SHM can not do that. The current NTP SHM can not do that. But that still makes it a PITA to debug. Just look at your recent experience with pyntpq. You realized how much easier the ascii query was to write than the binary one. QED > The existing C struct has been nothing but problems? OK, list them > for me. I'm not seeing it. See my recent message to Hal, I stopped at 6, but coulda gone on... RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpqNBRcK7Fs0.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Eric! On Thu, 20 Oct 2016 07:34:18 -0400 "Eric S. Raymond" wrote: > Hal Murray : > > > > e...@thyrsus.com said: > > > Same thing with JSON. The client can't predict when JSON will > > > come up the wire. > > > > The client is normally waiting in the big select. It will get > > woken up as soon as data is available. > > Oh, now I see what you mean. You're right. > > Hm. That's an argument for something with a wait or ready signal. > Not an argument for JSON in SHM. Might mean we need an auxiliary > semaphore to do sem_wait on; not a problem. Huh? POSIX in SHM has a semaphore! So there is an argument for JSON in POSIX SHM. > There's an easy way to take GPSD's heavy computation work out of the > budget that we haven't used yet. Uh, what heavy commputation? Prolly less than 100 LOC from KPPS to done out the door to ntpd. > You take a nanosecond-precision > timestamp on the first read of data from the device. You take a second > timestamp just before you ship. You bump the fix time by the > difference between the second and first timestamps. Oh, you mean like itt already does? > You might be right that the noise from JSON encoding/decoding time > does not add significantly to the jitter budget. We should test and > see. NO added noise. Obvious by inspection of the protocol. > > The problem with semaphores is that I don't see a clean way to > > initialize them and/or recover from crashes. > > Pre-POSIX SHM has the same problem and the same mitigation > strategies. It hasn't been a problem in practice. I disagree. PSOIX HSM is much better thought out. > > The advantage of a pipe or socket is that they fit in to the select > > we have. I don't believe the cycles associated with either will be > > significant, but I don't have numbers to back that up. > > > > Maybe we should start a side project to write some clean sample > > code an/or collect some timings. > > You are quite right. We should do exactly that. So, we are bock to JSON (or something not binary) in POSIX SHM? I suggest after 1.0 given the uncertainies... RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpmAggIVoiez.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Hal! On Thu, 20 Oct 2016 03:38:00 -0700 Hal Murray wrote: > g...@rellim.com said: > > Not a POSIX sharde memory one. > > What do you mean by POSIX in this context? How would I recognize one > if I was looking at it? We are talking about the shared memory functions that POSIX defines. read this: man 7 shm_orverview > > Say what? The existing protocol has serious problems. > > What sort of problems are there with the current one? Do we really need to go over this yet again? These have been covered for years. First, the structure needs to be polled, sometimes leading to long delayed and even missed samples. Second, a big one, the C structure is of loosely defined size and shape. Even on the same host you need to use the same compiler and word length for client and server. For example, you can't compile the client as 32 bit and the server as 64-bit. Third, you pretty much need to reboot the server when you change the struct size as deleting the old SHM segment while running is problematic. Fourth, teaching people to debug with ipcs is a PITA. Fifth, security is hopeless. Client and server both need to run as root to be somewhat safe and only 2 SHM slots can do that. After that the SHMs are wide open to anyone on the server to mess with. Sixth, so many of these problems are so hard to grasp that people get amnesia about them after on eday. Seventh... well. I could go on but I need more espresso, and the following should be more than enough to show the current SHM needs killing off. Sadly that will take a decade, after the next good solution is implemented. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgppj6E6Adon3.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Hal Murray : > > e...@thyrsus.com said: > > Same thing with JSON. The client can't predict when JSON will come up the > > wire. > > The client is normally waiting in the big select. It will get woken up as > soon as data is available. Oh, now I see what you mean. You're right. Hm. That's an argument for something with a wait or ready signal. Not an argument for JSON in SHM. Might mean we need an auxiliary semaphore to do sem_wait on; not a problem. > > JSON reflects one set of priorties: extensibility of the protocol and > > eyeball-friendless are more important than minimizing latency or jitter in > > most cases. > > But it doesn't add significant latency or jitter. Latency isn't critical in > the below 1 ms range. There is much more latency in SHM. Where do you think > the jitter is coming from? All the time critical work is done in gpsd. It > sends deltas. Interesting assertions. I see what you mean about the polling latency now. There's an easy way to take GPSD's heavy computation work out of the budget that we haven't used yet. You take a nanosecond-precision timestamp on the first read of data from the device. You take a second timestamp just before you ship. You bump the fix time by the difference between the second and first timestamps. You're no longer shipping raw device time, but that's OK. You're shipping GPSD's assertion that it belives clock time was X at system time Y, and the maximum distortion is due to a constant number of simple arithmetic instructions. You might be right that the noise from JSON encoding/decoding time does not add significantly to the jitter budget. We should test and see. > > but the same amount of time spent on (say) a POSIX SHM driver would have > > paid off better > > What does "POSIX SHM" mean? > > I think there are two parts to SHM. One is setting up a chunk of shared > memory. The other is passing data from one process to another. That part > needs memory barriers or semaphores or something like that. > > My man page for shmget says: > shmget - allocates a System V shared memory segment > CONFORMING TO >POSIX.1-2001, POSIX.1-2008, SVr4. > > Is that POSIX? Yes. > The problem with semaphores is that I don't see a clean way to initialize > them and/or recover from crashes. Pre-POSIX SHM has the same problem and the same mitigation strategies. It hasn't been a problem in practice. > The advantage of a pipe or socket is that they fit in to the select we have. > I don't believe the cycles associated with either will be significant, but I > don't have numbers to back that up. > > Maybe we should start a side project to write some clean sample code an/or > collect some timings. You are quite right. We should do exactly that. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Gary E. Miller : > Oh, please not! That is one of the big problems in the existing driver. > Way too compiler, CPU and OS dependent. Plus not extensible. The > exsting C struct has been nothing but problems. Maybe you don't buy into > JSON, but the C struct is a non-starter. Way too compiler, CPU and OS dependent? Dependent on lots of things, but important variables (endianness, word length, CPU, OS) are coupled on either side because the sending and receiving processes are running in the same memory space on the same hardware. I can see a theoretical possibility that we could get messed over if ntpd and its symbiont were built with different compilers. But ntpd convinces me this is not a problem in practice; look at the old version of struct pkt. Turns out ntpd got away for over a quarter century with the assumption that, on every host, struct fields are self-alligned with minimal padding. So much for that supposed variable (yes, this surprised me too). Plus not extensible? As Hal points out, if you version-number the struct properly you can add fields at the end ad libitum. The existing C struct has been nothing but problems? OK, list them for me. I'm not seeing it. -- http://www.catb.org/~esr/";>Eric S. Raymond signature.asc Description: PGP signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Hal Murray : > > e...@thyrsus.com said: > >> So, we all vote for JSON in POSIX SHM? > > I don't. I think this is one of the very rare cases where passing a C stru= > > ct as a binary blob makes sense. We want to minimize offset and jitter and > > there's no cross-architecture problem. > > Don't forget to include a version number. Or maybe 2. First field, or fields. > There is still a nasty problem of how to change the contents of that struct. > In general, you can't assume that you can update both ends at the same time. I expect changes to be rare. But yes. > If you normally add fields, I think you can swap in new code on the source > side as long as the version info is handled properly. I think you need 2 > version numbers. One is the current version. The other is the oldest > version that will work. That's a good plan. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
g...@rellim.com said: > Not a POSIX sharde memory one. What do you mean by POSIX in this context? How would I recognize one if I was looking at it? > Say what? The existing protocol has serious problems. What sort of problems are there with the current one? -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Eric! On Thu, 20 Oct 2016 00:02:38 -0400 "Eric S. Raymond" wrote: > Gary E. Miller : > > So, a shared memory driver for ntpd? > > It already has one. Not a POSIX sharde memory one. And the exsting one has serious problems. > I think the one it has would be as good as we > need if not for the fact that that version of SHM is obsolete and > unstandardized. Say what? The existing protocol has serious problems. > > > I don't think either is going to match the performance of a > > > shared-memory drop. > > > > So, we all vote for JSON in POSIX SHM? > > I don't. I think this is one of the very rare cases where passing a > C struct as a binary blob makes sense. We want to minimize offset > and jitter and there's no cross-architecture problem. Oh, please not! That is one of the big problems in the existing driver. Way too compiler, CPU and OS dependent. Plus not extensible. The exsting C struct has been nothing but problems. Maybe you don't buy into JSON, but the C struct is a non-starter. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpXQLAu0Mp24.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
e...@thyrsus.com said: >> So, we all vote for JSON in POSIX SHM? > I don't. I think this is one of the very rare cases where passing a C stru= > ct as a binary blob makes sense. We want to minimize offset and jitter and > there's no cross-architecture problem. Don't forget to include a version number. Or maybe 2. There is still a nasty problem of how to change the contents of that struct. In general, you can't assume that you can update both ends at the same time. If you normally add fields, I think you can swap in new code on the source side as long as the version info is handled properly. I think you need 2 version numbers. One is the current version. The other is the oldest version that will work. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
e...@thyrsus.com said: > Same thing with JSON. The client can't predict when JSON will come up the > wire. The client is normally waiting in the big select. It will get woken up as soon as data is available. > Originally GPSD JSON wasn't really supposed to be an externally exposed > protocol at all. GPSD ships both C and Python libraries to avoid this; I > wanted control of both ends so the protocol should change if it needed to. Ahh. OK. That's another interesting can of worms. > JSON reflects one set of priorties: extensibility of the protocol and > eyeball-friendless are more important than minimizing latency or jitter in > most cases. But it doesn't add significant latency or jitter. Latency isn't critical in the below 1 ms range. There is much more latency in SHM. Where do you think the jitter is coming from? All the time critical work is done in gpsd. It sends deltas. > but the same amount of time spent on (say) a POSIX SHM driver would have > paid off better What does "POSIX SHM" mean? I think there are two parts to SHM. One is setting up a chunk of shared memory. The other is passing data from one process to another. That part needs memory barriers or semaphores or something like that. My man page for shmget says: shmget - allocates a System V shared memory segment CONFORMING TO POSIX.1-2001, POSIX.1-2008, SVr4. Is that POSIX? The problem with semaphores is that I don't see a clean way to initialize them and/or recover from crashes. The advantage of a pipe or socket is that they fit in to the select we have. I don't believe the cycles associated with either will be significant, but I don't have numbers to back that up. Maybe we should start a side project to write some clean sample code an/or collect some timings. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Gary E. Miller : > So, a shared memory driver for ntpd? It already has one. I think the one it has would be as good as we need if not for the fact that that version of SHM is obsolete and unstandardized. > > I don't think either is going to match the performance of a > > shared-memory drop. > > So, we all vote for JSON in POSIX SHM? I don't. I think this is one of the very rare cases where passing a C struct as a binary blob makes sense. We want to minimize offset and jitter and there's no cross-architecture problem. -- http://www.catb.org/~esr/";>Eric S. Raymond signature.asc Description: PGP signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Eric! On Wed, 19 Oct 2016 21:30:28 -0400 "Eric S. Raymond" wrote: > Hal Murray : > > It has a different type of noise. There is no "ready" signal on > > SHM so the driver polls and the timing on that is just the luck of > > when ntpd was started. > > Same thing with JSON. The client can't predict when JSON will come > up the wire. Yes, but it can block until it happens. With SHM it has to stay away and poll. > NTP is a very different case. Minimizing jitter in the delivery chain > is all-important, extensibility and eyeball-friendliness matter barely > at all. The overhead of formatting the JSON at one end and decoding > at the other should be avoided if it can be. I'm not sure why the jitter is important? The timing dependent stuff is done before the JSON is sent. > You and others modeled my thought processes incorrectly. I left a > clue to what I was actually thinking in that communication between > GPSD and its client libraries can actually have any of three > transports - JSON, DBUS, or shared memory. I put in the > shared-memory transport specifically for use by real-time embedded > systems. So, a shared memory driver for ntpd? > From my point of view, the JSON driver in ntpd is a cute stunt that > doesn't really fit the context very well. I'm mildly flattered that > somebody thought GPSD JSON was cool enough to want to play with, but > the same amount of time spent on (say) a POSIX SHM driver would have > paid off better. Hmm, I thought I mentioned JSON encoding inside the POSIX SHM driver? Or you want to invent anotehr wheel? > > Would pipes work better? Or a network connection passing the same > > (binary) info as is in the SHM segment? > > As Gary just found out in connection with ntpviz, pipes pretty much > suck for this kind of work due to unpredictable buffering overhead. > I don't know whether or not that would be true of sockets. Yeah, major suckage. > I don't think either is going to match the performance of a > shared-memory drop. So, we all vote for JSON in POSIX SHM? RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp3DgmeifOTr.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Hal Murray : > It has a different type of noise. There is no "ready" signal on SHM so the > driver polls and the timing on that is just the luck of when ntpd was started. Same thing with JSON. The client can't predict when JSON will come up the wire. > I'm missing the big picture. The JSON driver was written because I (and > probably others) thought you thought that JSON was the right way to talk to > gpsd. Was that wrong and/or did anything change? That's complicated. Originally GPSD JSON wasn't really supposed to be an externally exposed protocol at all. GPSD ships both C and Python libraries to avoid this; I wanted control of both ends so the protocol should change if it needed to. Nowadays, if only because there are language bindings GPSD doesn't ship, I accept that other people are going to pull JSON off the wire and decode it. JSON reflects one set of priorties: extensibility of the protocol and eyeball-friendless are more important than minimizing latency or jitter in most cases. GPSD clients are not in general very sensitive to those - or, to put it differently, their sensitivity tends to be closer to human scale (0.1sec and up) than machine scale. NTP is a very different case. Minimizing jitter in the delivery chain is all-important, extensibility and eyeball-friendliness matter barely at all. The overhead of formatting the JSON at one end and decoding at the other should be avoided if it can be. You and others modeled my thought processes incorrectly. I left a clue to what I was actually thinking in that communication between GPSD and its client libraries can actually have any of three transports - JSON, DBUS, or shared memory. I put in the shared-memory transport specifically for use by real-time embedded systems. >From my point of view, the JSON driver in ntpd is a cute stunt that doesn't really fit the context very well. I'm mildly flattered that somebody thought GPSD JSON was cool enough to want to play with, but the same amount of time spent on (say) a POSIX SHM driver would have paid off better. > Can POSIX SHM talk to non-POSIX SHM? I assume the concepts are the same (map > a chunk of memory where I can see it). Are there different name spaces? There are. I looked at this and wasn't able to find a way to make it work. I'll poke at it again. > Would pipes work better? Or a network connection passing the same (binary) > info as is in the SHM segment? As Gary just found out in connection with ntpviz, pipes pretty much suck for this kind of work due to unpredictable buffering overhead. I don't know whether or not that would be true of sockets. I don't think either is going to match the performance of a shared-memory drop. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
I think you should leave it until we get our act together, and I think you should put that on the back burner until 1.0 is out. > For communication with GPSD, the SHM driver seems superior; it certainly has > lower processing overhead and therefore introduces less noise into the > delivery chain. It has a different type of noise. There is no "ready" signal on SHM so the driver polls and the timing on that is just the luck of when ntpd was started. I think the SHM driver is pretty ugly. It got a lot less ugly when you added the memory barriers a year or two ago. I think we should rewrite it to use the 2 counters trick so the readers are read-only. But let's save that discussion for another time. I'm missing the big picture. The JSON driver was written because I (and probably others) thought you thought that JSON was the right way to talk to gpsd. Was that wrong and/or did anything change? Can POSIX SHM talk to non-POSIX SHM? I assume the concepts are the same (map a chunk of memory where I can see it). Are there different name spaces? Would pipes work better? Or a network connection passing the same (binary) info as is in the SHM segment? -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Proposal - drop the GPSD JSON driver
Yo Eric! On Mon, 17 Oct 2016 18:28:22 -0400 (EDT) e...@thyrsus.com (Eric S. Raymond) wrote: > I have come to think the best thing to do about the GPSD/JSON driver > is just remove it. This note explains why. Please reply with > agreement, disagreement, or comment. Disagree. The defaults are broken, and it has modes that can never work, but it is fixable. > Because it works so badly, it is highly unlikely that anyone will > complain of its absence. Probably true, but not a reason not to fix it. > There might be an argument for fixing this driver rather than > dropping it, but I don't see that it actually does anything useful. Not yet, but the SHM driver also has problems. SHM has security problems, it is not extensible, it provides no synergy between the GPS and PPS signals. One nice extention to the JSON refclock might be to have it autostart gpsd if required. The GPSD JSON driver has documented 4 modes and undocumented 'features'. That html documentation for those modes does not match what the code does. Bad doc is not a reason to remove the driver. If ntpd forced the driver into the one configuration that actually works (just remove the bad options), then fix the doc and it is good to go. > For communication with GPSD, the SHM driver seems superior; it > certainly has lower processing overhead and therefore introduces > less noise into the delivery chain. It introduces no extra timing noise and passes some data, like timing source name, tdop, nSats, etc. that could be useful to ntpd. > I understand there's a port problem with SHM on the Mac, but I'd > rather fix that or write an equivalent driver using new POSIX > shared memory than chase bugs in this one. Seems to me any POSIX shared memory driver, which I would like to see, would basically have to implement the GPSD JSON protocol to work. We know the GPSD JSON is a good protocol so no need to come up with a new one. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpk65abb1f7D.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
