Re: ntpq: cruft/doc
Gary E. Miller : > > Is there a good library/package to do the parsing? Do we want to add > > another library to the list of requirements? > > In PHP it is built in. Reading or writing JSON is a one line function call. > > To write, you just take a nested object and tell PHP to write it as > JSON. To read, PHP creates a nested object to clearly and cleanly > represent the input. Same in Python. But we're talking C here. -- http://www.catb.org/~esr/";>Eric S. Raymond signature.asc Description: PGP signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Yo Hal! On Mon, 17 Oct 2016 03:22:42 -0700 Hal Murray wrote: > Is there a good library/package to do the parsing? Do we want to add > another library to the list of requirements? In PHP it is built in. Reading or writing JSON is a one line function call. To write, you just take a nested object and tell PHP to write it as JSON. To read, PHP creates a nested object to clearly and cleanly represent the input. And it is well documented: http://php.net/manual/en/book.json.php RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpgYhs9xQhmv.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
e...@thyrsus.com said: > Harder than CSV, where unless someone is being conscientious (and it's > certainly not required in in RFC 4180) you get no self-descriptive clues at > *all*? Er...no. Ah. Sorry. A chunk of context got lost in this discussion. You are talking about the general case. I'm talking about the case where I know the data well. >> If you want to convince me that JSON is great, rewrite the >> gpsd refclock to be a shining example. > I can't figure out what you're recommending here, and I want to because I > believe in taking you seriously even when I think you've just said something > actually *silly* for the first time in recorded history. Cleaning up the > parsing code in the driver? Changing the way it's generated in gpsd? What's > the axis of "better" here? I'm not sure. Probably ease of understanding and/or maintaining. (You can change the gpsd end if that helps.) I think it's simple to write/send stuff encoded in JSON. I don't know how to do the read/recv side. Is there a good library/package to do the parsing? Do we want to add another library to the list of requirements? One of the things that's missing by just saying "JSON" and expecting all your troubles to be over is a layer of type checking. I hate discovering at run time something that should have been detected by the tool chain. For the case of a simple struct, I'd expect the sending side would be one line per field. (with probably another line each for start and finish) Can the recv side be as simple? I'm expecting each line to be a subroutine call with a subr for each data type. (int, float, text, bool...) Is there a tool that turns a struct into the appropriate set of calls? What are the rules for JSON? Is the order of fields part of the specs? Could I do the recv with a readline and a big sscanf? -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Hal Murray : > > e...@thyrsus.com said: > >> CSV is readable by eye without a lot of effort. JSON is close to > encrypted. > > > Say *what*? Uh, I can only conjecture that you don't have a lot of actual > > experience with JSON. > > I admit that "encrypted" is an exaggeration, but the signal-noise is pretty > low. Yes, if I'm hunting for a specific chunk of data I can find it, but > it's hard to get the big picture and know which chunk(s) of data I need to > pay attention to. Harder than CSV, where unless someone is being conscientious (and it's certainly not required in in RFC 4180) you get no self-descriptive clues at *all*? Er...no. > I admit that I haven't worked much with JSON. Sure, it's probably the right > thing for complicated data structures. But I'm mostly interested in simple > tabular data where CSV works well. > > If you want to convince me that JSON is great, rewrite the gpsd refclock to > be a shining example. I can't figure out what you're recommending here, and I want to because I believe in taking you seriously even when I think you've just said something actually *silly* for the first time in recorded history. Cleaning up the parsing code in the driver? Changing the way it's generated in gpsd? What's the axis of "better" here? I actually think GPSD protocol *is* a shining example. I struggled with that design problem for three years, trying to come up with a wire protocol with the right readability and extensibility properties. Extensibility is a huge deal here because given the problem domain you never know when you might have to add another object type on the wire, like AIS or PPS. "I can just use JSON" was like a lightning bolt when I finally got it. There's nothing better for structured, self-describing datagrams. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
e...@thyrsus.com said: >> CSV is readable by eye without a lot of effort. JSON is close to encrypted. > Say *what*? Uh, I can only conjecture that you don't have a lot of actual > experience with JSON. I admit that "encrypted" is an exaggeration, but the signal-noise is pretty low. Yes, if I'm hunting for a specific chunk of data I can find it, but it's hard to get the big picture and know which chunk(s) of data I need to pay attention to. I admit that I haven't worked much with JSON. Sure, it's probably the right thing for complicated data structures. But I'm mostly interested in simple tabular data where CSV works well. If you want to convince me that JSON is great, rewrite the gpsd refclock to be a shining example. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Yo Hal! On Sun, 16 Oct 2016 13:50:28 -0700 Hal Murray wrote: > g...@rellim.com said: > > I was unaware of RFC 4180, and it mandates CRLF as line ending, not > > the \n that I used. I just pushed a fix to this. Someone familiar > > with RFC4180 should check the format is correct. > > My guess is that's for network traffic rather than local files. If > you make a local file with your local line ending, the web stuff > should do the conversion when it sends out a text file. Too much work. I can't tell when the file is written how it will be used, and RFC 4180 makes no such exceptions. I'm just gonna do the RFC4180 version. Been a long time since I have had sompatibilty problems with CSVs. Spreadsheet programs are pretty forgiving now. > What does python do with \n? Nothing, just another character. > Another guess is that you shouldn't use \n but rather > writeline. Then I have to write line by line, very inefficient. It was trivial to just change \n to \r\n. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpqgjcJj1W0O.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
g...@rellim.com said: > And I can't count the number of times people have asked me what the units > are because they can't figure it out. You should not have to read pages of > docc to find the current units. But I shouldn't have to waste valuable screen real estate to tell you what the units are after you know the answer. > Check out chronyc, they do a lot of things much better than ntpq. I wasn't impressed. I can't tell the difference between "123.4ms" and 123.4us" at a quick glance. This may be one of those things where we will have to agree to disagree. On the other hand, those columns are reasonably wide so I won't be surprised if there is a decent solution that keeps both of us happy. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
g...@rellim.com said: > I was unaware of RFC 4180, and it mandates CRLF as line ending, not the \n > that I used. I just pushed a fix to this. Someone familiar with RFC4180 > should check the format is correct. My guess is that's for network traffic rather than local files. If you make a local file with your local line ending, the web stuff should do the conversion when it sends out a text file. What does python do with \n? Is that a literal \n, or an end of line marker? Another guess is that you shouldn't use \n but rather writeline. (at least when making csv type files that are intended for network use) -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Gary E. Miller : > And I can't count the number of times people have asked me what the > units are because they can't figure it out. You should not have to read > pages of docc to find the current units. > > Check out chronyc, they do a lot of things much better than ntpq. I readily believe both of these things. Your next Python challenge after ntpviz will probably be fixing them in pyntpq. -- http://www.catb.org/~esr/";>Eric S. Raymond signature.asc Description: PGP signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Yo Hal! On Sun, 16 Oct 2016 03:27:54 -0700 Hal Murray wrote: > I think it's nice if the columns line up. The only time that I > remember recently when the current stuff didn't work is that you > can't see what's going on when the offset is less than a microsecond. Yeah, it is nice when they line up, but much of the time either almost all the precision is lost for a local refclock, or the 8 digits of precision is pointless for a WAN pool chimer. And I can't count the number of times people have asked me what the units are because they can't figure it out. You should not have to read pages of docc to find the current units. Check out chronyc, they do a lot of things much better than ntpq. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpseQCDgXBiL.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Yo Eric! On Sun, 16 Oct 2016 15:14:19 -0400 "Eric S. Raymond" wrote: > > Many CSV files have a comment line on top with the field names. > > Good idea. Gonna be *mandatory* for anybody generating CSV in our > pond. Also strict conformance to RFC4180, at pain of my extreme > displeasure. Data formats that aren't well-standardized and > self-describing give me hives - they always, always, *always* cause > trouble down the road. This has sorta mutated to a data format discussion. ntpviz added a summary csv file in the last week. It has column titles in the first line. I was unaware of RFC 4180, and it mandates CRLF as line ending, not the \n that I used. I just pushed a fix to this. Someone familiar with RFC4180 should check the format is correct. I chose CSV over JSON because the data is simple 2D and most people will just import the summary.csv file into Libreoffice, or similar, directly from their browser. CSV files are trivial to import, but Libreoffice has no obvious way to directly import JSON. Here is a link to a current summary file: https://rellim.com/ntpstats/day/summary.csv RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpoE2wfI0LqI.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Hal Murray : > > >> The first question is What does "mobilize" mean? > > Is that a rhetorical question? > > No, unfortunately, that's a serious question. I think it has some fine-print > meaning that I don't yet understand. Damn, I was afraid you were going to say that. Because I share your suspicion. Add one to the list of mysteries to be plumbed. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Hal Murray : > > e...@thyrsus.com said: > > About CSV, while I'm not opposed to it we have a practice guideline to use > > JSON for machine-parseable output. One reason is that JSON is better at > > being self-describing - the field names give you clues that CSV doesn't. > > CSV is readable by eye without a lot of effort. JSON is close to encrypted. Say *what*? Uh, I can only conjecture that you don't have a lot of actual experience with JSON. > Many CSV files have a comment line on top with the field names. Good idea. Gonna be *mandatory* for anybody generating CSV in our pond. Also strict conformance to RFC4180, at pain of my extreme displeasure. Data formats that aren't well-standardized and self-describing give me hives - they always, always, *always* cause trouble down the road. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
>> The first question is What does "mobilize" mean? > Is that a rhetorical question? No, unfortunately, that's a serious question. I think it has some fine-print meaning that I don't yet understand. If it wasn't for this discussion, I would assume it meant something like create a peer struct, but they get created and returned via ntpq -p and then filtered out at the client end. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
e...@thyrsus.com said: > About CSV, while I'm not opposed to it we have a practice guideline to use > JSON for machine-parseable output. One reason is that JSON is better at > being self-describing - the field names give you clues that CSV doesn't. CSV is readable by eye without a lot of effort. JSON is close to encrypted. Many CSV files have a comment line on top with the field names. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Hal Murray : > The first question is What does "mobilize" mean? Is that a rhetorical question? If you know the answer, it should go in the documentation. If you don't, you should dig up the answer and then it should go in the documentation. > Nit: there are some typos in docs/includes/ntpq-body.txt for the peers > commands. Doubtless. A patch would be welcome. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Gary E. Miller : > > The other is the UI? Do you prefer fancy click-around GUIs, or > > dumb/simple CLI with text output? > > I'm for a nice little text output CLI tool. Mostly for quick data > visualization on very dummb servers. Maybe with an optional csv output > so it can be used in a tool chain. I'm with Gary on this one. Given a well-designed CLI tool you can write a GUI wrapper around it without much difficulty (and Python is a particularly good tool for this) but the reverse is not true. About CSV, while I'm not opposed to it we have a practice guideline to use JSON for machine-parseable output. One reason is that JSON is better at being self-describing - the field names give you clues that CSV doesn't. > The first thing I would do is auto scale the output. Hardly anyone > remembers the output of 'ntpq -p' is all in milli seconds. Even less > often is millis seconds the best range. I like how chronyc does the > auto ranging. Agreed. > Many of the pages should be mashed up. Data is just spread out all over > in a jumbled fashion. Recently here someone was complaining that to > look at 'rv' data in ntpq you had to first go find out the association > ID of a peer (with associations), even though you already had the IP or > refclock number. I agree, the ntpq UI is badly designed. But I don't think we can just throw it out. My plan is to move it to Python, and factor it properly so the grotty bits of the UI are properly separated from a querying engine which can in principle be cleaner and more orthogonal. This will require defining an object API in Python for the client end of ntpq queries. Then we will be able to use that layer to write both a better CLI tool and (possibly) a nice GUI as well, with minimal code investment in either. -- http://www.catb.org/~esr/";>Eric S. Raymond signature.asc Description: PGP signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
g...@rellim.com said: >> If you were starting over, what would you do? I think there are two >> parts to that question. One is the on-wire API. > I think we are stuck with that part. RFC's right? There is nothing that says we can't add new stuff. I've been happy with the shift from mode 7 (binary) to mode 6 (text). > Maybe with an optional csv output so it can be used in a tool chain. Can't most tools that take CSV also use white space for a separator? > The first thing I would do is auto scale the output. Hardly anyone > remembers the output of 'ntpq -p' is all in milli seconds. Even less often > is millis seconds the best range. I like how chronyc does the auto ranging. One problem with auto scaling is that you have to print the units and that eats space on a line that is already tight. I think it's nice if the columns line up. The only time that I remember recently when the current stuff didn't work is that you can't see what's going on when the offset is less than a microsecond. > Recently here someone was complaining that to look at 'rv' data in ntpq you > had to first go find out the association ID of a peer (with associations), > even though you already had the IP or refclock number. It shouldn't be hard to fix that. Just make the rv stuff on the server check the argument and if it looks like an IP address do a scan to find the right slot. There is some flag you can pass to the DNS lookup stuff that tells it to do numerical only. I don't know if that takes things like "1234567". If it does, we'll have to test for the whole argument being a valid number or add a flag to say the rest is an IP address. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Yo Hal! On Fri, 14 Oct 2016 21:06:49 -0700 Hal Murray wrote: > > But ntpq has likely never been used in any scripts, and is so > > broken in so many ways that I'd prefer to just start all over. > > That sounds like you won't object if I "experiment" with the current > code. Have at it. > If you were starting over, what would you do? I think there are two > parts to that question. One is the on-wire API. I think we are stuck with that part. RFC's right? > The other is the UI? Do you prefer fancy click-around GUIs, or > dumb/simple CLI with text output? I'm for a nice little text output CLI tool. Mostly for quick data visualization on very dummb servers. Maybe with an optional csv output so it can be used in a tool chain. The first thing I would do is auto scale the output. Hardly anyone remembers the output of 'ntpq -p' is all in milli seconds. Even less often is millis seconds the best range. I like how chronyc does the auto ranging. Many of the pages should be mashed up. Data is just spread out all over in a jumbled fashion. Recently here someone was complaining that to look at 'rv' data in ntpq you had to first go find out the association ID of a peer (with associations), even though you already had the IP or refclock number. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpjvaM5B9E_G.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
> But ntpq has likely never been used in any scripts, and is so broken in so > many ways that I'd prefer to just start all over. That sounds like you won't object if I "experiment" with the current code. If you were starting over, what would you do? I think there are two parts to that question. One is the on-wire API. The other is the UI? Do you prefer fancy click-around GUIs, or dumb/simple CLI with text output? -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: cruft/doc
Yo Hal! On Fri, 14 Oct 2016 17:35:16 -0700 Hal Murray wrote: > If preserving old patterns is more important than sane operations, > I'd suggest that the non-l versions print out a warning if they skip > any slots. (But that changes the UI too.) I hate changes in ntp.conf that mean old config files crashes, and keeping the log format similar is equally important. But ntpq has likely never been used in any scripts, and is so broken in so many ways that I'd prefer to just start all over. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp4mEsUz_eaS.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
