Re: [chrony-users] DNS RR and chrony

[Bryan Christianson]

> On 18/03/2016, at 6:34 AM, Miroslav Lichvar  wrote:
> 
> 
> If you start chronyc with -c and write commands to the stdin, the
> tracking command will still print the report in the CSV format.

Oh - Thats cool thanks :)

> 
>> Would it be possible to add an optional argument to the internal chronyc 
>> commands to have them also return CSV format? Optionally JSON and XML would 
>> be nice formats as well.
> 
> Would JSON or XML be easier for you to parse? I guess it wouldn't be
> difficult to implement, I'm just not sure if it's worth the extra
> code.

Its semantically better - means the client code is not dependent on the field 
order but can just do lookup by name once the data is loaded. Anyway - its not 
essential, so up to you.

Thanks
B

— 
Bryan Christianson
br...@whatroute.net




--
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] DNS RR and chrony

[Bryan Christianson]

> On 18/03/2016, at 4:30 AM, Miroslav Lichvar  wrote:
> 
> 
> Also, chronyc now has a -c option to print reports in a CSV format,
> all values in the same units. It should be much easier to parse if
> anyone still needs to do that.
> 

The -c output  from chronyc looks really good and useful.

My OSX program, ChronyControl, runs a persistent connection to chronyc and 
sends it a ‘tracking’ command every (user configurable) few seconds. Currently 
this is of course parsing the display format of the tracking command.

To get CSV output I would have to run a new instance of chronic every few 
seconds and this seems like a lot of overhead.

Would it be possible to add an optional argument to the internal chronyc 
commands to have them also return CSV format? Optionally JSON and XML would be 
nice formats as well.

--
Bryan Christianson
br...@whatroute.net




--
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] DNS RR and chrony

[Ben Kochie]

On Thu, Mar 17, 2016 at 8:04 PM, Bryan Christianson 
wrote:

>
> > On 18/03/2016, at 6:34 AM, Miroslav Lichvar  wrote:
> >
> >
> > If you start chronyc with -c and write commands to the stdin, the
> > tracking command will still print the report in the CSV format.
>
> Oh - Thats cool thanks :)
>
> >
> >> Would it be possible to add an optional argument to the internal
> chronyc commands to have them also return CSV format? Optionally JSON and
> XML would be nice formats as well.
> >
> > Would JSON or XML be easier for you to parse? I guess it wouldn't be
> > difficult to implement, I'm just not sure if it's worth the extra
> > code.
>
> Its semantically better - means the client code is not dependent on the
> field order but can just do lookup by name once the data is loaded. Anyway
> - its not essential, so up to you.
>

I think implementing client libraries that can replace chronyc are a better
direction.  Or if chronyd were to implement a more standard protocol
library like http://www.grpc.io/ things would be easier to automate and
instrument.


>
> Thanks
> B
>
> —
> Bryan Christianson
> br...@whatroute.net
>
>
>
>
> --
> To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org
> with "unsubscribe" in the subject.
> For help email chrony-users-requ...@chrony.tuxfamily.org
> with "help" in the subject.
> Trouble?  Email listmas...@chrony.tuxfamily.org.
>
>

Re: [chrony-users] DNS RR and chrony

[Miroslav Lichvar]

On Fri, Mar 18, 2016 at 05:27:19AM +1300, Bryan Christianson wrote:
> The -c output  from chronyc looks really good and useful.
> 
> My OSX program, ChronyControl, runs a persistent connection to chronyc and 
> sends it a ‘tracking’ command every (user configurable) few seconds. 
> Currently this is of course parsing the display format of the tracking 
> command.
> 
> To get CSV output I would have to run a new instance of chronic every few 
> seconds and this seems like a lot of overhead.

If you start chronyc with -c and write commands to the stdin, the
tracking command will still print the report in the CSV format.

> Would it be possible to add an optional argument to the internal chronyc 
> commands to have them also return CSV format? Optionally JSON and XML would 
> be nice formats as well.

Would JSON or XML be easier for you to parse? I guess it wouldn't be
difficult to implement, I'm just not sure if it's worth the extra
code.

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org 
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org 
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] DNS RR and chrony

[Ben Kochie]

The CSV format looks great, I also tested the randomization which works
well.

The -v help output does't match the CSV, but that's a minor nit-pick. :)

Thanks for looking into this.  I'm still interested in writing a chrony
client in Go or Python so I can create a fully integrated metrics exporter.

- Ben Kochie

On Thu, Mar 17, 2016 at 4:30 PM, Miroslav Lichvar 
wrote:

> On Thu, Feb 25, 2016 at 12:51:49PM +0100, Miroslav Lichvar wrote:
> > On Wed, Feb 24, 2016 at 04:18:36PM +0100, Ben Kochie wrote:
> > > The end result here is that getaddrinfo() always sorts the output of
> IPv4
> > > results and chrony will pick the first N in that list.  For example I
> have
> > > a DNS record internally that has 8 servers, and I have chrony pick 4.
> > > Every node has the same identical 4 node list instead of a random
> sampling
> > > of the 4.
>
> > In order to fix that, I guess we need to either modify the hash
> > function to include some random variable initialized on start to make
> > the hashing random, or schedule the first poll of the sources in a
> > random order.
>
> The randomization of sources in the hash table is now implemented in
> git if you would like to try it.
>
> Also, chronyc now has a -c option to print reports in a CSV format,
> all values in the same units. It should be much easier to parse if
> anyone still needs to do that.
>
> --
> Miroslav Lichvar
>
> --
> To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org
> with "unsubscribe" in the subject.
> For help email chrony-users-requ...@chrony.tuxfamily.org
> with "help" in the subject.
> Trouble?  Email listmas...@chrony.tuxfamily.org.
>
>

Re: [chrony-users] DNS RR and chrony

[Miroslav Lichvar]

On Thu, Feb 25, 2016 at 12:51:49PM +0100, Miroslav Lichvar wrote:
> On Wed, Feb 24, 2016 at 04:18:36PM +0100, Ben Kochie wrote:
> > The end result here is that getaddrinfo() always sorts the output of IPv4
> > results and chrony will pick the first N in that list.  For example I have
> > a DNS record internally that has 8 servers, and I have chrony pick 4.
> > Every node has the same identical 4 node list instead of a random sampling
> > of the 4.

> In order to fix that, I guess we need to either modify the hash
> function to include some random variable initialized on start to make
> the hashing random, or schedule the first poll of the sources in a
> random order.

The randomization of sources in the hash table is now implemented in
git if you would like to try it.

Also, chronyc now has a -c option to print reports in a CSV format,
all values in the same units. It should be much easier to parse if
anyone still needs to do that.

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org 
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org 
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] DNS RR and chrony

[Miroslav Lichvar]

On Wed, Feb 24, 2016 at 04:18:36PM +0100, Ben Kochie wrote:
> The end result here is that getaddrinfo() always sorts the output of IPv4
> results and chrony will pick the first N in that list.  For example I have
> a DNS record internally that has 8 servers, and I have chrony pick 4.
> Every node has the same identical 4 node list instead of a random sampling
> of the 4.

Do the addresses have the same scope and type? I think getaddrinfo()
is supposed to sort only by that and not the actual addresses. With
glibc the precedence can be configured in gai.conf(5). If the
addresses have the same precedence, the order should be as was
received from the DNS server.

I suspect the problem you see is rather due to the way chronyd starts
polling NTP sources. With the pool directive, all addresses (up to 16)
are added as sources and the first four that respond are kept. But the
order in which they are polled depends on how they are sorted in a
hash table. Even if the sources are added to the hash table in a
random order, they will end up in the same slots (assuming there are
no collisions) and will be polled in the same order.

In order to fix that, I guess we need to either modify the hash
function to include some random variable initialized on start to make
the hashing random, or schedule the first poll of the sources in a
random order.

As a workaround you could try four server directives using the same
hostname instead of one pool. With that, the selection of the servers
should depend on getaddrinfo() and not the hashing.

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org 
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org 
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

[chrony-users] DNS RR and chrony

[Ben Kochie]

When using pools in the config, chrony is subject to some implementation
"problems" with libc's getaddrinfo() on many platforms.  This breaks DNS
round-robin as served by the DNS server.

There is a long standing "bug" in several libc implementations due to
strict adherence to RFC 3484 Rule #9.  There were many long arguments about
this in the 2007 era, with no resolution.

Thankfully RFC 6724 obsoletes 3484, but nobody's implemented it yet, and
it's not likely to get backported to stable distributions like Debian.

The end result here is that getaddrinfo() always sorts the output of IPv4
results and chrony will pick the first N in that list.  For example I have
a DNS record internally that has 8 servers, and I have chrony pick 4.
Every node has the same identical 4 node list instead of a random sampling
of the 4.

It would be nice if chrony were have an option to shuffle the list before
selecting.

Something like this:
pool pool.ntp.org iburst maxsources 4 shuffle

[0]: https://tools.ietf.org/html/rfc6724

- Ben Kochie