Re: [chrony-dev] New online/offline command

[Christian Ehrhardt]

On Thu, May 24, 2018 at 2:45 PM, Miroslav Lichvar 
wrote:

> On Thu, May 24, 2018 at 02:11:58PM +0200, Christian Ehrhardt wrote:
> > On auto-deployed config changes of virt systems (with plenty of switch
> > interfaces for example) I've seen three digits per second (rare, but
> > existing).
>
> And each one runs all NM dispatcher scripts? I suspect the bottleneck
> is in the execution of the scripts and the chronyc binary, not in
> handling of the request in chronyd. It takes just few system calls per
> source.
>

That was just the max of the event's I've seen.
I did not see them clogging up due to chrony yet.
I just wanted to share that these events can be frequent and to mind about
that in the design of your new command.


> Have you tried profiling?
>
> > secondevent
> > 1 net event 1 triggers "probe I"
> > 2 Events 2-20 trigger 19 more
> > ...
> > 7 probe I completes and returns to caller "event 1"
> > 8 the formerly waiting probe II (just one) runs now
> > ...
> > 14probe II completes and returns to all callers Events 2-20
> >
> > This is synchronous + batching as I'd think of it
>
> This sounds like a major complication of the code. If I understand it
> correctly, chronyd would need to cache responses and check when was a
> request received.
>

I wanted to raise awareness, not insist on implementing it.
If you are sure your code is rather fast also in huge setups then you are
good.
Just ensure that the code is either safe against concurrent
calling/execution or we might get odd rare and hard to debug issues.

> You wrote later:
> >   "How about "onoffline" or "maybeonline"?"
> >
> > I'd like auto-onoffline or something like it more than any of the two
> above
>
> That looks better, but would it be correct to call it "auto" when it
> works only once when the command is issued and not all the time,
> unlike the "auto_offline" option does for instance?
>

You put too much meaning in too much words already :-)
Without an auto- prefix I'd have wondered at first on which rational it
does the online/offline.
With auto-onoffline I felt better, but I see the similarity to auto-online
being an issue.

"check-onoffline" maybe, but that sounds as if it would be a read-only
check action.
If you want go with "onoffline", it is not too bad if we make sure the
manpage entry is clear enough.


> --
> Miroslav Lichvar
>
> --
> To unsubscribe email chrony-dev-requ...@chrony.tuxfamily.org with
> "unsubscribe" in the subject.
> For help email chrony-dev-requ...@chrony.tuxfamily.org with "help" in the
> subject.
> Trouble?  Email listmas...@chrony.tuxfamily.org.
>
>


-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

Re: [chrony-dev] New online/offline command

[Miroslav Lichvar]

On Thu, May 24, 2018 at 02:11:58PM +0200, Christian Ehrhardt wrote:
> On auto-deployed config changes of virt systems (with plenty of switch
> interfaces for example) I've seen three digits per second (rare, but
> existing).

And each one runs all NM dispatcher scripts? I suspect the bottleneck
is in the execution of the scripts and the chronyc binary, not in
handling of the request in chronyd. It takes just few system calls per
source.

Have you tried profiling?

> secondevent
> 1 net event 1 triggers "probe I"
> 2 Events 2-20 trigger 19 more
> ...
> 7 probe I completes and returns to caller "event 1"
> 8 the formerly waiting probe II (just one) runs now
> ...
> 14probe II completes and returns to all callers Events 2-20
> 
> This is synchronous + batching as I'd think of it

This sounds like a major complication of the code. If I understand it
correctly, chronyd would need to cache responses and check when was a
request received.

> You wrote later:
>   "How about "onoffline" or "maybeonline"?"
> 
> I'd like auto-onoffline or something like it more than any of the two above

That looks better, but would it be correct to call it "auto" when it
works only once when the command is issued and not all the time,
unlike the "auto_offline" option does for instance?

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-dev-requ...@chrony.tuxfamily.org with "unsubscribe" 
in the subject.
For help email chrony-dev-requ...@chrony.tuxfamily.org with "help" in the 
subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-dev] New online/offline command

[Christian Ehrhardt]

On Thu, May 24, 2018 at 11:54 AM, Miroslav Lichvar 
wrote:

> On Wed, May 23, 2018 at 04:44:30PM +0200, Christian Ehrhardt wrote:
> > No I'm fine with a single command.
> > If it is low on cpu consumption and fast.
> > We have to consider that in some environments  you might have plenty
> > (really a lot) of changes ongoing.
> > And you might get a notification for each of them.
>
> How much is plenty?


On auto-deployed config changes of virt systems (with plenty of switch
interfaces for example) I've seen three digits per second (rare, but
existing).


> I guess we could use a delay and batch all
> requests in some interval (e.g. one second), but I think I'd prefer if
> it was executed synchronously. It's simpler and there will be no
> surprises if someone runs the new command and then follows it with the
> old online command, which might have no effect.
>

This has two things in it:
a) make it synchronous - ack for all the reasons you outlined
b) need batch/coalesce - this still is true as the async nature can already
come from the network event handling

So imagine a timeline (seconds at the top) with the update process taking 6
seconds

secondevent
1 net event 1 triggers "probe I"
2 Events 2-20 trigger 19 more
...
7 probe I completes and returns to caller "event 1"
8 the formerly waiting probe II (just one) runs now
...
14probe II completes and returns to all callers Events 2-20

This is synchronous + batching as I'd think of it

- We can't skip probe II as network might have changed in a way "probe I"
was unable to pick up
- Running one full probe per event can become a hue backlog and a
thundering herd problem




> > With just one command neither Stephens nor my suggestion make sense.
> > How about just "probe" then?
>
> I'm not sure. What exactly are we probing here? There are no packets
> exchanged. It's just a local check of the routing configuration.
>

You wrote later:
  "How about "onoffline" or "maybeonline"?"

I'd like auto-onoffline or something like it more than any of the two above


> --
> Miroslav Lichvar
>
> --
> To unsubscribe email chrony-dev-requ...@chrony.tuxfamily.org with
> "unsubscribe" in the subject.
> For help email chrony-dev-requ...@chrony.tuxfamily.org with "help" in the
> subject.
> Trouble?  Email listmas...@chrony.tuxfamily.org.
>
>


-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

Re: [chrony-dev] New online/offline command

[Miroslav Lichvar]

On Thu, May 24, 2018 at 11:54:49AM +0200, Miroslav Lichvar wrote:
> > With just one command neither Stephens nor my suggestion make sense.
> > How about just "probe" then?
> 
> I'm not sure. What exactly are we probing here? There are no packets
> exchanged. It's just a local check of the routing configuration.

How about "onoffline" or "maybeonline"?

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-dev-requ...@chrony.tuxfamily.org with "unsubscribe" 
in the subject.
For help email chrony-dev-requ...@chrony.tuxfamily.org with "help" in the 
subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-dev] New online/offline command

[Miroslav Lichvar]

On Wed, May 23, 2018 at 04:44:30PM +0200, Christian Ehrhardt wrote:
> No I'm fine with a single command.
> If it is low on cpu consumption and fast.
> We have to consider that in some environments  you might have plenty
> (really a lot) of changes ongoing.
> And you might get a notification for each of them.

How much is plenty? I guess we could use a delay and batch all
requests in some interval (e.g. one second), but I think I'd prefer if
it was executed synchronously. It's simpler and there will be no
surprises if someone runs the new command and then follows it with the
old online command, which might have no effect.

> With just one command neither Stephens nor my suggestion make sense.
> How about just "probe" then?

I'm not sure. What exactly are we probing here? There are no packets
exchanged. It's just a local check of the routing configuration.

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-dev-requ...@chrony.tuxfamily.org with "unsubscribe" 
in the subject.
For help email chrony-dev-requ...@chrony.tuxfamily.org with "help" in the 
subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.