ntpq/mode6 cleanup
>> For a small project, I think we should make mode6/ntpq require the cookie on >> everything but getting the cookie, and we should make sure that there is no >> amplification when getting the cookie. > That would break compatibility with ntpq from classic NTP. There are 2 areas I'm interested in. First is to make sure there is no way to get amplification on reflection. I'm willing to break compatibility to get that. The second would be a big cleanup. I'd like to do something like split the current daemon into several parts, for example: server, client, refclocks, ntpq-server, NTS-KE server Splitting out ntpq-server would be a good start. Again, I'm willing, even expecting, to break compatibility. Handwave, strawman... Using TCP rather than UDP avoids reflection problems. Most of ntpq would work fine if we put all the counters info read only SHM. We don't need a lock. The results may be slightly inconsistent. The mru list stuff won't work in simple read-only SHM, at least with the current approach of scanning the list in chronological order. It almost doesn't work as is. If the list is big enough to be interesting for busy servers, it takes a long time to scan it. Too long to be useful. Or maybe we should shift to SNMP. I hate that level of obfuscatiion, but if somebody likes it and is willing to run with it, I'll put things in SHM. But suppose we scan it in physical order, and sort things out at the client? That also solves the problem of the current approach never finishing on a busy server because the data changes faster than it can be retrieved. The other tool in the mru area would be to log interesting stuff. But I haven't worked out a simple/clean version of "interesting". Putting things in SHM introduces version control issues. I think they are not a problem as long as the stuff on the wire is text rather than offsets. Then all we have to keep in sync is the ntpq-server and ntp-server and ntp-client. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Updated to-do list?
[Context is a multi-threaded Go echo server] > I can manage to do that. I'll send you the C code off list. Here is an outline of the big picture: Linux and FreeBSD have a SO_REUSEPORT option. The idea is that you can open several sockets on the same port number. The kernel will hash on source IP address and source port. (so packets for a connection go to the same thread -- the thread can save state for a connection) The main server thread sets up worker threads Each worker thread opens a socket The worker threads bump counters. The main thread will read/collect and print the info. The worker threads have an option to spin for N microseconds between recvfrom and sendto. The idea is to simulate crypto work. The limiting factor on packet throughput is the kernel thread That thread puts arriving packets on the right socket queue and wakes up a waiting thread if there is one So larger packets reduce the load on the kernel thread for a given traffic load in bits/second (rather than packets/second). Handwave... The CPU load for a NTP server is roughly constant in terms of bits/second. So you want to assign the kernel thread to one core (leaving the other hyperthread idle) and disable interrupt coalescing. Then fill up the rest of the CPUs on the chip with worker threads. It's roughly a microsecond per packet for the kernel thread, a microsecond each for recvfrom and sendto and (handwave) a microsecond for basic NTP server work. Up a level... There is a companion client side that runs several worker threads sending to the server. You can run it on multiple old/slow PCs to get enough traffic to saturate the target server. There is an option to send N extra packets, like ping -l. I call them "in flight". Things are setup assuming that no packets are lost. So you have to be sane on picking combinations of multiple clients and extra packets in flight. - Up another level... I have some hackish scripts that fire up a server fire up several clients read/reset the server stats wait a bit read server stats print results loop for various CPU load/delays or packet lengths -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Updated to-do list?
> On 04/08/2023 2:45 PM PDT Hal Murray wrote: > > Given that the things I have been turning in are not in the direction > > we are headed, what should I be working on? Other than trying to do a > > Golang port by myself, or revisiting the more than a-year-old list? > MR 1307 and/or 1309 is high on my list. I was hoping somebody else would be > interested. Grab a shovel and a Philosopher's Stone. > What's on your year old list? TLDR back in June 2021, I kicked off the previous to-do list by revisiting the in-tree devel/TODO.adoc file [1] and randomly commenting on the points. [2] This led to ESR kicking off the provisional new list with the sole item of porting NTPsec (probably) to Golang [3]. I kicked in my tuppence[4]. * check on the state of C to Rust/Golang transpilers (They sucked) * Archipelligo model but with threads rather than discrete processes to enable per thread seccomp. * Implement a toy that hits all of the (sys)calls used by NTPsec quickly (for seccomp again) * try to break up sizeable opaque code blocks (maybe enabling things like refclockd and testframe) * Testing ** compilers other than GCC/Clang ** kernels other than Linux (token FreeBSD/Darwin) ** other libC implementations, file Hierarchies * Update the docker runners to be better * sort out my miscellaneous merge requests (whatever frippery I think up) with the result primarily being [5] * Port away from C with minor diversions to * Resolve CVEs * close issues to have less than 20 open * More extensive unit-tests * Releases [1] https://gitlab.com/NTPsec/ntpsec/-/blob/master/devel/TODO.adoc [2] https://lists.ntpsec.org/pipermail/devel/2021-June/009761.html [3] https://lists.ntpsec.org/pipermail/devel/2021-June/009775.html [4] https://lists.ntpsec.org/pipermail/devel/2021-June/009779.html [5] https://lists.ntpsec.org/pipermail/devel/2021-June/009798.html > For a small project, I think we should make mode6/ntpq require the cookie on > everything but getting the cookie, and we should make sure that there is no > amplification when getting the cookie. That would break compatibility with ntpq from classic NTP. > How much do you know about Go? Can you write a multi-threaded echo server? > > I have a collection of hacks for measuring performance of NTP servers. Down > hill and with a tailwind, I can get 1,000,000 packets per second. I have C > and Rust versions. It would be neat to try a Go version. I can manage to do that. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
