Patch to make bin_test work without pylib
The clients needing it are still beinginstalled. I have another patch for that later.-30- 0003-Make-waf-bin_test-work-with-enable-pylib-none.patch Description: Binary data ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Removing md5 usage from ntpsec
> On 04/16/2024 2:34 AM PDT Shreenidhi Shedi via devel wrote: > > Hi All, > > Currently ntpsec systemd service doesn't start if openssl fips is enabled. > The same issue was there in ntp as well. > > Looks like ntp is ready with the fix. More info at: > https://bugs.ntp.org/show_bug.cgi?id=3753 > > It would be great if ntpsec can provide a fix as well; many mainstream > distros are already providing the ntpsec package, so it's better to have it > working in fips mode. They appear to be using a modified version of the in tree copy of MD5 they already have. We could drag the one we inherited out of mothballs, but we have chosen not to do so at this time. JamesB -30- ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Hack for monitoring NTP servers
> On 04/11/2024 12:39 PM PDT Hal Murray via devel wrote: > > > If somebody feels like hacking, something like this should be fun. > > The idea is to setup a ntpd server watching the servers you want to monitor. > (noselect on the server line does that) > > The new code is a program that watches that server to see if the servers to > be > monitored are responding correctly and sends you email if they aren't. If you were thinking of adding way too many servers you might want to replace the bubble sort around lines 1709-1728 of ntp_proto.c I would not write such a thing as it would get me banned from Google. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: CI is broken
> On 04/15/2024 9:15 AM PDT Matt Selsky via devel wrote: > > On Sun, Apr 14, 2024 at 01:30:47AM -0700, Hal Murray via devel wrote: > > I just pushed some code. The CI stuff sent me a Failed pipeline message. > > > > > > [0K[31;1mERROR: Job failed: failed to pull image > > "registry.gitlab.com/gitlab-or > > g/gitlab-runner/gitlab-runner-helper:x86_64-v16.11.0" with specified > > policies > > [always]: Error response from daemon: manifest for > > registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-v16.11 > > .0 not found: manifest unknown: manifest unknown (manager.go:250:0s) > > > > https://gitlab.com/NTPsec/ntpsec/-/jobs/6619973110/raw > > > > There are 9 copies that looked the same to me. > > Hi Hal, > > I don't see anything on GitLab's status history page at > https://status.gitlab.com/pages/history/5b36dc6502d06804c08349f7 but I expect > there's nothing for us to do and we can just wait for GitLab to fix the > underlying issue (assuming it hasn't been resolved already). It is resolved already. I assume from briefly looking at the logs that it was a new release of the gitlab runner helper that broke it. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: What next?
On 03/17/2024 at 2:00 PM PDT, Hal Murray via devel wrote: > > Is anybody thinking about what we should be doing? > > > Here is my list: > > Port to Windows > Does anybody know anything about Windows? > Is there a decent POSIX environment? > How well does waf work on Windows? > We can get the magic code from ntp-classic. I have forgotten almost everything about MS Windows. It claims to have a POSIX environment, but it is not. IIRC waf works on MS Windows; our main configuration does not so much. The portability shim should still be available at the git-conversion tag, fitting it in will be annoying. > I think we should split ntpd into several independant programs. > More in another message. I gave up on that notion; I lacked the patience to do it. > I think we need a good SNTP client. Something like the old ntpdate. > I'm looking for a clean example. > This would be a good opportunity to experiment with Go and/or Rust. > > Getting off the ground. > There is a chicken-egg problem with getting started when using NTS. TLS > needs the time to check certificates. I think we can do something like skip > the date part of certificate checking, then come back and see if the > certificates pass the date-check after we have a candidate date. Dusting all the corners would be irritating. > Alternate port for use with NTS. > There is a lot of blocking/filtering on port 123. NTS-KE includes > specifying the port to use. We should be able to listen on another port too. > I haven't looked carefully. This feels like medium complexity. Yeah, the IETF NTP WG shot down the notion of NTP alternative port. ... > We should test the config file stuff to see that all the options at least get > past the parser. Better would be to actually run the code. I think somewhere in the middle might be a program that takes config files and dumps them into some format that is easy to eyeball and machine parse. > We should check FIPS mode. Do any of the CI options include FIPS? > I got half way there by building OpenSSL to include FIPS mode but I haven't > made the config file to use it. None of the CI runners support FIPS140-2 at the moment. I don't know how to make them either. > I'd like a script that checks the certificates. When do they expire? That sounds like a simple wrapper around 'openssl x509' would work. If OpenSSL does not work, you are probably looking at something much heavier in the front. > I'd like a script that finds out who signed a certificate and pokes around in > my local certificate collection and tells me a filename so I can add that to > a > server line in the config file. The idea is to make sure that we are using > the right root-cert rather than one from a CA that was arm twisted by your > local repressive govt or broken into by the KBG or NSA. Perhaps we could call it cert-sweep and also dump the hash, notAfter and other data from the certificates to standard out as well. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
RE: Addressing the Python 2, AsciiDoc classic, and AsciiDoc 3 tangle
> On 01/10/2024 7:49 PM PST Matt Selsky via devel wrote: > > Does anyone have links handy to statements from the various projects > on their long-term support status? Not as such. Hitting hub.docker.com I find no Python:2.6* images, Python:2.7* last pushed 21 April 2020, and Centos last pushed 15 September 2021. Checking packages.debian.org, packages.ubuntu.com, and packages.fedoraproject.org for AsciiDoc and Python I find only Debian Buster has AsciiDoc classic, while Debian Buster, Debian Bullseye, Fedora 38, Fedora 39, Fedora Rawhide, Ubuntu Focal, and Ubuntu Jammy claim support of Python 2.7. A modest google search suggests no distro builds packages for AsciiDoc3. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Addressing the Python 2, AsciiDoc classic, and AsciiDoc 3 tangle
I suggest that Asciidoc3 should be dropped as it is not well-supported anywhere else and incurs additional maintenance costs. To address the support of AsciiDoc classic, we can use a Debian Bullseye image until around July/August 2024, as it has AsciiDoc classic. The current Debian Bookworm, which will replace Bullseye, does not support AsciiDoc classic. The image maintainer last updated the Bookworm and Bullseye images in mid-December, while the Centos 6/7 and Python 2 image maintainers last updated them a few years ago. Fred is the only reliable support for Python 2.6, and we should look for a maintained image that has Python 2.6, in case he is unavailable. We should consider dropping support for AsciiDoc3 and move the Python 2.7 and AsciiDoc classic checks to Debian Bullseye. Also, it has been proposed that we should drop support for Asciidoc classic and Python 2 in August 2024 and 2025, respectively. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Release
> On 12/17/2023 9:49 PM PST Hal Murray via devel wrote: > > Fred Wright said: :::snip::: > > There are also a bunch of warnings with some compilers, which might be > > worth > > looking at. They're often fairly easy to fix, and sometimes indicate actual > > problems. > > Which compilers? Or rather which OS/distros? > > Can we set things up so that the gitlab CI stuff tells us about warnings? > > James suggested adding the compiler flag that turns warnings into errors. > That won't work on the old old version of Bison that has a missing default or > something like that. I also would be interested in seeing these errors for myself. there is also 'clang -Weverything' and some weblinks - https://gitlab.com/NTPsec/ntpsec/-/security/vulnerability_report - https://app.codacy.com/gl/NTPsec/ntpsec/dashboard - https://scan8.scan.coverity.com/#/project-view/50864/11242 I am also attaching a file derived from the output of clang. A previous attempt to send a more complete listing had issues.-Weverything (x86_64 Linux Clang 16) 1889 -Wunsafe-buffer-usage 1382 -Wpadded 375 -Wextra-semi-stmt 167 -Wdouble-promotion 103 -Wsign-conversion 81 -Wshorten-64-to-32 65 -Wimplicit-int-conversion 58 -Wmissing-variable-declarations 56 -Wdeclaration-after-statement 53 -Wunused-macros 52 -Wimplicit-int-float-conversion 32 -Wcovered-switch-default 23 -Wimplicit-float-conversion 20 -Wdisabled-macro-expansion 17 -Wformat-nonliteral 16 -Wunreachable-code-break 14 -Wunreachable-code 12 -Wfloat-conversion 10 -Wcast-align 8 -Wimplicit-fallthrough 5 -Wcast-function-type-strict 5 -Wbad-function-cast 4 -Wused-but-marked-unused 4 -Wreserved-identifier 2 -Wextra-semi 2 -Wconditional-uninitialized 2 -Wcomma 1 -Wunreachable-code-return 1 -Wstring-conversion 1 -Wreserved-macro-identifier -Weverything (arm64 macOS clang 15) 1310 -Wpadded 404 -Wunused-macros 158 -Wpoison-system-directories 99 -Wsign-conversion 82 -Wshorten-64-to-32 57 -Wmissing-variable-declarations 54 -Wdeclaration-after-statement 52 -Wimplicit-int-conversion 28 -Wcovered-switch-default 22 -Wcast-align 17 -Wformat-nonliteral 16 -Wunreachable-code 13 -Wunreachable-code-break 8 -Wimplicit-float-conversion 6 -Wdouble-promotion 5 -Wbad-function-cast 4 -Wused-but-marked-unused 3 -Wfloat-conversion 2 -Wextra-semi 2 -Wdisabled-macro-expansion 2 -Wconditional-uninitialized 1 -Wunreachable-code-return 1 -Wstring-conversion ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Missing clockwork
> On 12/17/2023 11:49 PM PST Hal Murray via devel wrote: > > > Anybody recognize this? I've seen a missing file once before. I think it > was > clockwork.?? > > It works if I try it again. It sounds like a race condition in our wscript files or waf. How willing are you to sink time into this, I think it's a losing proposition. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: What does gitlab's "Successful pipeline" mean?
On Dec 5, 2023 23:45, Hal Murray via devel wrote: Does that mean no warnings? No errors slipped past any nets in their way.If not, how are we expected to learn about code that generates warnings on obscure systems?Maybe we should add -Werror or such to CFLAGS. The most obscure system tested is FreeBSD. All compilers are clang or gcc.All the world really are Linux boxen. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: How does the parser work?
On Dec 4, 2023 13:49, James Browning wrote:The host phase of Waf build generates tablegen which in turn generates keywords.h IIRC. I have no idea how the internals work. I seem to recall that tablegen.c was badly scored by Codacy.Oops, keyword-gen and a rated; I need to get my memory replaced.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: How does the parser work?
On Dec 3, 2023 23:45, Hal Murray via devel wrote: ntp_parser.y contqains: %token T_Tinker %token T_Tlsciphers %token T_Tlsciphersuites I'd expect those tokens to come from the keywords header file. But tlsciphers isn't in the keyword list. tlscipehrswuites is in the list.The host phase of Waf build generates tablegen which in turn generates keywords.h IIRC. I have no idea how the internals work. I seem to recall that tablegen.c was badly scored by Codacy. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Asciidoc question
On Dec 3, 2023 18:49, Hal Murray via devel wrote:What does the $$ after the +aga+ do? |+year+ |One generation file element is generated per year. The filename suffix consists of a dot and a 4 digit year number. |+age+$$ |This type of file generation sets changes to a new element of the file set every 24 hours of server operation. The filenameIt appears it inserts it in the default font after 'age' which seems to be monospaced. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Release
> On 12/02/2023 8:09 PM PST Hal Murray via devel wrote: > > > I think you should release what we have as soon as it is convenient. > > There are many more things I would like to include but we aren't making much > progress so it's time to do it. Referring to the first four items on the release checklist. 1. It is approximately time for a release. 4. The buildbots are not reporting any unplanned regressions; there are always issues to be addressed. 2a. An early warning has been sent to the devel list. The NEWS and devel/TODO files are not up to date. 2b. There have been 144 commits since 1.2.2; 53 are from Hal Murray, 47 are from Matt Selsky, 31 are from James Browning, and everyone else totals 13. 3. I think we should nix the devel/TODO* files and update the NEWS file before cutting a release, but I do not see any other major blockers to a soon(ish) release. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: I/we need a lesson in git and/or gitlab and/or merge requests
Hal Murray wrote:
>
> Merge requests seem reasonable if all goes well. My work flow is roughly:
> download the patch (URL plus ".patch")
> scan it
> maybe apply and test
> approve and merge
Ah, my work turbulence is incompatible with your workflow.
> But things go downhill if I don't like something. What I get from James is
> an
> update to the MR, a patch to the patch. That makes reading/checking the
> patch
> harder and clutters up the git log.
>
> What if I don't like the description of a patch?
>
> Merge has an option to reduce all the patches to one. But often that isn't
> appropriate.
"git rebase -i" allows you to selectively drop, reorder, squash,
and/or amend commits. At least, that is what I have heard. I use
GUI tools.
> git works so well for most things. I think I/we are missing something in the
> workflow.
Reasonable communication, etiquette, and politics on my part?
> Should we be throwing away merges and making new ones rather than patching
> them?
Yes, if it works better than my work-turbulence not working with
your workflow. We will not run out of integers.
> How do I backup a bunch of commits that turned into a MR so I can make them
> better and try again?
>
> I'm on a list or two where patches are distributed via email. git has
> several
> commands for that. Iterations usually have a v1 v2 ... as part of the
> Subject. Often individual parts will be approved. It's a lof of clutter in
> the email stream but the discussion gets archived in email rather than hidden
> over in a MR.
>
> Is there a way in gitlab to approve only one of the patches rather than all
> of
> them? I think I could do that by downloading the patch which is several
> email
> messages, editing out the one I want... Again, if that was the right thing
> to
> be doing, I'd expect git to support it which it probably does if you use
> their
> email mode.
I am unaware of many things, including any potential means of
cherry-picking any particular commits via GitLab. Offline, I might
try the following.
$ git checkout origin/master
$ git am ${path_to_mr_patches}
$ git format-patch ${number_of_commits}
$ git checkout origin/master
$ $EDITOR ${specific_patch_file}
$ git am ${specific_patch_file}
Should I return to throwing patches at people and see what gets
picked up?
-30-
___
devel mailing list
devel@ntpsec.org
https://lists.ntpsec.org/mailman/listinfo/devel
Re: Time for a release?
> On 10/31/2023 2:56 AM PDT Hal Murray via devel wrote: > > Is updating PIVOT.h on your checklist and/or should I update it now while > I/we > think of it? I do not see a reply on list; I will now bleat pointlessly. Updating PIVOT.h is not as far as I can tell mentioned anywhere in the release process documentation. Yes, PIVOT.h probably should be updated. I had a scheme for when to update the pivot time; It does not matter what it was. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Time for a release?
> On 10/31/2023 2:28 AM PDT Hal Murray via devel wrote: > > > Here are the open issues the caught my eye: > > https://gitlab.com/NTPsec/ntpsec/-/issues/806 > > I think we should fix that. Or at least try. It sounds like a bug in > ntp_control. I just tried rv xxx for some xxx that was a reasonable assid. > It didn't print any garbage. Anybody got a handy test case? > > Looking at the code... > It fills a buffer with 8 " %.2f", then calls the routine that prints that as > name=value. > That won't work with spaces in there -- well, maybe it will, but it depends > on > what the parser in ntpq does. I'd expect it to call the routine that prints > it as name="value". But I don't know what ntpq is doing... We should print > that stuff in a nice table. It was an issue in 1.2.1 and 1.2.2, Hal solved it already. A slightly longer overview is in #806. The short version is I forgot to initialize the first byte of storage and whatever cruft was in there up to the first NUL got in. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Time for a release?
> On 10/29/2023 10:50 PM PDT Hal Murray via devel wrote: > > The last time this was suggested, I encouraged waiting until we fixed mssntp. > Well, I think we have it fixed but we haven't found anybody to test it. > > So I think it's time to get ready for a release. > > Time for lots of testing. And documentation checking/cleanup. > > Does anybody have any features that should or must go in or bugs we should > fix? > (I haven't looked through issues yet.) Highest priority MRs I have are 1331 and 1344. Bringing in some of the others would be nice. I want to rework 1341 though and now is not the time. Some of my other merge requests address issues that have been reported. > What is the policy on ntpq documentation? We have tuned the code for use > with our version of ntpd, but it still mostly(?) talks to the old > Mills/classic version. I noticed lots of references to multicast and > broadcast in the man page. We removed the code that supported that stuff > ages ago. The *cast references are now clutter if you are interested in our > code, but might be relevant if you are looking at an old old system. Should > we leave the *cast documentation in or clean it out? Huh, I'd have thought it would be in devel/hacking.adoc, maybe not. I suggest something like the following boilerplate... # Due to insecurabablility and dubious timing issue non unicat modes # are not supported in NTPsec, ntpq still supports reporting non- # unicast clocks to support legacy software. > I have 3 hacks that were used to debug talking to Samba. Is a subdir under > attic a reasonable place for them? I would have stuck them in contrib/. I don't think people consider me reasonable. I would've like to see !1173 and !1174 merged, but I think that time has passed. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Too many merge requests
> On 10/14/2023 7:16 PM PDT Hal Murray wrote: > > > James said: > > MR 1333 should address the issue where every response in an > > mssntp-restricted > > subnet gets lost in ntp_signd. > > Is there any reason to approve it now when we don't know if it works yet? I have tested it a bit; It prevents non-MSSNTP traffic from getting sucked into ntp_signd and vanishing. Errors in ntp_singd generate an error message or two. Hopefully, anyone tripping over this in the future will help us debug it rather than just looking at the Samba Wiki crapping on us and leaving. > Are all the changes inside an #ifdef? (and thus unlikely to break normal > operations) Yes, #ifdefs contain all of it except for maybe_log() and struct do_we_log, which I see as having potential uses besides ntp_signd. > > MR 1331 addresses several issues with ntpdig and MAC handling. > > I hate big python programs. It would be nice if somebody else took a look at > that one. I asked Matt Selsky to merge it in early August and am still waiting to hear back. > > MR 1325 holds lost patches. > > What does "lost patches" mean.? If they are useful, it will be much easier to > approve them if they are split up -- one problem will hold up the whole > collection. Ah, let me rephrase: I have almost given up on them ever being merged. MR 1325 has 48 patches in /patchsets. 11 are standalone patches, and the rest are in 9 patch chains. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Too many merge requests
I currently have eight merge requests open; I would like other people to merge two. MR 1333 should address the issue where every response in an mssntp-restricted subnet gets lost in ntp_signd. MR 1331 addresses several issues with ntpdig and MAC handling. OTOH, I am still preparing MRs 1340, 1337, 1336, 1330, and 1282, while MR 1325 holds lost patches. In detail, MR 1333 restores the removed MS-SNTP bypass logic (fixing the MACless, typical symmetric, and NTS cases). It then adds a rate-limited logging mechanism, then it adds logging to ntp_signd where none pre-existed and finally cleans up the code a little. A look at MR 1331 reveals: First, it tightens the timing by an insignificant amount while resolving an obscure issue. Then, it addresses another more elusive issue. Later, it Fixes an issue with (previously) untruncated long MACs. Finally, it improves the timing of subsequent packets insignificantly while ensuring they do not get additional MACs. You can stop reading here. The worst of the not ready is 1325, which has too many patches in the tree. MR 1340 would drop Asciidoc3, Ascioidoc-py2 and Asciidoc-py3 (<9.0.0) support from NTPsec. I need to think more about MR 1282. It's been years since anyone cared about NTPsec docker images, so MR 1330 can drop. Running bin_test against the installed files should be necessary, but no one cares for MR 1336. Consistent docs and code regarding the Address/Protocol Family must be essential to resurrect MR 1337. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: mssntp option breaking time service: tester(s) wanted
> On 10/04/2023 3:41 PM PDT Hal Murray wrote: > > > Does anybody have details on how MSSNTP signing works? > > If we can find that, we can write some POSIX code to test things. I have test tools that should push the right buttons IF I can get the correct magic number in the key ID. After running it with a rainbow table, it did not work. Roughly... 1) The client does client things I should not have to care about 2) The client sends an NTP request with a magic key ID and 16-byte NUL MAC 3) after correctness checks, the server forms a reply and then sends it like this. a) ntpsec connects to Samba (works) b) ntpsec sends a length (tested/seems to work) c) ntpsec sends serialized struct samb_key_in. d) samba responds with a length (always 12 IIRC) e) samba continues with serialized struct samba_key_out 4) ntpsec reads that and concludes that the packet is not correctly signed and tries to throw a couple of new error messages. > There is a link in ntpd/ntp_signd.c > http://msdn.microsoft.com/en-us/library/cc212930.aspx > But I didn't find anything interesting there. (Maybe my browser was filtering > something.) I see walls of text that I do not want to read. > We still need to test with real Windows at least once to make sure our test > code does the right thing. Yes, I probably should. -30- ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
mssntp option breaking time service: tester(s) wanted
James Browning wrote: I have a patch series for ntpsec, which fixes time service for unauthed NTP traffic. It should also work on NTS. I have yet to test it there, though. I do not have a test rig for MS-SNTP. I would appreciate it if anyone could test it against MS-SNTP. Attaching a pcap (packet capture) file of the UDP traffic on port 123 (NTP) for the duration would also be nice. recentish git head and patch series at https://gitlab.com/na280/ntpsec.git in the branch 23H14-ms-sntp-frob; I do not develop in the default branch. For those with GitLab accounts, the merge request and Issue are at: https://gitlab.com/NTPsec/ntpsec/-/merge_requests/1333 and https://gitlab.com/NTPsec/ntpsec/-/issues/785 -30- ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: I just pushed ntsstats and ntskestats
> On 09/26/2023 10:09 PM PDT Hal Murray wrote: > > > After glancing at the tops and bottoms fo some of the HTML docs, I think I > > have some issues to correct. > > I think the last line of the man pages used to have the version number. I'm > not seeing that now. > > Is that because I'm doing something special to get the date in the version > string ( --build-desc=xxx to waf) or did that fall throug the cracks somehow? > > (Or was it my imagination?) I would love to say "I think it's your imagination;" how I think it slipped thought the cracks at some point. It would appear I failed to heed my own advice. +// Be very careful about changing things here. +// Everything is included in HTML doc forming. +// Borrow a second set of eyes. -- JamesB192 I will work a bit on tracking down why it is gone. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: I just pushed ntsstats and ntskestats
> On 09/24/2023 6:09 PM PDT Hal Murray via devel wrote: > > > There was a request for ntpviz to show NTS traffic. We now have the log > files > to make that possible. That's good, I had some ntpviz glue that works with a different way of doing that. I'll check if I can graft it to work with yours. > Please look at the indentation around the info for the *stats options in both > the man page for ntp.conf and the html pages for ntp_conf and monopt. I think the entire section need to be reformatted, but I'm not sure how yet. > The web pages have a box around the sample lines for the log file. That's > missing from the man pages. Ah, I misread that earlier; you'd probably have to stick the samples in a table for that to show in man. After glancing at the tops and bottoms fo some of the HTML docs, I think I have some issues to correct. As previously mentioned ntp_conf has a Mills reference that might go away. The horizontal rules at the bottom of decode, driver_howto, driver_trimble and kernpps are not consistent with the rest of the HTML docs. The driver_* pages seem to be inconsistently formatted. rate seems to have a duplicated footnote. dirver_howto and rollover are the only docs with images at the end, and sitmap has seemingly inconsistent 'links to top' and many missing links. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: I just pushed ntsstats and ntskestats
On Sep 24, 2023, 18:09, Hal Murray via devel wrote: There was a request for ntpviz to show NTS traffic. We now have the log files to make that possible. Any documentation wizards?I am willing to break out some instant expert credentials on this until someone better shows up.Please look at the indentation around the info for the *stats options in both the man page for ntp.conf and the html pages for ntp_conf and monopt.I will get on that. The web pages have a box around the sample lines for the log file. That's missing from the man pages.I will take a look at it. Would you like a single or double-line 'box' for the tables? ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Tangle with -4 and -6
> On 09/17/2023 at 10:36 PM PDT, Hal Murray wrote:
>
> -4 and -6 work on the server line in ntp.conf but are not documented
The closest I can see to an implementation is in ntpd/ntp_parser.y
lines 390-395. I infer this to mean that it sets the Address Family
of peer->srcaddr to a matching value. Which I could draft a document of.
> -4/ipv4 and -6/ipv6 "work" on the command line, but they don't do what the
> documentation says. The man page says:
>Force DNS resolution of following host names on the command line to
>the IPv4 namespace.
> What they do is turn off setting up sockets for the other protocol.
I could write some code to do both and update the doc to reflect
that but not merge it.
> I'm not sure what the NTS server does if, say, the system doesn't support
> IPv6
> when it tries to listen on an IPv6 address.
A couple of possible cases here: First if a host truly doesn't support
IPv6, it will probably generate a failure at some point that gets
logged> OTOH a host that has disconnected IPv6 will probably set up a
peer entry that will persist unless something removes it.
> The network side sets up two flags: ipv4_works and ipv6_works
> The command line -4 and -6 flags turn off the other _works flag.
There are variables in ntpd.c for whether to try choosing a particular
AF or go with the default. After checking which AFs work, there is
the following code...
if (ipv4_works && ipv6_works) {
if (opt_ipv4)
ipv6_works = false;
else if (opt_ipv6)
ipv4_works = false;
} else if (!ipv4_works && !ipv6_works) {
msyslog(LOG_ERR, "INIT: Neither IPv4 nor IPv6 networking
detected, fatal.");
exit(1);
} else if (opt_ipv4 && !ipv4_works)
msyslog(LOG_WARNING, "INIT: -4/--ipv4 ignored, IPv4 networking
not found.");
else if (opt_ipv6 && !ipv6_works)
msyslog(LOG_WARNING, "INIT: -6/--ipv6 ignored, IPv6 networking
not found.");
Replace with something more like the following.
if (metal_af == AF_NONE) {
msyslog(LOG_ALERT, "INIT: The Network does not work.");
exit(1);
} else if ((metal_af == argv_af) || (metal_af == AF_UNSPEC)) {
sys_af = argv_af;
} else if (peer_af == AF_UNSPEC) {
sys_af = metal_af;
} else {
msyslog(LOG_WARNING, "INIT: Ignoring requested family %d", argv_af);
}
> I wrote the DNS code for both server/pool and NTS. I don't remember how the
> -4/-6 options work (and a quick look didn't refresh my memory). I don't
> remember ever checking the above flags or thinking about doing it.
It uses the AF of peer->srcaddr, AFAICT the command line doesn't
affect it even indirectly.
> Note that there are 2 DNS lookups on the NTS path, one for the NTS-KE server
> and another if the server returns a name/address rather than using the
> default
> of the same address as was used for the NTS-KE lookup.
I was going to try to write something relevant here, but I decided
not to.
> I'm pretty sure the command line processing doesn't do any DNS lookups.
> It roughly adds a server line, and does a DNS lookup with the constant-only
> (no net traffic) flag so that slot won't get delayed behing a real DNS lookup
> that is skow.
>
> --
>
> I think we should clean up this area. That includes:
I would also suggest cleaning up libntp/{initnetwork,isc_net}.c to
remove isc_result as we only seem to care if we succeed and not the
why of failures.
> Making sure DNS lookups don't use an address for a disabled protocol.
A minor patch in ntp_dns.c may help with that.
if ((sys_af == AF_UNSPEC) || (sys_af = peer_af)) {
hint.af = peer_af;
} else if (peer_af = AF_UNSPEC) {
hint.af = sys_af;
} else {
msyslog(LOG_WARNING, "DNS: Requested supported family %d", peer_af);
}
> Add enable/disable -4/ipv4 -6/ipv6 to ntp.conf
> Note that these will have a backwards meaning from the -4 on the command line.
> -4 on the command line <=> disable -6
>
>
> Does this make sense?
> Am I missing anything?
A corner case and minor details, perhaps.
> This will take a lot of testing.
Ah, yes, the other reason why I can't get anything merged.
> We should move the command line code in config_peers to that checks for a
> numeric address to the main processing loop.
We should do it before the main loop starts; the
check is probably affordable.
static bool is_sane_resolved_address()
in ntp_cconfig.c is written for that.
It seems that -4 and -6 also apply to the interface commands in the
config file and via mode 6.
___
devel mailing list
devel@ntpsec.org
https://lists.ntpsec.org/mailman/listinfo/devel
Re: Is python2 dead?
> On 09/12/2023 12:00 AM PDT Hal Murray via devel wrote: > > > Gary said: > > Please, no. Go is a garbage collected language. Just what NTPsec does not > > need, random, unpredictable delays. It would appear there is a way to turn off GC under runtime/, Given this, it might be reasonable to schedule run a light GC at an odd corner of every minute and a heavier one less often. > I was thinking of the Python code in ntpclients/ and pylib/ > Is there anything in there that is time sensitive? Maybe ntpdig with time sitting or the SNMP agent? I think the premise there is everything is human speed. > There are lots of ways to inject timing bumps before we get to garbage > collecting. cache, scheduler, interrupts, CPU speed, ... > > Do you have any data on Go GC times? It is probably somewhere in the previous considerable discussion back in June/July 2021. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Is python2 dead?
On Sep 4, 2023 14:46, Matthew Selsky via devel wrote:On Mon, Sep 04, 2023 at 02:25:46PM -0700, Gary E. Miller via devel wrote: > From RHEL: > > "The RHEL 8 AppStream Lifecycle Page puts the end date of RHEL 8's > Python 2.7 package at June 2024." > > https://access.redhat.com/solutions/4455511 Hi Gary, Is the implication that we can safely drop python2 support after June 2024? Are there any other distributions that ship python2 that we want to maintain support for? How much does it cost us to maintain python2 support in our own code?By dropping 2.6 support we would eliminate the need to check for argparse in a few place, glue to replace ordereddict?, be able to use with for resource allocatoion, and the need to pretend to test on 2.6. By dropping 2.7 we could probably assume secrets which simplifies ntpkeygen, simplify ntp.poly, be able to drop the now oldoldstable? runner testing for asciidoc on python 2 support, and also have the option of adding type hinting.I am probably missing a whole bunch of stuff though.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: nts and ntske log files
On Aug 28, 2023 18:10, James Browning wrote:On Aug 28, 2023 17:10, Hal Murray wrote: James Browning said: > The NTP solution would be to convert the mess to l_fp which > will work for a bit less than 13 years. Thanks. l_fp is the right answer. How did you get 13 years? I get 136. Did you drop/typo the 6?Expires February 2036 minus the current dateis about 12 years and 5 months maybe ish...Oh, wait I am supposed to use rollover funcs tocompensate for throwing away more than halfof the epoch at the beginning. > My joke would be to have it as a long long of micro-seconds which would be > good for hundreds of thousands of years. l_fp has 32 bits of fraction. micro takes 20 bits. So using micro seconds would gain 12 bits. So that would be 557xxx years.My revised guess would've been around 54400.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: nts and ntske log files
> On 08/28/2023 3:08 PM PDT Hal Murray via devel wrote: > > > I just pushed the first cut. > > No documentation yet. > > Like sysstats and usestats, ntsstats and ntskestats get logged every hour. > > If you look at the output from ntpq -c nts, the counters fall into two > clumps, > one for NTS and one for NTS-KE. All the counters get logged in the same > order. > > Should we add counters for IPv4 and IPv6? > I think I'll merge the CPU usage. (getrusage provides separate user and > system CPU slots) > > There are 3 cases for the server side of NTS-KE: > OK: Worked > Failed: SSL got started but something didn't work (http?) > NoSSL: SSL didn't get started (telnet?) > I think I'll add wall/CPU counters for the no-SSL case. > > Should we split things client/server rather than NTS/NTS-KE? With the > current > split, you can't get NTS packets per NTS-KE without looking in two separate > files. > > What's the right type/units for wall time and CPU time? The current code is > double and seconds. That won't ever overflow, but some day in the far future > a very busy KE server might drop low bits. We could solve that by using > timespec. Should I switch to timespec now so we never have to think about > this again? Can ntpq handle timespecs as elapsed times rather than time of > day? ... It probably should be at the point where a double is inadequate to represent current time as a double due to leaking nanoseconds. The NTP solution would be to convert the mess to l_fp which will work for a bit less than 13 years. My joke would be to have it as a long long of micro-seconds which would be good for hundreds of thousands of years. > I'd like ntpq to display both the total and recent counters like sysstats > does. > > The code that does hourly logging has to remember the totals when it prints > things so it can subtract them off from the new totals next time. It's easy > for me to return the old values but slightly inconvenient to do the subtract. > > How hard is it to do the subtract in ntpq? I didn't do it in ntpq. I wrote a macro that builds a couple of getters per attribute (in ntpd/ntp_proto.c), and abused it. Then a wrapper abusing the getters was added in 'ntpd/ntp_control.c'. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Release, project sleeping, and running down the clock.
> On 08/22/2023 3:29 AM PDT Hal Murray wrote: > > > James Browning said: > > If the project is sleeping, then you may as > >well cut the new release now. > > I think we should do a normal release. That includes scanning the issues and > merge requests. And lots of testing. > > I'm working on making a couple of new stats files for NTP packets using NTS > and NTS-KE. That should enable fixing #795 (ntpviz) and provide data for > thinking about #797 (TLS server priority bit). > > I think we should fix the mssntp hack or decide to rip it out. Some people will accuse us of throwing away perfectly good code either way. I am getting bogged down in the weeds working on an asynchronous implementation of the MS-SNTP shim. > > I would > >appreciate it if people commented on my merge > >requests 1331, 1333, and 1335. > > 1335: > sysstats are not resetting hourly when not logged to a file > ntp.poly.polychr() is broken on Python 2 (incorrectly reported as Python > 3 > on the ML) > ./waf requires Python development headers when building no ntp.ntpc or > Python w/ FFI ntp.ntpc > Minor text fixups in many Python files > 9 tests/libntp/*y do not have pyshebang headers > 6 test files, two wscript files, and ntp.agentx all lack SPDX > licenses > declarations > 9 pylib files and 8 test files lack license attribution > wscript files in tests and pylib lack UTF-8 coding lines > > That looks like 4 unrelated things in one MR. There is only one "approve" > button. > I'm interested in the first one, but not interested enough to try to dig it > out from all the other stuff. I can and will drop the other commits quickly enough. > 1333: > 2403ed08 - 1 commit from branch NTPsec:master > 18336fa1 - ms-sntp: works up to trying to connect to the socket. > 5535c882 - ms-sntp: Add excessive logging. > > Do we have a test setup yet? There was a lot of discussion a few weeks ago, > but I don't remember seeing a recipe for one. I have one that seems to need fixing. It keeps returning 4+12 octet messages that Samba wouldn't/couldn't sign the return message. > Again, There is only one "approve" button and I'm not going to approve > anything that says "excessive logging". > > If you need more info for debugging... There are several approaches. > > 1) Use the normal msyslog, but make sure it doesn't turn into a self-DoS. > You can add some rate limiting. I am using msyslog, but I did not add rate limiting. It logs once per packet failure condition, and it is single elimination. I need to look at rete limiting. > 2) You can use the DEBUG stuff. I've never been happy with that -- too much > other printout to wade through. > > 3) You can add the msyslog stuff with an ifdef or if (0) that defaults to > off. You edit the file and rebuild to turn on your extra printout. This > seems to work well if you are likely to want the same printout in the future. > You change a (0) to (1) rather than typing in the whole long line again. > > > 1331: > ntpdig appends a new mac after each host dug > ntpdig can try to take the trusted attribute of a byte string. > the authenticator bits in ntp.packet do not truncate digests > > I can't quite figure out what that means. Does it print something stupid? > Crash? > How would I test it? The first commit addresses running authenticated ntpdig against a hostname with multiple addresses adds an authenticator before getting time from each IP. So the second address gets two, the third three, and so on, causing ntpd to drop them as invalid. The second commit addresses coding errors in ntpdig. It tries to get access to two attributes from a string (think uint8_t[]) the neither has nor can have them. for the third commit if ntpq/ntpdig authenticates with a long hash (possibly sha2/sha3/blake) the authenticator is not truncated at 4+20 bytes causing ntpd to drop the packet as invalid. > My head doesn't like python. I'd be happier if somebody else tested/approved > this. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: What does bison do with undefined tokens?
> On 08/22/2023 12:09 AM PDT Hal Murray via devel wrote: > > > ntpd/ntp_parser.y has this line: > > %token T_Timingstats > > T_Timingstats is never defined. All the other similar tokens are defined in > ntpd/keyword-gen.c I think that is a stub definition. It gets written into the resulting c and h files with a number. Additionally line 658 seems to ensure it get added as a stub option for 'stats'. Properly the lines should probably be filled out or removed. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Release, project sleeping, and running down the clock.
If the project is sleeping, then you may aswell cut the new release now. Otherwise I wouldappreciate it if people commented on my mergerequests 1331, 1333, and 1335. My other mergerequests 1335, 1330, and 1324 can wait a while.It is coming up on 8 months since 1.2.2 wasreleased.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
outstadning? (non-)issues
- ms-sntp is broken - turning ms-sntp on breaks all time service - ms-sntp is blocking on every packet - IPv6 address to refid is broken with FIPS 140-2 - byte strings lack .hostname and .trusted in ntpdig - in ntp.packet, many hashes are not truncated to 20 bytes - when running ntpdig against a name with multiple addresses each address adds a MAC - 6 Python files lack shebang lines, two coding lack encoding lines, 9 lack SPDX statements, and 17 lack copyrights at an earlier check - ntploggps used/uses count_used_satellites before definition - protostats do not reset hourly unless login to a file - polychr() was/is broken on Python 3 ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: NTPsec 1.2.2a released
> On 08/04/2023 6:35 PM PDT Fred Wright via devel wrote: > :::snip::: > > I notice that the two commits for that don't seem to be in any branch. > Having commits only "owned" by a tag and not a branch seems fragile. I do not think it is particularly fragile; I mean the garbage collector is that aggressive, is it? > In the process of investigating this, I noticed that only the master > branch is visible in the WebGUI, even though there are actually 25 > branches in the repo. I also only see the branch master. Then again I am not expecting to see 26 branches because we have 25 tags and master. BTW master at one point or another contained all of those tags. > Having a separate branch for each patch release could get somewhat > cluttered, but it's straightfoward to use a single branch for all patch > releases, as long as folks aren't allergic to merge commits. > > Fred Wright If it bothers you, I could add a merge request to add a 1.2.2 branch. I have a few CI changes to backport. I am not going to create another merge request unless I can get confirmation someone besides me wants it. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: [Git][NTPsec/ntpsec][master] 3 commits: Minor cleanups to restrict
On Jul 5, 2023 22:36, Matthew Selsky via devel wrote:Has this fix been submitted upstream to https://github.com/ThrowTheSwitch/UnityIt has now. https://github.com/ThrowTheSwitch/Unity/pull/680 ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
CI oops in the option testers
The Alpine Linux and Gentoo weekly option testers are not correctly failing on missing Python 2. Also, the CentOS and Fedora runners seem to be missing pkg-config packages. I have a possible partial workaround as !1327___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Warnings from unity
On Jun 21, 2023, 12:54, Hal Murray via devel wrote:> That usually means there is no "default:" case in a switch. OK, but where did the unity code come from and/or have we cloned it or are we tracking what they do? Or ...I think it occasionally gets updated from GitHub [1].[1] https://github.com/ThrowTheSwitch/Unity___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: master broken on macos
> On 04/13/2023 11:57 AM PDT Hal Murray via devel wrote: > > > I merged James' latest version for printing out the OpenSSL version during > configure. > > It barfs on macos. > https://gitlab.com/NTPsec/ntpsec/-/jobs/4105995926#L126 > > > The log file says: > > Checking for OpenSSL != 1.1.1a : yes > OpenSSL version : The > configuration failed > (complete log in /Users/gitlab/builds/NTPsec/ntpsec/build/config.log) > Cleaning up project directory and file based variables 00:00 > ERROR: Job failed (system failure): exit status 2 > > How do I get to the "complete log"? A couple of possibilities: one is to append '||cat build/config.log' to lines 386 and 402 (ish) of .gitlab-ci.yml; the other would involve adding something not quite wholly unlike the following to those YAML branches artifacts: paths: - build/config.log They can be combined, and I need to think of the right questions earlier. The former dumps it in the CI log, and the latter should be in a downloadable tarball ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Updated to-do list?
> On 04/08/2023 2:45 PM PDT Hal Murray wrote: > > Given that the things I have been turning in are not in the direction > > we are headed, what should I be working on? Other than trying to do a > > Golang port by myself, or revisiting the more than a-year-old list? > MR 1307 and/or 1309 is high on my list. I was hoping somebody else would be > interested. Grab a shovel and a Philosopher's Stone. > What's on your year old list? TLDR back in June 2021, I kicked off the previous to-do list by revisiting the in-tree devel/TODO.adoc file [1] and randomly commenting on the points. [2] This led to ESR kicking off the provisional new list with the sole item of porting NTPsec (probably) to Golang [3]. I kicked in my tuppence[4]. * check on the state of C to Rust/Golang transpilers (They sucked) * Archipelligo model but with threads rather than discrete processes to enable per thread seccomp. * Implement a toy that hits all of the (sys)calls used by NTPsec quickly (for seccomp again) * try to break up sizeable opaque code blocks (maybe enabling things like refclockd and testframe) * Testing ** compilers other than GCC/Clang ** kernels other than Linux (token FreeBSD/Darwin) ** other libC implementations, file Hierarchies * Update the docker runners to be better * sort out my miscellaneous merge requests (whatever frippery I think up) with the result primarily being [5] * Port away from C with minor diversions to * Resolve CVEs * close issues to have less than 20 open * More extensive unit-tests * Releases [1] https://gitlab.com/NTPsec/ntpsec/-/blob/master/devel/TODO.adoc [2] https://lists.ntpsec.org/pipermail/devel/2021-June/009761.html [3] https://lists.ntpsec.org/pipermail/devel/2021-June/009775.html [4] https://lists.ntpsec.org/pipermail/devel/2021-June/009779.html [5] https://lists.ntpsec.org/pipermail/devel/2021-June/009798.html > For a small project, I think we should make mode6/ntpq require the cookie on > everything but getting the cookie, and we should make sure that there is no > amplification when getting the cookie. That would break compatibility with ntpq from classic NTP. > How much do you know about Go? Can you write a multi-threaded echo server? > > I have a collection of hacks for measuring performance of NTP servers. Down > hill and with a tailwind, I can get 1,000,000 packets per second. I have C > and Rust versions. It would be neat to try a Go version. I can manage to do that. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Updated to-do list?
Given that the things I have been turning in are not in the direction we are headed, what should I be working on? Other than trying to do a Golang port by myself, or revisiting the more than a-year-old list?___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: MR 1307 and/or 1309
On Mar 31, 2023, 17:27, Hal Murray via devel wrote: Has anybody tried them?Probably only me; I am a perennial screw-up and everything I touch cracks.1307 is my implementation of the socket refclock and 1309 contains an alternative to ntpshmmon from gods and another similar script working on chrony socks.Gpsd should start after and stop before ntpsocklist, otherwise gpsd will not send to the socket or gripe that the listener is dead.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: I've broken something
> On 03/23/2023 4:55 PM PDT Hal Murray wrote: > > James Browning said: > >> Where/how do I get ubuntu-latest? > > I would suggest `docker push registry.gitlab.com/na280/ntpsec`, but it seems > > that it rarely is acceptable, or go to the Ubuntu website and download > > Jammy. > > Are you sure about Jammy? Where did that come from? I am reasonably sure about it. Jammy comes from [1] which come after looking at ./dockerfiles/ubuntu-latest to get ubuntu:latest > I found a download page for Ubuntu 22.04.2 LTS (Jammy Jellyfish) > But 22.04 is almost a year old which doesn't match my expectations for > "latest". > It is the latest for their LTS. > > I have 22.10 (Kinetic Kudu) > It works. > > I have 20.04.6 LTS (Focal Fossa), but that's running on a Raspberry Pi rather > than a PC. > It works. > > I guess I'll settup Jammy on a PC. [1] hub.docker.com/_/ubuntu Seriously though, docker works great for this, and you don't need to glass wipe a machine. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: I've broken something
> On 03/23/2023 12:46 AM PDT Hal Murray via devel wrote: > > > The pipeline fails on: > Name: ubuntu-latest-basic > Name: ubuntu-latest-refclocks > Name: macos-basic > Name: macos-refclocks > > All 4 get the same clump of errors: > :::snip::: > > This code area isn't wonderful. It leaves a lot of stuff in global variables > so it can decrypt stuff it just encrypted. The code works on all my test > cases. > > Anybody know what version of OpenSSL macos or ubuntu-latest are using? There > was a new version released recently: 3.0 => 3.1, I think. None of the other > distros I test with are using 3.1 yet. ubuntu-latest runs 3.0.2 [1], and my Ubuntu box with 3.0.8 passes. > Where/how do I get ubuntu-latest? I would suggest `docker push registry.gitlab.com/na280/ntpsec`, but it seems that it rarely is acceptable, or go to the Ubuntu website and download Jammy. > We should patch the configure stuff to print out the version of OpenSSL that > it finds. I have a branch that does that badly. [2] [1] https://gitlab.com/na280/ntpsec/-/jobs/3987240899 [2] https://gitlab.com/na280/ntpsec/-/commits/tlscheck ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Be very quiet, the project is dozing off.
This is the second post this month, and it's three weeks after my dropped patches on the first.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Here are a couple of patches related to seccomp,
I wrote and tested these on a bleeding-edge Ubuntu box. I have yet to
try this on other Linux flavors. First is a patch to make the secomp
trap handler on Linux more helpfully verbose. Then a patch that can
incrementally tighten the syscall filter to calls listed in a text
file.
The patch in the previous mail had the arguments backward for the
syscall resolving functionFrom 8b99fd609647ff0567c83d0d05991c874ce8278e Mon Sep 17 00:00:00 2001
From: James Browning
Date: Mon, 27 Feb 2023 16:30:51 -0800
Subject: [PATCH 1/2] Attempt to make seccomp errors useful not Lassie
---
ntpd/ntp_sandbox.c | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/ntpd/ntp_sandbox.c b/ntpd/ntp_sandbox.c
index 369602357..8d82c8edd 100644
--- a/ntpd/ntp_sandbox.c
+++ b/ntpd/ntp_sandbox.c
@@ -568,8 +568,11 @@ static void catchTrap(int sig, siginfo_t *si, void *u)
UNUSED_ARG(u); /* unused ucontext_t */
msyslog(LOG_ERR, "ERR: SIGSYS: got a trap.\n");
if ( si->si_syscall ) {
- msyslog(LOG_ERR, "ERR: SIGSYS/seccomp bad syscall %d/%#x\n",
- si->si_syscall, si->si_arch);
+ char *call = seccomp_syscall_resolve_num_arch(
+ si->si_arch, si->si_syscall);
+ msyslog(LOG_ERR, "ERR: SIGSYS/seccomp bad syscall %d(%s)/%#x\n",
+ si->si_syscall, call, si->si_arch);
+ free(call);
}
#ifndef BACKTRACE_DISABLED
backtrace_log();
--
2.38.1
From 187249e022aee2ad9ea0d3a2a88c6873d84f8d12 Mon Sep 17 00:00:00 2001
From: James Browning
Date: Mon, 27 Feb 2023 10:39:29 -0800
Subject: [PATCH 2/2] Remove seccomp code from sandbox(), parsing a fixed text
file
---
devel/seccomp-list.txt | 162 +
etc/seccomp.list | 106 +
include/ntpd.h | 1 +
ntpd/ntp_sandbox.c | 262 ++---
ntpd/ntpd.c| 10 ++
5 files changed, 341 insertions(+), 200 deletions(-)
create mode 100644 devel/seccomp-list.txt
create mode 100644 etc/seccomp.list
diff --git a/devel/seccomp-list.txt b/devel/seccomp-list.txt
new file mode 100644
index 0..60b5d074f
--- /dev/null
+++ b/devel/seccomp-list.txt
@@ -0,0 +1,162 @@
+SCMP_SYS(accept),
+SCMP_SYS(access),
+ SCMP_SYS(adjtimex),
+ SCMP_SYS(bind),
+ SCMP_SYS(brk),
+ SCMP_SYS(chdir),
+ SCMP_SYS(clock_adjtime),
+ SCMP_SYS(clock_gettime),
+ SCMP_SYS(clock_settime),
+ SCMP_SYS(close),
+ SCMP_SYS(connect),
+ SCMP_SYS(exit),
+ SCMP_SYS(exit_group),
+ SCMP_SYS(fcntl),
+ SCMP_SYS(fstat),
+ SCMP_SYS(fsync),
+ SCMP_SYS(futex), /* sem_xxx, used by threads */
+ SCMP_SYS(getdents), /* Scanning /etc/ntp.d/ */
+ SCMP_SYS(getegid), /* Needed on Alpine */
+ SCMP_SYS(getgid), /* Needed on Alpine */
+ SCMP_SYS(getdents64),
+
+#ifdef __NR_getrandom
+ SCMP_SYS(getrandom), /* Added in 3.17 kernel */
+#endif
+#ifdef __NR_ugetrlimit
+ SCMP_SYS(ugetrlimit), /* sysconf */
+#endif
+#ifdef __NR_getrlimit
+ SCMP_SYS(getrlimit), /* sysconf */
+ SCMP_SYS(setrlimit),
+#endif
+ SCMP_SYS(getrusage),
+ SCMP_SYS(getsockname),
+ SCMP_SYS(getsockopt),
+ SCMP_SYS(gettimeofday), /* mkstemp */
+ SCMP_SYS(getuid), /* Needed on Alpine */
+ SCMP_SYS(ioctl),
+ SCMP_SYS(link),
+ SCMP_SYS(listen),
+ SCMP_SYS(lseek),
+ SCMP_SYS(membarrier), /* Needed on Alpine 3.11.3 */
+ SCMP_SYS(munmap),
+ SCMP_SYS(newfstatat),
+ SCMP_SYS(open),
+#ifdef __NR_openat
+ SCMP_SYS(openat), /* SUSE */
+#endif
+ SCMP_SYS(poll),
+ SCMP_SYS(pselect6),
+ SCMP_SYS(read),
+ SCMP_SYS(readv), /* nscd getaddrinfo() provider */
+ SCMP_SYS(recvfrom),/* Comment this out for testing.
+* It will die on the first reply.
+* (Or maybe sooner if a request arrives.)
+*/
+ SCMP_SYS(recvmsg),
+ SCMP_SYS(rename),
+ SCMP_SYS(rt_sigaction),
+ SCMP_SYS(rt_sigprocmask),
+ SCMP_SYS(rt_sigreturn),
+#ifdef __NR_rseq
+ SCMP_SYS(rseq), /* needed by glibc-2.35+ for resumable sequences */
+#endif
+ SCMP_SYS(sigaction),
+ SCMP_SYS(sigprocmask),
+ SCMP_SYS(sigreturn),
+#ifdef __NR_select
+ SCMP_SYS(select), /* not in ARM */
+#endif
+ SCMP_SYS(sendto),
+ SCMP_SYS(setsid),
+#ifdef __NR_setsockopt
+ SCMP_SYS(setsockopt), /* not in old kernels */
+#endif
+ SCMP_SYS(socket),
+ SCMP_SYS(socketcall), /* old kernels */
+ SCMP_SYS(stat),
+ SCMP_SYS(statfs64), /* from getaddrinfo after lid open */
+#ifdef __NR_time
+ SCMP_SYS(time), /* not in ARM */
+#endif
+ SCMP_SYS(sysinfo),
+#ifdef HAVE_TIMER_CREATE
+ SCMP_SYS(timer_create),
+ SCMP_SYS(timer_gettime),
+ SCMP_SYS(timer_settime),
+#else
+ SCMP_SYS(getitimer),
+ SCMP_SYS(setitimer),
+#endif
+ SCMP_SYS(write),
+ SCMP_SYS(writev), /* Needed on Alpine 3.11.3 */
+ SCMP_SYS(unlink),
+
+/* Don't comment out this block for testing.
+ * pthread_create blocks signals so it will crash
+ * rather than generate a trap.
+ */
+ SCMP_SYS(clone), /* threads */
+ SCMP_SYS(clone3),
+ SCMP_SYS(kill), /* generate signal */
+ SCMP_SYS(madvise),
+ SCMP_SYS(mprotect),
+ SCMP_SYS(set_robust_list),
+ SCMP_SYS(sendmmsg), /* DNS lookup */
+ SCMP_SYS(socketpair),
+ SCMP_SYS(statfs),
+ SCMP_SYS(uname),
+
Here is a patch to make the secomp trap handler on Linux more helpfully verbose.
I wrote and tested this on a bleeding-edge Ubuntu box. I have yet to
try this on other Linux flavors.
I also have a patch that can incrementally tighten the syscall filter
to calls listed in a text file. I will work on that one a bit more
first, however. The code for both is also at
https://gitlab.com/na280/ntpsec/-/commits/23B27-seccompFrom 8b99fd609647ff0567c83d0d05991c874ce8278e Mon Sep 17 00:00:00 2001
From: James Browning
Date: Mon, 27 Feb 2023 16:30:51 -0800
Subject: [PATCH 1/2] Attempt to make seccomp errors useful not Lassie
---
ntpd/ntp_sandbox.c | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/ntpd/ntp_sandbox.c b/ntpd/ntp_sandbox.c
index 369602357..8d82c8edd 100644
--- a/ntpd/ntp_sandbox.c
+++ b/ntpd/ntp_sandbox.c
@@ -568,8 +568,11 @@ static void catchTrap(int sig, siginfo_t *si, void *u)
UNUSED_ARG(u); /* unused ucontext_t */
msyslog(LOG_ERR, "ERR: SIGSYS: got a trap.\n");
if ( si->si_syscall ) {
- msyslog(LOG_ERR, "ERR: SIGSYS/seccomp bad syscall %d/%#x\n",
- si->si_syscall, si->si_arch);
+ char *call = seccomp_syscall_resolve_num_arch(
+ si->si_arch, si->si_syscall);
+ msyslog(LOG_ERR, "ERR: SIGSYS/seccomp bad syscall %d(%s)/%#x\n",
+ si->si_syscall, call, si->si_arch);
+ free(call);
}
#ifndef BACKTRACE_DISABLED
backtrace_log();
--
2.38.1
___
devel mailing list
devel@ntpsec.org
https://lists.ntpsec.org/mailman/listinfo/devel
Re: New Defects reported by Coverity Scan for ntpsec
On Feb 7, 2023 18:23, Hal Murray via devel wrote: Thanks. matthew.sel...@twosigma.com said: > No. We run the Coverity CI job weekly via a schedule, ... > I'll work on running Coverity post-merge. I agree that running it every merge is overkill. A button that says run-now would be nice if we are working on fixing Coverity problems. Can you poke it by hand? Not as such, no. But it is easy for an authorized user to trigger a scheduled run at GitLab. It's under ci > schedules on the left sidebar.How does Coverity fit into the release procedure?"Check with the buildbot reports, assure that there are no unplanned regressions on the supported platforms."Along with flawfinder and semgrep presumably.Should we schedule releases after a Coverity run? I think it probably should be.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: issue blocking input_handler
> On 02/03/2023 11:35 PM PST Hal Murray wrote: > > James Browning said: > >> How about fixing the bug at ithe source rather than patching around it? > > I tried and failed then I came up with that. > > Did you get O_NONBLOCK turned on? > Something like this: > err = fcntl(sockfd, F_SETFL, O_NONBLOCK); I did not, but it is now. Working, and HTML page source added. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: issue blocking input_handler
> On 02/03/2023 2:51 PM PST Hal Murray wrote: > > James Browning said: > > I found a problem with input_handler(). If a reference clock passes a > > blocking file descriptor, input_handler can block forever. > > Who did that? Which driver? I ran across it working on a chrony socket refclock. The scrap is in merge request !1307. I have a suspicion about the gpsd JSON driver as well. > How about fixing the bug at ithe source rather than patching around it? I tried and failed then I came up with that. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
issue blocking input_handler
I found a problem with input_handler(). If a reference clock
passes a blocking file descriptor, input_handler can block
forever. Symptoms include but are not limited to UDP sockets not
working, several signal types being ignored, and the main loop
stopping in its' tracks. I've included a patch to handle it for
you, but there has to be a better way.From 2da42fec2b314fa9dd60ab4190b6b686f7eeeca0 Mon Sep 17 00:00:00 2001
From: James Browning
Date: Fri, 3 Feb 2023 11:14:54 -0800
Subject: [PATCH] Fix a small issue in input_handler with excessive glue.
---
ntpd/ntp_io.c | 23 ---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/ntpd/ntp_io.c b/ntpd/ntp_io.c
index 0b00cd1c9..d1abe3071 100644
--- a/ntpd/ntp_io.c
+++ b/ntpd/ntp_io.c
@@ -2355,9 +2355,26 @@ input_handler(
maintain_activefds(fd, true);
} else {
/* drain any remaining refclock input */
- do {
-buflen = read_refclock_packet(fd, rp);
- } while (buflen > 0);
+ do
+ {
+fd_set rfds;
+struct timespec no = {0, 0};
+FD_ZERO();
+FD_SET(fd, );
+if(1 == pselect(
+ fd + 1, , NULL, NULL,
+ , NULL
+ ))
+{
+ buflen = read_refclock_packet(
+ fd, rp
+ );
+}
+else
+{
+ buflen = 0;
+}
+ } while(buflen > 0);
}
}
#endif /* REFCLOCK */
--
2.38.1
___
devel mailing list
devel@ntpsec.org
https://lists.ntpsec.org/mailman/listinfo/devel
Re: First step on ntp_control cleanup
On 02/02/2023 1:13 AM PST Hal Murray wrote: > > > ntp_control is the code in ntpd that processes mode6 requests from ntpq. > > There are 3 types of variables you can read: > global variables (rv 0) > peer (server) variables > refclock variables > > I've fixed up the first type. It is now easy to add new global variables. > Most will be a 1 line edit. > > There is still lots of ugliness to clean up. > > Please test, I may have broken a few obscure cases. I fixed a few. Clock variables seem not to be working. Also sys_var_list outputs the daemon_version variable. > I tweaked a few cases of the ntpq printout to indent a few clumps of counters > that are a subset of the slot above the indented ones. It seems natural to > me. Feedback? > > I'm scheming to cleanup sysstats. Is there anything you have wanted to know > that it doesn't show? ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: New SHM layout from gpsd
> On 01/20/2023 2:11 AM PST Hal Murray wrote: > > > The 31 bit idea seems strange/ugly to me. How did you decide to do it that > way? It is either Richard's fault or, more likely, mine. I proposed replacing the current SHM, and I need to communicate better. My alternate had a shared half-era counter instead of the upper 32 bits for each time stamp. It is the sort of EEish micro-optimization that some people would fawn over. It was also going to use POSIX shared memory which is out. > Why is it better than 32 unsigned bits? Is there some case that works with 31 > bits that breaks with 32? No benefit you would consider worth the downsides. > I think there is a case that works for 32 unsigned that doesn't work for 31. > Consider code that gets updated to use 64 bit time_t but they forget to update > the SHM interface. That will pick up the 32nd bit and do the right think for > another 68 years. NTP-associated time beginnings... (skip this probably) 1968-01-20T03:14:08 - second half of NTP era 0 1970-01-01T00:00:00 - POSIX era 0 1980-01-01T00:00:00 - GPS week 0 1999-08-17T00:00:00 - GPS week 1024 2019-04-08T00:00:00 - GPS week 2048 2036-02-07T06:28:16 - first half of NTP era 1 2038-01-19T03:14:08 - second half of POSIX era 0 2038-11-16T00:00:00 - GPS week 3072 2058-07-02T00:00:00 - GPS week 4096 2078-02-15T00:00:00 - GPS week 5120 2097-10-01T00:00:00 - GPS week 6144 2104-02-26T09:42:24 - second half of NTP era 1 2106-02-07T06:28:16 - first half of POSIX era 1 2117-50-18T00:00:00 - GPS week 7168 2137-01-01T00:00:00 - GPS week 8192 > An alternative would be to make the new high-bit slots into 64 bits and make > the rule use-them, ignore the old slot. That would eat 2 more dummy words. I would prefer we kill it and put something a little better specified in its' place. Not the thing I originally specced because sockets are the new awesome. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
I am bidding for power and have yet more branches for consideration.
If I were a maintainer of the NTPsec group, I could merge the following branches on my own. If only a developer, I could still approve other people's merge requests. https://gitlab.com/NTPsec/ntpsec/-/merge_requests/1304 !1304 - Enable debugging symbols by default, with an option to !disable them. https://gitlab.com/NTPsec/ntpsec/-/merge_requests/1305 !1305 - Update the NEWS.adoc file with things that should be in !it that have not made it there. https://gitlab.com/NTPsec/ntpsec/-/merge_requests/1272 !1272 - Adds a preempt option for clocks, making them act as if !the pool option added them. https://gitlab.com/NTPsec/ntpsec/-/merge_requests/1299 !1299 - Change the shared memory refclock to be compatible with !the latest gpsd. https://gitlab.com/NTPsec/ntpsec/-/merge_requests/1290 !1290 - Use the ntp.poly module, check it, and fix polychr() !when it does not behave correctly. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: ✘64-bit time_t on glibc 2.34 and up
> On 01/13/2023 1:06 PM PST Hal Murray wrote: > > > If we make any changes to SHM, we should switch to a setup where the memory is > read only. The idea is to allow multiple readers. > > The trick to implementing that is to have 2 counters. > X and Y are initialized to the same value. > The writer bumps X, updates the data, then bumps Y. > The reader grabs Y, grabs the data, then grabs X. > If X and Y are the same the data is valid. If not, try again. I've heard you mention this before, and I specced it by calling them bookends instead and sticking them at opposite ends of a 4KiB page-sized struct. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
My ignorance was Re: ✘64-bit time_t on glibc 2.34 and up
> On 01/12/2023 5:10 PM PST Gary E. Miller wrote:
>
>
> Yo All!
>
> Cross-posted to gpsd-dev and devel@ntpsec.org
>
> Recent glibc (2.34 and up) and recent Linux kernels, allow 64 bit
> time_t on 32-bit Linux without much work.
>
> But...
>
> How to get that 2038 compatible time to ntpd and chronyd? That is a
> much bigger problem.
>
> Extracted from include/ntpshm.h:
>
> struct shmTime
> {
> int mode;
> volatile int count;
> time_t clockTimeStampSec;
> int clockTimeStampUSec;
> time_t receiveTimeStampSec;
> int receiveTimeStampUSec;
> int leap; // not leapsecond offset, a notification code
> int precision; // log(2) of source jitter
> int nsamples; // not used
> volatile int valid;
> unsignedclockTimeStampNSec; // Unsigned ns timestamps
> unsignedreceiveTimeStampNSec; // Unsigned ns timestamps
> int dummy[8];
> };
>
> Note the struct size depends on the size of an int, and the size of time_t.
>
> This is no problem for newer musl on 32-bits. An int is 32-bits and
> time_t is 64. Assuming all clients use the same version musl.
>
> This is a problem for glibc on 32 bits. And int is 32-bits, but time_t
> is a compile time option (32 or 64 bits).
>
> How does ntpd know what size time_t to use? And thus know the size of
> shmTime? How do we know portably, preserving backwards and forwards
> compatibility?
>
> In hindsight, maybe shmTime should have started with a 1 char version
> field,or magic field. But, no such luck.
>
> Options (for 32-bit only):
>
> 1. Do nothing, stick with 32-bit time_t. Fail in 2038.
>
> 2. Allow 64-bit time_t and let incompatible ntpd fail.
>
> 3. Add run time options to gpsd and ntpd to specify time_t size.
>
> 4. gpsd and ntpd always use 64-bit time_t going forward. Admin needs
> to mix and match.
>
> 5. 1st process to open SHM(0) wins, the other process checks the size
> to know the contents.
>
> 6. Create a new way to pass time from gpsd to ntpd and chronyd.
>
> Also note, chrony sockets have a similar problem:
>
> #define SOCK_MAGIC 0x534f434b
> struct sock_sample {
> struct timeval tv;
> double offset;
> int pulse;
> int leap; // notify that a leap second is upcoming
> int _pad;
> int magic; // must be SOCK_MAGIC
> };
>
> Where timeval is:
>
> struct timeval {
> time_t tv_sec;
> suseconds_t tv_usec;
> };
> ```
>
> Since the sample has a magic value, maybe that can be used to detect versions.
>
> This makes my head hurt...
IIRC there are a few users of those interfaces; linuxptp, gpsd,
classic NTP, NTPsec, and chrony. I would jokingly suggest
something like the following as a replacement. Purely based on
the misassumption that SHM internally works on page-sized blocks
and probably via POSIX shared memory which would allow names.
Also, the mode should be 640, and the root user should not be
remotely involved.
```c
typedef struct {
uint32_t left_book;
uint32_t mode; // modes 4-7 match this
uint32_t half_epoch; /* number of 2^31 seconds since
1900-01-01T00:00:00Z */
uint32_t c_seconds; /* coarse time from string mangling
probably */
uint32_t c_nseconds;
uint32_t f_seconds; /* fine time probably from a pulse
per second */
uint32_t f_nseconds;
signed int32_t leap;
signed int32_t precision;
uint32_t padding[1013]; // I think it should be 1014
uint32_t right_book;
} pshm_t;
```
The low two bits of mode could be the validity of coarse and
fine time, respectively. Anyone needing microsecond or NTP
partial seconds can math it.
Posted to gpsd issue #152 earlier, gpsd-dev, and NTPsec devel.
- JamesB192
I'm supposed to put a signature here...
___
devel mailing list
devel@ntpsec.org
https://lists.ntpsec.org/mailman/listinfo/devel
I want to suggest four prioritized merge requests for inclusion.
I suggest that four of my merge requests be evaluated for inclusion. 1298 is the highest priority. It fixes a zero-day issue where the CI runner for Python coverage fails because the underlying distribution has shifted. After that would be 1290, which changes the ntpclients to use ntp.poly more tests ntp.poly, and revises it when there is undesirable behavior. 1272 adds the preempt option for clock sources, allowing clocks from sources other than the pool to be automatically removed if they fail to meet quality standards. Merge request 1258 includes a copy of the fudge option language from classic NTP and then changes it. If any feature of these isn't good, I would appreciate a heads-up to drop them from consideration. Also, I am temporarily shelving the other. Check again next week though. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Three infrastucture CI builds for consideration.
I suggest that three of my merge requests be evaluated for inclusion. 1253 has a script and documentation, which should greatly simplify the release process for next time (estimated to be late June/ early July) with documentation for it. Also, it adds an update package repo feature to buildprep (except for zypper). 1256 should update the build version code and support dist and distcheck features in the same. Also, it adds the HTML documentation to the tarball (if available) under ./html while sticking the man pages (again, if available) under ./man . 1292 should ix the builder and runner to test cross-building targeting armhf. If any feature of these isn't good, I would appreciate a heads-up to drop them from consideration. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: clockmaker
> On 12/23/2022 7:51 AM PST NTPfiend via users wrote:
>
>
> My instance of ntpsec gave up after just over a year uptime, so I went for a
> re-installation (later R Pi, now 64 bit). Today (23 Dec) I attempted to run
> clockmaker but it did not get far:
>
> [Line 210]
> SyntaxError: Missing parentheses in call to 'print'. Did you mean
> print("Configuring locale...")?
>
> I made the suggested mod, which looks like a Python2 to Python3 thing, and
> the same happened 2 lines later, and for a number of other lines. "Fixing"
> the lot, clockmaker is now running and doing what it should do. No smoke
> (yet) as a result of my hack.
>
> I'm a user not a coder, hence the posting here. It's entirely possible I've
> misunderstood something.
No one has tried to update clockmaker, and has stuck around
long enough for it to take.
Merge requests have been pushed to NTPsec/www to see if
they adhere.
Ignore the evil crossposting.
___
devel mailing list
devel@ntpsec.org
https://lists.ntpsec.org/mailman/listinfo/devel
Re: We need to test leap smearing :)
> On 12/22/2022 7:49 PM PST Hal Murray via devel wrote: > > > Fred Wright said: > > If you make it 24 hours, there's the question of whether that means 86400 > > seconds or 86401. :-) > > That ones easy. It's 86400 smeared seconds and 86401 real seconds. > > That's the whole point of smearing. :) Stretching 86399 seconds need not apply. Also, planning for no leap seconds is probably premature. Could you check back when the race of mortal men is no more or the sun has burned out? Maybe not even then. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: tests/option-tester is broken
> On 12/20/2022 12:24 AM PST Hal Murray wrote: > > > > Bumping waf (2.0.23 tested) resolves this, and it only happens for the > > extension because, for some raisin, waf is outputting to the wrong file name > > of ntpc rather than ntpc.cpython-311-x86_64-linux-gnu.so , go figure. > > > The default of the FFI library + ctypes is unaffected. > > Interesting. Thanks. > > waf 2.0.23 doesn't fix it for me. Maybe I didn't do it right. > Do I just download it and raname it to waf? Or do I have to do some magic to > install it? > > One of the options being tested by test-all is --enable-pylib=ext > What does that do? > > test-classic does --enable-pylib=ffi > That seems to work. `curl -o waf https://waf.io/waf-2.0.24` The link on the page said 2.0.23 . There are two methods atm to provide ntp.ntpc the first is a Python extension that goes in the same directory as the rest of the NTP module, and the newer is a C library that can be used by using and Foreign Function Interface wrapper (on Python that's ctypes). The latter is the default, and the same pure Python module can be used for multiple Python versions. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: tests/option-tester is broken
> On 12/19/2022 4:19 PM PST Hal Murray via devel wrote: > > > Anybody remember this area? > > With a fresh clone on Fedora which uses > Python 3.11.0 > > > test-all/test.log:Trouble with test-all > > Looking in test-all/test.log > > The build looks clean. It dies in the checking that all the programs run well > enough to print their version info: > > running: main/ntpd/ntpd --version FAILED > Expected: ntpd > > Got (stdout): ntpd ntpsec-1.2.1+135-g62cb02f74 > > The Got looks good to me so the tester is broken? > > Then the python stuff > > running: main/ntpclients/ntpdig --version FAILED > Expected: ntpdig > > Got (stderr): ntpdig: can't find Python NTP library -- check PYTHONPATH. > No module named 'ntp.ntpc' > > I've forgotten how this stuff works. I think the "." in there is really a > slash. Python is looking for ntp/ntpc.py > > I see ./test-minimal/main/pylib/ntpc.py in all the others but not in test-all > Same for libntpc.so > > The log file shows: > [ 96/137] Linking test-all/main/pylib/ntpc > with no errors. > > > I get the same results after I chop off my 2 recent commits. Bumping waf (2.0.23 tested) resolves this, and it only happens for the extension because, for some raisin, waf is outputting to the wrong file name of ntpc rather than ntpc.cpython-311-x86_64-linux-gnu.so , go figure. The default of the FFI library + ctypes is unaffected. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: More cookie keys
> On 12/18/2022 6:02 PM PST Hal Murray via devel wrote: > > > The commit message for that is lacking the blank line after the summary > > line. This means that some git tools treat the entire commit message as the > > summary, creating obnoxiously long lines in their output. It's too late to > > fix the existing commit message without a forced update, but it would be > > good to avoid that in the future. > > Thanks for the heads-up. > > This seems like a bug in git. Is there a reason they haven't added a check > for this? I think it should go in a hook that gets called before the commit happens. > What is a forced update? > > Is there any way to delete commits from gitlab? If it were local, I could > just nuke the top two and resubmit. Well, no, but actually, yes. It involves locking, unprotecting, and forced pushing, followed by re-protecting and unlocking the 'master' branch of the remote repository. Quite possibly with a side of garbage collecting housekeeping/maintenance commands. IIRC. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Release coming, take 2
> On 12/15/2022 1:21 PM PST Matthew Selsky via devel wrote: > > > Hi all, > > I plan to cut a release on Thu 12/22/2022. I'm sorry about the delays in > getting this release out the door. > > If there's anything that absolutely must be in this release and can't wait > until the next release, please let me know. Otherwise, I plan to tidy up the > NEWS file as needed and then ship what we have. The first commit of !1294 must be in place when the release happens, some variant of the second should be in place, and the third could but can probably wait until afterword. They make devel/make-tarball (but only w/ git), update the NEWS file, and add a make-tarball testing runner. All my other cra^d^d^d stuff can wait. > Going forward, I'll try to make sure that I'm doing releases more regularly, > maybe 6 months or so? > > Thanks everyone for your patience. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: devel/make-tarball broken
> On 12/14/2022 3:05 PM PST Hal Murray via devel wrote:
>
>
> Thanks for testing and fixing.
>
> I'm not familiar with that corner of tar/sed. What does the extra S do?
> Where is that documented?
> tar --transform="s:^:ntpsec-${V}/:" ...
> tar --transform="s:^:ntpsec-${V}/:S" ...
'Do not apply transform to symbolic link targets' [1] Which I
got from googling 'gnu tar transform' IIRC. It is also a
gnuism, but I do not see a portable transform type option as
tar on macOS uses -s and some BSDs seem to lack even that.
> I added PIVOT.h last Jan 29, so this has been broken for almost a year.
>
> Is gitlab running option-tester weekly or monthly? Can it test make-tarball
> too?
I don't think we need a bunch of runners testing devel/make-tarball,
one should be sufficient.
> Matt:
> I have updated PIVOT.h
> Is testing make-tarball on Mark's list?
> (before doing the mechanics of the release)
despite not being Matt...
no, but it's called in devel/release before any repo changes
are made.
[1] https://www.gnu.org/software/tar/manual/html_section/transform.html
___
devel mailing list
devel@ntpsec.org
https://lists.ntpsec.org/mailman/listinfo/devel
devel/make-tarball broken
[178/315] Compiling libntp/clockwork.c ../../libntp/ntp_calendar.c:18:10: fatal error: PIVOT.h: No such file or directory 18 | #include "PIVOT.h" | ^ compilation terminated. Default verbosity log and minimal solving patch attached. I also think merge requests !1256, !1253, and !1237 should be merged, and that !1292, !1290, and others should be evaluated (although probably later). + basename /home/jamesb/ntpsec/devel + [ devel != devel ] + set -e + [ != ] + cat ../VERSION + V=1.2.1 + echo version is 1.2.1 version is 1.2.1 + rm -fr .tmp + cd .. + git ls-files + find build -print + grep \.[0-9]$ + cd .. + tar --transform=s:^:ntpsec-1.2.1/: -T devel/MANIFEST -czf ntpsec-1.2.1.tar.gz + rm MANIFEST + mv ../ntpsec-1.2.1.tar.gz . + set -e + mkdir .tmp + cd .tmp + tar -xzf ../ntpsec-1.2.1.tar.gz + cd ntpsec-1.2.1 + ./waf configure --refclock=all --enable-doc Setting top to : /home/jamesb/ntpsec/devel/.tmp/ntpsec-1.2.1 Setting out to : /home/jamesb/ntpsec/devel/.tmp/ntpsec-1.2.1/build Checking for program 'asciidoctor' : /usr/bin/asciidoctor Checking for asciidoctor version >= 1.5.8 : 2.0.16 --- Configuring host --- Checking for 'gcc' (C compiler) : /usr/bin/gcc Checking compiler version : 12.2.0 Checking for program 'ldconfig' : /usr/sbin/ldconfig Checking for program 'm4' : /usr/bin/m4 Checking for program 'bison' : /usr/bin/bison Checking for program 'awk': /usr/bin/awk Checking for program 'sh' : /usr/bin/sh Checking for program 'pkg-config' : /usr/bin/pkg-config Checking for systemd : yes systemd unit directory: : /lib/systemd/system --- Configuring main --- Checking for endianness : little Checking for library m: yes Checking for library rt : yes Checking for library pthread : yes Checking for library execinfo : not found Checking for library bsd : not found Checking for library ssp : not found Checking for library ssp_nonshared: yes Checking if C compiler supports -fstack-protector-all : yes Checking if C compiler supports -fPIC : yes Checking if C compiler supports -pie -fPIE: yes Checking if C compiler supports -Qunused-arguments: no Checking if C compiler supports -Wcast-qual : yes Checking if C compiler supports -Wdisabled-optimization : yes Checking if C compiler supports -Wfloat-equal : yes Checking if C compiler supports -Wformat: yes Checking if C compiler supports -Wformat-security : yes Checking if C compiler supports -Wformat-signedness : yes Checking if C compiler supports -Wimplicit-function-declaration : yes Checking if C compiler supports -Winit-self : yes Checking if C compiler supports -Winvalid-pch : yes Checking if C compiler supports -Wmissing-declarations : yes Checking if C compiler supports -Wmultichar : yes Checking if C compiler supports -Wpacked: yes Checking if C compiler supports -Wpointer-arith : yes Checking if C compiler supports -Wshadow: yes Checking if C compiler supports -Wsuggest-attribute=noreturn: yes Checking if C compiler supports -Wwrite-strings : yes Checking if C compiler supports -flto : yes Checking if linker supports -Wl,-z,now : yes Checking if linker supports -Wl,--strip-all : yes Checking if linker supports -Wl,-z,relro: yes Checking for type struct if_laddrconf : no Checking for type struct if_laddrreq: no Checking for type struct timex : yes Checking for type struct ntptimeval : yes Checking for field time_tick in struct timex: no Checking for field modes in struct timex: yes Checking for field time.tv_nsec in struct ntptimeval: no Checking for field tai in struct ntptimeval : yes Checking sizeof long: 8 Checking sizeof time_t (time.h) : 8 Checking for OpenSSL/libssl (via pkg-config): yes Checking for OpenSSL/libcrypto (via pkg-config) : yes Checking for function _Unwind_Backtrace : yes Checking for function adjtimex : yes Checking for function backtrace_symbols_fd : yes Checking for function ntp_adjtime : yes Checking for
Re: build on Python 2.6 broken and strongly suggested minimum version
On Dec 8, 2022, 13:22, Hal Murray via devel wrote: > I also think that we should drop all Python versions before 3.7 from > first-tier support and only continue supporting them if it is not > inconvenient or there is sufficient proven demand. Older but still supported versions of Debian and CentOS are still using Python 2, but it's 2.7 rather than 2.6. I think we should keep supporting 2.7. But I don't do much work with Python. How much of a pain is supporting Python 2? It is not currently much of a pain to support Python 2. I was trying to phrase it so that if no one is (provably) using it or it became a hassle, we would have an out.> Given the lack of complaints, I suspect no one uses 2.6 anymore, which can be > dropped. Is there a supported distro that is still using 2.6?2.4- OS4Depot2.5- Maemo Fremantle2.6- Centos 6- Rosa Server 6.92.7- AIX- BlackArch- Slackware 14.2Everything else has Python 3 as an option.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
build on Python 2.6 broken and strongly suggested minimum version
Selsky broke the build[1] back in May 2020 when switching the python_scripts variable in /wscript to a set when Python 2.6 does not support sets. Given the lack of complaints, I suspect no one uses 2.6 anymore, which can be dropped. I also think that we should drop all Python versions before 3.7 from first-tier support and only continue supporting them if it is not inconvenient or there is sufficient proven demand. [1] 574d871ed 2020-05-01 "Use a set instead of a list for python_scripts so we don't have duplicates"___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: getargspec gone in Python 3.11?
On Dec 8, 2022, 01:05, Hal Murray via devel wrote: I have a system using Python 3.11 It says: == ERROR: test_packetize (__main__.NtpqRvInfoStats.test_packetize) Test ntp.util.packetize by coqtavoric gavage and scatology. -- Traceback (most recent call last): File "/home/murray/ntpsec/play/ntp4/main/tests/pylib/test_util.py", line 1346, in test_packetize shot_test(self, ntp.util.packetize, cases) File "/home/murray/ntpsec/play/ntp4/main/tests/pylib/test_util.py", line 1364, in shot_test if "msg" in inspect.getargspec(classy.assertEqual).args: ^^ AttributeError: module 'inspect' has no attribute 'getargspec' -- It worked with Python 3.10 Google found this: https://stackoverflow.com/questions/61933427/deprecationwarning-inspect-getar gspec-is-deprecated-since-python-3-0 DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0My bad. I have a workaround in !1291 until they deprecate unittest.TestCase.subtest or hasattr or some such. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Fork this was Re: Fixing the dang CI pipelines, release
> On 11/27/2022 7:42 AM PST James Browning via devel wrote: > > > Around July 10, 2022, I authored several commits to improve the success rate > of the CI pipelines. At the time, I was too daft to realize that some changes > I made broke two of the on-push Debian stable builders. In the time since the > on-push macOS builders have broken. There are seven merge requests not > authored by me at the moment, and the only way to merge them is to override > the usual merge request process. Someone should merge request 1281, which I > authored to disable the offending feature in the two builders. I will add > another merge request to allow the failure of the on-push macOS runners. > > Also, the last merge into the tree was in mid-August; Enough time has passed > that a release can be made. > Given the general lack of commits to fix the CI pipeline for not quite five months, apparent lack of work on whatever queues there are to join the group and project, releases, etc. I am temporarily forking NTPsec until at least one, and hopefully, all of those issues will be resolved. (plus any I've missed). Content will probably be posted to ntp.jamesb192.com with the name (at least) partially changed to 'gallus ntp' with appropriate asset changes (but not today). This missive is crossposted to the users and devel lists at ntpsec.org; please remove the other list unless replies are relevant to both lists. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Fixing the dang CI pipelines, release
Around July 10, 2022, I authored several commits to improve the success rate of the CI pipelines. At the time, I was too daft to realize that some changes I made broke two of the on-push Debian stable builders. In the time since the on-push macOS builders have broken. There are seven merge requests not authored by me at the moment, and the only way to merge them is to override the usual merge request process. Someone should merge request 1281, which I authored to disable the offending feature in the two builders. I will add another merge request to allow the failure of the on-push macOS runners. Also, the last merge into the tree was in mid-August; Enough time has passed that a release can be made.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
CI and things
I quit the project a while back and have yet to be allowed to join the group (greater scope). I am more useless than usual for the duration, as I can't tag my work or bless other peoples' work. I had forgotten that I could not merge my work for 6-10months before that (fun). I have a branch 2022f03-ci-patch that solves/works around some periodic CI runners' issues. Was there going to be a release soonish? If so, my merge request can wait. Otherwise, the project should be woken up but not stunned. Also, the mailing list seems laggy. Is there anything that anyone can do to reduce the lateness?___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
re: Authentication in ntpq
Hal Murray wrote:
> Some commands in ntpq require authentication.
> I just tried it, and it wants an MD5 password. MD5 is way old. Is there a way
> to switch to AES?
The default is set to MD5 in ntpq:69 and packet.py:729; might I suggest
updating it to 'AES-256' or "SHA{,3-}512"? They would get truncated until
(presumably) NTPv5 comes in and replace the authentication trailer with an
extension.
The keytype command(?) is also available via ntpq -c "keytype AES-256" and the
'keyid' command to go with it. Help is in the usual places.___
devel mailing list
devel@ntpsec.org
https://lists.ntpsec.org/mailman/listinfo/devel
Fwd: ntpsec | A list of everything I've worked on, can find, and hasn't been merged recently. (#739)
> Subject: ntpsec | A list of everything I've worked on recently, can find, and > hasn't been merged. (#739) > James Browning (https://gitlab.com/jamesb_fe80) created an issue: #739 > (https://gitlab.com/NTPsec/ntpsec/-/issues/739) commits, lines changed, merge request number, distorted branch name |--:|:|--:|:--- | 1 | 8 | !1233 | [1] 21f04-trifecta | 4 | 80 | !1237 | 21f09-pylib-mode6-auth | 2 | 658 | !1253 | 2022b07-release | 4 | 304 | !1255 | 2022b15-fast-ntpq | 4 | 70 | !1256 | 2022b15-fast-waf-distcheck | 1 | 15 | !1258 | [1] 2022c14-doxxing | 4 | 535 | !1259 | 2022c17-bug-hunt | 5 | 110 | !1260 | 2022c15-ntpviz-splat | 1 | 40 | !1271 | 2022e18-lfp-abs | 1 | 15 | !1272 | 2022e21-preempt | 1 | 9 | !1273 | 2022e10-no-polychr | 1 | 9 | !1274 | 2022e10-ntpheat | 1 | 6 | !1275 | 2022e22-suse | 1 | 14 | | 2022c22-pthread-1 > 1. Used to be longer... > 2. Fail to clear up failing OpenSUSE runners re libaessiv tests With the expected delay, this should drop onto the NTPsec devel mailing list ridiculously early Monday morning in UTC-7. People not on GitLab should comment on the list. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Release, wildcards, etc
On Apr 20, 2022 07:30, Matt Selsky via devel wrote:Hi Hal, I'd like to get https://gitlab.com/NTPsec/ntpsec/-/merge_requests/1264 merged and then do the release. Is there anything else that we want in the release?Yes, but in the interest of helping get it out on this calendar the rest of my merge requests can wait. Joining the NTPsec group at GitLab would be nice.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Getting ready for a release, wildcards
On Apr 20, 2022 7:50 AM, Matt Selsky via devel wrote:Hi Hal, I don't think we should have a knob for disabling wildcards. This is not the sort of knob that operators expect (what other software provides such a knob?) and we're just adding another code path to test. Are there any other release blockers? If not, I'll update the NEWS for the user-facing/high impact changes and cut a release candidate.Inconsistent output formatting for 'ntpq sysstats'? Bytes output of test data in 'waf check -v'? Potentual mac truncation in Python 3? Nah.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: NTPsec on Windows ??
Hey Achim,On Feb 16, 2022 9:43 AM, Achim Gratz via devel wrote:Hal Murray via devel writes: > Can you run it again with -k? That might find some more. That doesn't change anything for the configure phase.Could you go through an remove all the mandatory=True instances and equivalent from configure. Then would you run build verbosified and get the arguments of a compile and maybe a couple links? Regards, Achim. I'm sure there are dash style arguments specific to libaes_siv, I don't think it would get built for Windows though.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: waf: --disable-attic
On Feb 15, 2022 1:16 PM, Hal Murray via devel wrote: I have a simple patch that will turn off building things in attic/ Should we make off be the default and change that to --enable-attic?Probably, tomorrow never comes. How's it going with dropping Python 2.6, oh we aren't.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Time for a release??
On Feb 15, 2022 1:05 PM, Hal Murray via devel wrote:> How about a release at the end of February to give people time to wrap things up? The group that started this would like it by the 24th.No, but there's always the next one. I do have lists though. I am not in a position to push my stuff past a certain point. A RC (Release Candidate) would be a placeholder. Have we ever done one?.Never as such, essentially though this would be a stretch to release. We've done those, this time just has an early tarball. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: NTPsec on Windows ??
On Feb 14, 2022 1:08 AM, Hal Murray via devel wrote:Does anybody have a POSIX c environment on Windows and/or have a friend who does?I'd have to blow off the dust, also not strictly Posix even if using WSL2.. I'd like to know how close we are to being able to run on Windows. I expect several low level routines may be missing. ntp_adjtime is a good example. I'd like too get a list.Not close. There should still be adapters in the history. It's not that simple due to wrong flags. It should be a simple as mkdir xxx cd xxx download using tar wget https://ftp.ntpsec.org/pub/releases/ntpsec-1.2.1.tar.gz tar -xzf ntpsec-1.2.1.tar.gz cd ntpsec-1.2.1 or using git git clone g...@gitlab.com:NTPsec/ntpsec.git cd ntpsec Then: ./waf configure -k ./waf build -k -k tells waf to keep going rather than bail on the first fatal error. waf needs Python (2 or 3) We need bison We expect OpenSSL libraries (but see below) We need what Linux/BSD calls a python development environment. If Windows doesn't have the equivalent, we should be able to build ntpd without it. It's only needed by utilities like ntpq and friends. But that will take a small amount of work to add an option to configure.Which built python on windows? The one from cl, mingw/cygwin, or wsl(2?) (Rhetorical) This will drop the need for libssl ./waf configure --disable-nts but we still need libcryptoTls stuff look like it's listed in the "Security and Identity" section. There may be other optional requirements that I can't think of right now. This may provide info: ./waf configure --help The source of buildprep may be helpful --- Of course, I don't know anything about Windows. So, it may be much more complicated than that. If so, I'd like to get a high level view of what it would take to get far enough to learn what low level APIs are missing.Iirc our wscript files force the wrong kind of command line arguments for cl and possibly also clang on windows.I'd happily donate a pitcher of beer (or the local equivalent) to the project if we can arrange the logistics. ___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: Time for a release??
On Feb 13, 2022 10:42 PM, Hal Murray via devel wrote:> How about a release at the end of February to give people time to wrap things up? Sounds good to me. Who has what that needs wraping up?I have some stuff I think needs to be in:- simplified release machinery !1253 #603- mode 6 client side auth fixes !1237Also a lot of stuff I want in:- move keygone out of contrib/- libaes_sev test fix !1238- some bugfixes !1241, !1233,And other stuff that can probably wait for 1.2.3 or 1.3... Richard is working on the Trimble driver. I need to take another look at the NMEA driver. - >> It's been almost a year since 1.2.1 I scrambled reading some dates. It's only 6 months. Eh, it's blursday again.___ devel mailing list devel@ntpsec.org https://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq version troubles
On Fri, Jul 9, 2021 at 4:15 PM Hal Murray via devel wrote: > > > When I start ntpq, I get this message: > ntp.ntpc wrong version '1.2.1+17-gbc270426b-dirty 2021-07-09/11:45:21' != > '1.2.1' It's probably a result of me trying to be clever. IIRC it's comparing the version of the ntp python module (left) with the version of the FFI stub (right). Neither the source nor the build system seems to have changed in the interval. It's looking at the libntpc.so from a tarball built package and the python module built from git. > It works OK. > > That's built with > --build-desc=xxx on the configure line to waf > on a system using git with some local edits. > > It works with the same command line on a system without git. > > ./waf check is happy. > running: main/ntpclients/ntpq --version OK > It works when run by hand out of $build. > $ ./hgm/main/ntpclients/ntpq --version > ntpq ntpsec-1.2.1+17-gbc270426b-dirty 2021-07-09/11:45:21 > $ > > Is this a problem with my setup? Maybe getting the wrong library? > Or a bug in the version check and/or install code? Not a problem for you/your installs specifically. Getting the packaged library which should be the same save for the version string. Certainly an issue with the version check code. Should only be happening for developers. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Support for i386
On Wed, Jun 30, 2021, at 11:59 AM Hal Murray wrote: > > > > The build epoch has been replaced with a hardcoded timestamp which will be > > manually updated every nine years or so (approx 512w). This makes the > > binaries reproducible by default. > > What source file holds that timestamp? libntp/ntp_calendar.c in ntpcal_get_build_date > Where is it used? > One place is the version string. Where else? Not used in the version string by default. step_systime in libntp/systtime.c wrapdate_init in ntpd/ntp_wrapdate.c trimble_start in ntpd/refclock_trimble.c I seem to have misplaced my call graph generator. I will not be digging deeper than that. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Support for i386
On Wed, Jun 30, 2021, at 5:17 AM Hal Murray via devel wrote: > > > e...@thyrsus.com said: > > The remaining blocker is that the NTP packet format would need to be > > redesigned. > > No, we just have to play the wrap around game when converting from network > time to unix time. If the network time is before the build time add 1<<32 > seconds. And drop the high bits when going in the other direction. > > Pick your favorite constant if you don't like build time. But then you have > to remember to update it occasionally. Too late, I already changed it that way at the end of December (!1197 I think) f76af8b7, I messed up again. Remove --build-epoch and replace it with arbitrary --build-desc text. Passing '--build-desc=$(date -u +%Y-%m-%dT%H:%M:%Sz)' restores the previous default extended version. The build epoch has been replaced with a hardcoded timestamp which will be manually updated every nine years or so (approx 512w). This makes the binaries reproducible by default. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Work plan prpoosal for the next year
I do not consider myself an expert C developer. I think the proposed schedule is overly serial. ntpkeygen and keygone for example have no dependency on pylib/ IIRC. Also none of the other ntpclients/ depend on ntpq. This would (in theory) pull CLIENTS up to month 5 (late November/December) I think the open 'issue' count would be lower if people actually tended to close issues. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
Are there any C to golang or rust transpilers that work reasonably well? The last time I checked the best rust transpiler generated rs files that were just shallow glosses and the golang transpiler was somewhat inadequate and verbose. An early-started long-lived tread for each of DNS, clock- manipulation, and client I/O. Probably with several for refclocks and server functionality. Linux seccomp can work per thread but only subtractively. It would be great if we had a sandbox test tool that hits everything NTPsec uses in a few minutes. Then we could update the sandboxing so that downstream would not have to. Maybe break up some of the less testable code blocks into more manageable pieces and have tests for them. Tests deeper than just getting the version string. Tests on platforms that are not GNU/Linux w/ GCC. Tests on other architectures, see if there is a way to update docker for the runners. re: Gentoo and recent Ubuntu failures. sort out everything in my scrap piles and get the good bits merged. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
On Fri, Jun 18, 2021, at 6:08 AM Hal Murray via devel wrote: > > As long as we are in blue sky mode... > > What was the name for your attempt to get a GPSD style replay of old data? > Did we ever figure out why that didn't work? It was called testframe IIRC... https://blog.ntpsec.org/2017/02/22/testframe-the-epic-failure.html > The GPSD code is one way: Input => output. There is no back and forth, no > request => response which changes internal state. Does that pattern make > things too complicated? If we plan ahead, could we do something that would > help? ... Maybe there are some huge opaque blocks of code that do not act reliably. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: visiting the todo list
On Tue, Jun 8, 2021, at 5:09 AM Hal Murray wrote: > > > devel@ntpsec.org said: > > IIRC gpsd has a read-only monitor program. I'm not sure how much of the code > > could be applied to revise ntpd/refclock_shm.c, but I think that quite a bit > > of it could. Which with it hopefully being read-only addresses a couple of > > bullet points. I also seem to remember that some PTP software can export via > > the same SHM struct which would drag in a third. > > gpsmon uses a TCP connection to gpsd. There is no shared memory involved. cgps will use shared memory if "shared memory" is passed on the command line. gpsmon is a bit like ntpd/ntp_signd.c ignored except for getting backhanded every time warnings pop up. > What sort of monitor do you have in mind? > With only a little hacking, we could make NMEA sentences available. Probably something a lot like a clone of ntpshmmon (from the gpsd tree), which does read that shared memory mapping NTPsec uses. JSON is not the subject of those bullet points. > > I had a foolish notion for a revised SHM driver but it just dropped off the > > list. > > There is a way for the client/reader to be read only. Fixing this has been on > my list for years. > > The idea is to use two counters. Call them X and Y. The writer does: > Bump X > update data > Y = X > The reader does > grab Y > grab data > grab X > if grabbed X and Y are equal, data is valid. > The key is that the reader references X and Y in the opposite order from how > the writer updates them. If the reader and writer are accessing the data at > the same time, the reader gets the old Y and new X. I was thinking something more JSON-like parseable, with unlimited extensibility and distinct edges. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpleapfetch regression.
On Mon, Jun 7, 2021, at 8:44 AM Mike Simpson via devel wrote:
>
> Hi folks.
>
> Ntpleapfetch was coping with the leap-seconds.list file from the ietf for a
> bit then it regressed back to being confused by the formatting of the
> signature again.
I see it. potential patch attached later. TLDR the spacing did it
six-month mess with the downstream dance adding a space. macOS,
Solaris and 15 BSD variants will probably break.
> Also, we are still trying to use /var/NTP for I believe NTS related logging.
> Can this be moved to /var/log/something as SELinux really doesn’t like it.
> nts or ntp/nts?
I'll have to pencil in a reminder to submit a change to the default
value of NTP_VAR which is only used for 'statsdir', from there it
branches a bit.
> Speaking of SELinux, I can send a patch to their ML to allow creation and use
> of the NTS tcp port now that it’s all iana assigned as I have the necessary
> custom module to let fedora and centos work.
Above my paygrade probably. I would say submit it there as well as our
repository for packaging/.
diff --git a/ntpclients/ntpleapfetch b/ntpclients/ntpleapfetch
index 5e8500b..039b8a6 100755
--- a/ntpclients/ntpleapfetch
+++ b/ntpclients/ntpleapfetch
@@ -310,7 +310,7 @@ verifySHA1() {
# or maybe a space and a tab. remove the tab, wherever it may be. Add some 0x
# so it can be run through printf to restore missing leading zeros.
#
-local FSHA1="`grep '^#h' $1 | tr -d '[:cntrl:]' | sed -e's/^#h/0x/' -e's/ / 0x/g'`"
+local FSHA1="`grep '^#h' $1 | tr -d '[:cntrl:]' | sed -e's/^#h[ \t]*/0x/' -e's/ / 0x/g'`"
FSHA1=`printf '%08x%08x%08x%08x%08x' $FSHA1`
if [ -n "$FSHA1" -a \( "$FSHA1" = "$DSHA1" \) ]; then
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
visiting the todo list
IIRC gpsd has a read-only monitor program. I'm not sure how much of the code could be applied to revise ntpd/refclock_shm.c, but I think that quite a bit of it could. Which with it hopefully being read-only addresses a couple of bullet points. I also seem to remember that some PTP software can export via the same SHM struct which would drag in a third. I had a foolish notion for a revised SHM driver but it just dropped off the list. I am reasonably sure that the pool association code needs to be revisited later. I seem to recall it over replaces a dropped server and then culls. Maybe a mechanism to limit the number of ephemeral remote cocks per source. Replacing IPv4 and IPv6 only sockets could probably work unless it hit a hypothetical misrouted multihomed server. Then it would not know where to send packets from a bit like my 'twinsock' reduction. snprintb looks incredibly useful, I agree, it should get used more, quite possibly even with the proposed ntpshark helper the bulk of which I think should part of pylib and accessible to ntpq and ntpdig. I'm sure that file capabilities would help reduce the need for initial root and drop root as mentioned. I proposed to break ntpd into separate processes (rant redacted). Replacing the once per second timer would be nice for power saving. How about an unbalanced binary tree for event tracking. I think it'd be O(n) memory and O(log n) to O(n) insert/lookup/delete times. I'd also suggest removing the uptime counter in favor of a monotonic start time variable. I probably messed up the chance of adding reconfiguration needed detection and possibly the version string cleanup. Also, the implementation now does not like (semi-)shallow git clones. Are the changes to the documentation and the conformant SNMP subagent done? ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
release probably today
My vote is to release, and I suspect other developers will mostly pass. It would probably be nice if some developers could wander back. huh, I have more commits recently than everyone else combined. 56 James Browning 15 Richard Laager 12 Matt Selsky 8 Hal Murray 3 Mark Atwood 3 Miroslav Lichvar 2 Brendan Bank 1 Eric S. Raymond 1 Sami Farin ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: get ready for 1.2.1 on 2021-06-01
On Thu, May 27, 2021, at 10:08 AM Mark Atwood via devel wrote: > > Hi! > > Please get ready for 1.2.1 on 2021-06-01 > > Major contributors, please chime in, and merge your work that you think is > ready. > Please update the …/NEWS and the …/devel/TODO files with descriptions of your > work. > > This will be the least release cranked out by myself. Matt Selsky will be the > release manager henceforth. I think quite strongly that !1229 should be fixed & merged or 66ed781a -- `ntpq: Display dual column protocol stats ...` reverted. Currently ntpq in git head is incompatible with NTP classic and ntpsec prior e8b59ad4 two commits before. !1228 will probably get merged Saturday and only change CI runner config. Stop reading now. I have several branches that are not/might never be ready. - bonjour registration for macOS w/o CFLAGS+=" -ldns_sd" - a rewrite of the OpenSSL helper that does not work on macOS atm - A branch to reduce the number of UDPx sockets to 2 that will never get tested properly. - a branch to add aes_siv tests that breaks on OpenSUSE IIRC - an alternate vision of the SHM driver I can't imagine properly - apparently an attempt to get the clients working on MS Windows And other stuff I have long forgotten. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: user variables, anybody use them?
On Wed, Mar 10, 2021 at 2:16 PM Hal Murray wrote: > > >> The server side of ntpq supports writing and reading things of the form > >> "foo=bar". > > >> I'm slightly surprised Eric hasn't riped that out by now. > > >> Does anybody use it? How/why? ,,, > > > No, I don't. I had this foolish notion though (or it might've been > Achim) not > > long after (during?) the rainbow control debacle about prepending a > modifier > > character (say @ or such) to a variable which is a list of variables and > it > > would return all of the variables in that list. > > The current code has some slots marked DEF (default). I think it is only > used by ntpq sysinfo. The implementation is "rv 0" or just "rv" with an > empty > list of names that it wants to retrieve. > DEF is also implemented for 'cv #' and 'rv #', IIRC. > My plan was to add more flags. I'm happy with your @ suggestion. > > Please start collecting a list of things you would like to retrieve this > way. > The 2 that I use often are the NTS statistics and the MRU statistics: > ntsinfo > and monstats. > That does not seem to be how sysinfo works. There probably should be a request for peoples' lists on the users, devel, and maybe announce lists. The hopefully complete list (from ntpq) is as follows. l_sysinfo="peeradr,peermode,leap,stratum,precision,rootdelay,rootdisp,rootdist,refid,reftime,sys_jitter,clk_jitter,clk_wander,authdelay" l_kerninfo="koffset,kfreq,kmaxerr,kesterr,kstflags,ktimeconst,kprecis,kfreqtol,kppsfreq,kppsstab,kppsjitter,kppscalibdur,kppscalibs,kppsjitexc,kppsstbexc,kppscaliberrs" l_sysstats="ss_uptime,ss_numctlreq" l_sysstats2="ss_reset_r,ss_received_r,ss_thisver_r,ss_oldver_r,ss_badformat_r,ss_badauth_r,ss_declined_r,ss_restricted_r,ss_limited_r,ss_kodsent_r,ss_processed_r,ss_reset,ss_received,ss_thisver,ss_oldver,ss_badformat,ss_badauth,ss_declined,ss_restricted,ss_limited,ss_kodsent,ss_processed" l_monstats="mru_enabled,mru_hashslots,mru_depth,mru_deepest,mru_maxdepth,mru_mindepth,mru_maxage,mru_minage,mru_mem,mru_maxmem,mru_exists,mru_new,mru_recycleold,mru_recyclefull,mru_none,mru_oldest_age" l_authinfo="authreset,authkeys,authfreek,authklookups,authknotfound,authencrypts,authdigestencrypts,authcmacencrypts,authdecrypts,authdigestdecrypts,authdigestfails,authcmacdecrypts,authcmacfails,authkuncached,authexpired" l_ntsinfo="nts_client_send,nts_client_recv_good,nts_client_recv_bad,nts_server_recv_good,nts_server_recv_bad,nts_server_send,nts_cookie_make,nts_cookie_decode,nts_cookie_decode_old,nts_cookie_decode_too_old,nts_cookie_decode_error,nts_ke_probes_good,nts_ke_probes_bad,nts_ke_serves_good,nts_ke_serves_bad" l_iostats="iostats_reset,total_rbuf,free_rbuf,used_rbuf,rbuf_lowater,io_dropped,io_ignored,io_received,io_sent,io_sendfailed,io_wakeups,io_goodwakeups, l_timerstats="timerstats_reset,timer_overruns,timer_xmts" I have two separate lists for sysstats atm because I have them processed through different handlers. which would get worse if I did packets/s columns. More likely I could add another merge request to get ignored for the duration. Also the list id for a recent git version. 1.2.0 lacks the *_r variables from sysstats. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Five column output and old compatibility in branch 21C11-stat-quad branch please test.
This is mostly what it says in the title. It prints out ??? in columns where data is unavailable. This only mostly works for sysstats at the moment, as a couple of fields are included from elsewhere and not dualized in the daemon. Please, test this and tell me what it looks like broken and where. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq meets log files
On Wed, Mar 10, 2021 at 4:51 AM Hal Murray via devel wrote: > > James Browning said: > >> We need to keep in mind how to make this work in a multi threaded > >> environment. > > I think you would need to deglobalize some variables first. > > What do you have in mind? > I tend to overfocus on early optimizations and other trivial things. Neither of the arguments passed to read_clockstatus from the converted mockup is used, instead, the variable res_associd is pulled from being global. > I think there is an Atomic "library" that is mostly the compiler being > smart > enough to recognize a few special cases. I'm pretty sure the Intel > hardware > directly supports > foo += xx > and hence > foo++ > if you use the right instructions. So we have to use the right macros but > we > don't need locks. > > There are a few places where it might be nice to update a set of counters > at > the same time. For example, the number of packets received and the number > processed or dropped or ... I'm willing to live with that fuzz on a busy > system. > IIRC, I saw that was mentioned as a C11, feature and not C99. > > > > >> There is a slight complication. We need the name that comes back with a > >> variable to be a meaningful description. > > So, drop the Millsian object notation (or whatever it is called) for > > something closer to JSON? Protobuf need not apply. > > I'm not interested in JSON. > > We could add a pretty name to the table and a way for ntpq to ask for the > pretty name. > You lost me put up a mockup, please. > > > > > For example, after butchering all the following it occurred to me that > one > > could just ad a CB (callback) or HOOK value. I mean it is not like there > is a > > sudden lack of space for them. > > I wouldn't have called it "callback". There is no back part. > > My straw man is that each slot in the table has a name (that ntpq asks > for), > flags, type tag, pointer to variable, and print routine. My print routine > is > probably your callback. I'm expecting that we will need a half dozen or > so > print routines for the common cases (int, long int, unsigned long..., > float, > double...) and a half dozed or so slots that need a dedicated print > routine. > Unless you make the handlers smarter I think there will be more than 6 corner cases that need extra help. It would probably be best to separate the low easy corners from the more difficult. ctl_putdata and ctl_putunqstr are probably lower-level implementations than you intend. ctl_putuint is common, ctl_putint less so. ctl_putts, ctl_putadr, ctl_putrefid, ctl_putarray, and ctl_putstr are also already there. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: More on ntp_control cleanup
On Wed, Mar 10, 2021 at 4:15 AM Hal Murray via devel wrote: > > How do I compute at compile time the offset of a field into a struct? > > The context is putting it into a table. > ]offsetof(struct, field)[ wrapped in macros. I pulled that from the include/json.h file of gpsd > I know about structs and unions. Unions are type-unsafe. Is there a > type-safe way to handle a struct that needs several variations? > > The struct will have a type field to tell me which variation it is. I > will > have a macro to build each variation.There is only one place that > looks at > the variations. It will do a select on the type field with a few lines of > code for each variation. So all in all, it's pretty clean. I'm just > looking > for a way to do better. > I typed in ]c tagged union[ after getting curious a while back, my bulimic parrot level instant expert credentials for it are long expired. I remember though that I just hit the first SO post[1] on the topic. [1] https://stackoverflow.com/questions/42844423/writing-a-safe-tagged-union-in-c ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq: user variables, anybody use them?
On Sun, Mar 7, 2021 at 10:17 PM Hal Murray via devel wrote: > > The server side of ntpq supports writing and reading things of the form > "foo=bar". > > I'm slightly surprised Eric hasn't riped that out by now. > > Does anybody use it? How/why? ,,, > No, I don't. I had this foolish notion though (or it might've been Achim) not long after (during?) the rainbow control debacle about prepending a modifier character (say @ or such) to a variable which is a list of variables and it would return all of the variables in that list. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpq meets log files
On Mon, Mar 8, 2021 at 2:28 PM Hal Murray via devel
wrote:
> I'm poking around in ntp_control because I want to add the CPU time that
> ntpd
> has used for some tests I'm trying to run.
>
> We have our share of crufty code, but this is the stuff that annoys me the
> most. What should be simple turns into a pain in the ass.
>
> -
>
> We have several log files that record statistics hourly. After they write
> out
> a line, they clear the counters. ntpq can't ask for the totals since
> restart
> - the data is gone.
>
> James has recently fixed part of that. ntpq sysstats now prints 2 columns.
>
> I'd like to open this area up for discussion.
>
> I'd like the counters behind all log files to keep the totals too. There
> are
> 2 ways to implement that. One is for the active counters to be the
> totals.
> The hourly logic would maintain a second copy that holds the value to be
> subtracted from the totals to get the this-hour values. The other is for
> the
> active counters to hold the this-hour values. To get the totals you would
> add
> the values from the second copy. I think I prefer the active counters to
> hold
> the totals but I have not strong preferences.
>
> We need to keep in mind how to make this work in a multi threaded
> environment.
>
I think you would need to deglobalize some variables first.
> It would be neat to have 2 more columns - packets per second. I don't
> know if
> that will fit.
>
> I'd like to have a ntpq command for each log file. I like the 2 column
> approach. Currently, the slots in each column have a separate name. We
> could
> compress that by returning something like:
> foo=23,45678
> that is 2 values for one name.
>
Not until the ramp to 2.0, That screams here lie incompatible changes.
> ntpq could save a copy of the data so it could print the since-xxx values.
>
> We could get last-day values by saving 24 copies of the last hour data.
>
> There is at least one comment in ntpq.py saying, roughy, this table should
> be
> in ntpd so that we don't have to edit here too when something changes.
> "rv 0"
> returns a subset of a big table marked "default". We could easily add
> more
> flags, one for each log file.
>
> There is a slight complication. We need the name that comes back with a
> variable to be a meaningful description.
So, drop the Millsian object notation (or whatever it is called) for
something closer to JSON? Protobuf need not apply.
> What's the best way to clean up this mess? We discussed this before, but
> I
> don't remember any consensus, or anybody stepping up to do their favorite
> way.
>
I messed up a little which is where the CV_NAME and CASE_ macros came from.
> The current approach needs 3 (I think) tables. One to define a set of
> tags,
> one to translate the name of each variable to a tag, and the 3rd to print
> out
> the data for that tag.
>
4+ tables, at least 3 long lists of preprocessor macros, and too much
copy/paste code.
> I propose we dump the tags and put everything for each slot into struct.
> What does it need?
> name that ntpq uses to ask for this slot
> flags
> pointer to value to be printed
> routine to print it
>
> We can setup macros to build the struct.
>
> Does that make sense?
>
Not really, but I am not very effective. I would like to be able to
implement something like the following, but it is probably too fussy...
For example, after butchering all the following it occurred to me that one
could just ad a CB (callback) or HOOK value. I mean it is not like there is
a sudden lack of space for them.
```py
clock_var_e = {
'name': { RO|DEF|QSTR, 'clockname' },
'timecode': { RO|DEF|QSTR, 'p_lastcode' },
'poll': { RO|DEF|UINT, 'polls' },
'noreply': { RO|DEF|UINT, 'noresponse' },
'badformat': { RO|DEF|UINT, 'badformat' },
'baddata': { RO|DEF|UINT, 'baddata' },
'flags': { RO|DEF|UINT, 'flags' },
}
clock_var_h = {
'fudgetime1': { RO|DEF, clock_do_fudgetime1 },
'fudgetime2': { RO|DEF, clock_do_fudgetime2 },
'stratum': { RO|DEF, clock_do_stratum },
'refid': { RO|DEF, clock_do_refid },
'device': { RO|DEF, clock_do_device },
'clock_var_list': { RO, clock_do_varlist },
}
def read_clockstatus(buffer, statusmask):
if not (REFCLOCK)
ctl_error(CERR_BADASSOC)
return
if res_id == 0:
peer = findpeerbyassoc(res_associd);
else:
if sys_vars.sys_peer and
FLAG_REFCLOCK_vars.sys_peer.cfg.flags)):
peer = sys_vars.sys_peer;
else:
for (peer = peer_list; peer; peer = peer->p_link):
if (FLAG_REFCLOCK & peer->cfg.flags)
break;
if not (peer and peer.flags_REFCLOCK)
ctl_error(CERR_BADASSOC)
return
refclock_control(>srcadr, NULL, );
qset = set()
gotvar = False
iset = set(qstr.split(','))
for key in iset:
if key in clock_var_e or key in clock_var_h:
qset = qset.add(key)
else:
Re: What is the purpose of devel/ifdex-ignores?
On Sat, Mar 6, 2021, 3:33 PM Hal Murray via devel wrote: > From ./devel/ifdex-ignores > USE_SCM_BINTIME # to grab timestamp for recv packet > USE_SCM_TIMESTAMP # " > USE_SCM_TIMESTAMPNS # " > > None of those symbols are used by our code. > > Should I just delete them? > > What is the idea for USE_xxx? Is there some interesting history I've > forgotten? > They seem to have moved there around 11efdf9fa35bfcfc4adeabe168a4f35f2e0c5f2c 2015 Sep 30 from HAVE_*, dunno my instant expert credentials expired. Probably to indicate intent to use not just presence. > ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: We need to capture data on corner cases
On Fri, Feb 5, 2021, at 3:20 PM Hal Murray via devel wrote: > > [Context is retrying DNS lookups when an interface appears.] > > James Browning said: > > When combined with some other code in the DNS path it is wrong-headed. > "let's > > retry DNS every 5 minutes or whenever someone acts on the netlink > socket, and > > pack on extra pool servers until we have twenty." It will probably come > back > > in a diminished form later. If you absolutely must dns_try_again > periodically > > it a couple of single line fixes to ntp_io:837 and ntp_io:325, switching > to > > true and maybe a new period respectively. > > Our code is full of quirks/hacks that catch obscure real-world cases. > Most of > the time, the code is just clutter. On the other hand, if your > environment > triggers that case, that small chunk of code is wonderful. > > I think we need a structured way of collecting and documenting this sort > of > thing. > > Should we put a big comment in the code? With a stylized tag so we can > find > them all. If you are working on other stuff, that's just adding to the > clutter. > I read on the Internet that comments are useless. I occasionally notice them despite the fade to gray tendency. Yeah, I ripped out the following. It was essentially invisible in my not-an-ide and the rat brain is fallible. - /* - ** Classic Bug 2672: Some OSes (MacOSX, Linux) don't block spoofed ::1 - */ > Should we collect the descriptions in another file and add pointers both > ways? > Possibly. If it happens I would like to start the name suggestions with 'codescars' after [1] and something I thought I had read on an Internet site but can no longer find there. [1] http://www.labelscar.com/ ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
