[Citadel Development] Remove the warning-this-is-unfinished warnings from ctdldump.c and ctdlload.c
Most of these tools are now building successfully on FreeBSD.
[Citadel Development] only build chkpw and chkpwd if we can
[Citadel Development] Re: Removed background and restart from citserver.
All righty then. The current code in git master is running on FreeBSD. Here's the deal: 1. You need ldap client library (`pkg install openldap26-client`) even if you aren't using LDAP 2. `gmake` and `gcc` are required. It won't build with whatever FreeBSD is using natively. 3. The configure script now detects whether it needs `-lresolv` and/or `-lintl` automatically 4. We now omit the `chkpw` and `chkpwd` programs on systems where they won't work. I'd actually like to remove system auth completely but I don't know if anyone is using it. On my FreeBSD system I was able to get the server running and I ran the new `loadtest` utility with a few dozen threads beating the crap out of it. I abruptly killed the server with SIGKILL and SIGQUIT numerous times, during periods of high write activity, and even under the worst conditions it did not corrupt the database. Looking really good here.
[Citadel Development] loadtest: run with a warning instead of exiting when not all threads connect.
[Citadel Development] Include -lresolv only if we have it
[Citadel Development] Include -lintl where we need it on systems that have it
[Citadel Development] Re: Removed background and restart from citserver.
UUU--- Drive 0 on my desktop is fux0red. This was my main workstation and also where I had my FreeBSD vm installed. Time to rebuild.
[Citadel Development] Re: Removed background and restart from citserver.
nd just like that, my main machine at home bit the dust. *grumble*
[Citadel Development] Re: Removed background and restart from citserver.
>So FreeBSD's rc system sends a SIGTERM when you do "doas service >citserver stop". This should run the same code that cleanly shuts >down citserver when you do a ".ATN y" from the text client. All the Perfect. That's exactly the desired behavior, and it will be "even more that way" now. I have to go examine other BSD rc scripts, but I'm guessing they simply do "&" to put a process in the background, and then they keep track of the pid so they can SIGTERM it later. SIGINT and SIGTERM are handled by running the very same shutdown code that the server's "DOWN" command runs (which is what the text client's "<.A>dmin erminate-server ow" command executes). Unmount the databases cleanly, then exit(0) to GTFO. What's new, however, is that the database "environment" is now on mmap+disk instead of just in-memory. I don't know how we went for so many years without knowing that we had that wrong. This accomplishes two things: 1. The server no longer goes SIGSEGV when we try to close the environment 2. The database never, ever shows up as corrupted when we restart it In other words, the damn journaling system works correctly now. The funny thing is, it used to work fine even in its old form, so I don't know what's changed in the last few versions of Berkeley DB. And here we are, with a BDB implementation that's now rock solid, as a result of a two month long development effort intended to get us *off* BDB. But with that done, I'm gonna focus on FreeBSD for a while.
[Citadel Development] loadtest: pretty display when running with hundreds of threads
[Citadel Development] fulltext: be smarter about exiting silently if there's nothing to do.
[Citadel Development] fulltext: after the indexer runs for 60 seconds (tunable), yield the thread
[Citadel Development] loadtest: minor changes
[Citadel Development] Re: Removed background and restart from citserver.
So FreeBSD's rc system sends a SIGTERM when you do "doas service citserver stop". This should run the same code that cleanly shuts down citserver when you do a ".ATN y" from the text client. All the BSDs opurrate this way, and since it is a standard way of doing things, SMF (Solaris/OpenSolaris/Illumos) also supports this method of daemon shutdown. Probably also a good idea to handle SIGQUIT and SIGKILL this way (for the odd platform that says these aren't untrappable). If FreeBSD's rc system can't get a process to exit cleanly after a short period of time, it will SIGKILL, and that's most likely lead to database corruption on my installation. Thu Aug 31 2023 23:39:48 EDT from IGnatius T Foobar Subject: Re: Removed background and restart from citserver. citserver should exit cleanly when it receives a SIGINT, SIGQUIT, or SIGKILL. This may be why using FreeBSD's rc system caused badness on my system. And it so happens that FreeBSD is the reason I pulled that section of code out. This is an opportunity to remove all of the "clever" stuff and really focus on portable, standard code. I do feel that there's no longer any reason why a server program should have to do the work to put itself into the background. The operating system's supervisor program should be doing that. I'm sure I'll be learning more about how FreeBSD's rc system works. Seems like it's held pretty closely to Unix System III. That's fine, because I've always hated the System V init scripts. I did release what we've got so far as Citadel 991, with some of the FreeBSD changes in place, but I'm going to keep poking at it until it builds cleanly without needing help. Also I have an ACloudGuru account paid for by my employer and that lets me stand up cloud sandboxes for testing things. I'm going to start using that for an occasional build test, I think. The only problem is that the sandboxes automatically self-destruct after four hours, so it's not useful for much more than testing. Right now I've got the latest Easy Install build running on a Scamazon instance, with a 30-thread load test pounding away at it. I figure it's better to abuse their SSDs than mine.
