[Citadel Development] Re: serv_extensions.c: style cleanup from 30,000 feet
git push from an airplane? showoff Mon Nov 27 2023 11:43:38 EST from rss <> Subject: serv_extensions.c: style cleanup from 30,000 feet No, I'm not takin a broader view of the code. I'm actually on an airplane.
[Citadel Development] (null)
FreeBSD's default MTA does appear to support LMTP, but there is no documentation in making it use LMTP as opposed to just writing files to disk. Sat Sep 23 2023 00:13:27 EDT from IGnatius T Foobar Ok, so it has to stay, then. Does sendmail not support LMTP?
[Citadel Development] (null)
FreeBSD's periodic scripts invoke sendmail. The sendmail subsystem then invokes some sort of sendmail-like program, by default it's sendmail. I configured the sendmail subsystem to invoke citmail. So ya, it's still needed for FreeBSD installations who want their system maintenance scripts to put stuff in Citadel. Thu Sep 21 2023 10:05:14 EDT from IGnatius T Foobar *sigh* Another day, another person who won't listen. Quick poll for fellow Citerati out there. Does anyone think we still need the `citmail` utility? As a refresher, this program is an MDA, analagous to `procmail` or `maildrop`, that files incoming mail into Citadel mailboxes after it has been received by some third-party MTA such as Postfix or Sendmail. However, Citadel has for a long time offered an LMTP socket, and I believe all modern MTAs can use LMTP to drop local mail to local users. So what I'm looking for is someone to say "You're correct about that, IG; spawning a process to invoke the MDA is so 20 years ago and nobody does it that way anymore" so that I can discontinue `citmail` entirely. (The number of sites using Citadel Server with a third party MTA is vanishingly small to begin with -- but not zero.)
[Citadel Development] Re: Write server PID to citadel.lock in addition to locking it.
PID files are usually in /var/run and are suffixed with "pid" as seen in the attached screenshot of my /var/run Mon Sep 18 2023 10:21:10 EDT from rss <> Subject: Write server PID to citadel.lock in addition to locking it. This will be useful for systems like FreeBSD that need to know a PID to monitor.
[Citadel Development] Re: Removed background and restart from citserver.
I am not. What is the secret incantation that makes it build on FreeBSD? Fri Sep 08 2023 22:41:47 EDT from IGnatius T Foobar Subject: Re: Removed background and restart from citserver. Kitty -- I hope you're having good results on FreeBSD -- please let me know if it works for you.
[Citadel Development] Re: Removed background and restart from citserver.
The rc system fully detaches it from the entire call stack, in the end it just looks like init ran the daemon. The rc system is really just a series of shell scripts, nothing too fancy. That's why PID files are so important, the rc system uses those to figure out which PID to send signals to. Fri Sep 01 2023 16:40:26 EDT from IGnatius T Foobar Subject: Re: Removed background and restart from citserver. I have to go examine other BSD rc scripts, but I'm guessing they simply do "&" to put a process in the background, and then they keep track of the pid so they can SIGTERM it later.
[Citadel Development] Re: Removed background and restart from citserver.
FreeBSD's in-base (default) compiler is clang. Moving all of base from gcc to clang was a monumental effort. Fri Sep 01 2023 23:30:53 EDT from IGnatius T Foobar Subject: Re: Removed background and restart from citserver. All righty then. The current code in git master is running on FreeBSD. Here's the deal: 1. You need ldap client library (`pkg install openldap26-client`) even if you aren't using LDAP 2. `gmake` and `gcc` are required. It won't build with whatever FreeBSD is using natively. 3. The configure script now detects whether it needs `-lresolv` and/or `-lintl` automatically 4. We now omit the `chkpw` and `chkpwd` programs on systems where they won't work. I'd actually like to remove system auth completely but I don't know if anyone is using it. On my FreeBSD system I was able to get the server running and I ran the new `loadtest` utility with a few dozen threads beating the crap out of it. I abruptly killed the server with SIGKILL and SIGQUIT numerous times, during periods of high write activity, and even under the worst conditions it did not corrupt the database. Looking really good here.
[Citadel Development] Re: Removed background and restart from citserver.
So FreeBSD's rc system sends a SIGTERM when you do "doas service citserver stop". This should run the same code that cleanly shuts down citserver when you do a ".ATN y" from the text client. All the BSDs opurrate this way, and since it is a standard way of doing things, SMF (Solaris/OpenSolaris/Illumos) also supports this method of daemon shutdown. Probably also a good idea to handle SIGQUIT and SIGKILL this way (for the odd platform that says these aren't untrappable). If FreeBSD's rc system can't get a process to exit cleanly after a short period of time, it will SIGKILL, and that's most likely lead to database corruption on my installation. Thu Aug 31 2023 23:39:48 EDT from IGnatius T Foobar Subject: Re: Removed background and restart from citserver. citserver should exit cleanly when it receives a SIGINT, SIGQUIT, or SIGKILL. This may be why using FreeBSD's rc system caused badness on my system. And it so happens that FreeBSD is the reason I pulled that section of code out. This is an opportunity to remove all of the "clever" stuff and really focus on portable, standard code. I do feel that there's no longer any reason why a server program should have to do the work to put itself into the background. The operating system's supervisor program should be doing that. I'm sure I'll be learning more about how FreeBSD's rc system works. Seems like it's held pretty closely to Unix System III. That's fine, because I've always hated the System V init scripts. I did release what we've got so far as Citadel 991, with some of the FreeBSD changes in place, but I'm going to keep poking at it until it builds cleanly without needing help. Also I have an ACloudGuru account paid for by my employer and that lets me stand up cloud sandboxes for testing things. I'm going to start using that for an occasional build test, I think. The only problem is that the sandboxes automatically self-destruct after four hours, so it's not useful for much more than testing. Right now I've got the latest Easy Install build running on a Scamazon instance, with a 30-thread load test pounding away at it. I figure it's better to abuse their SSDs than mine.
[Citadel Development] Re: Removed background and restart from citserver.
citserver should exit cleanly when it receives a SIGINT, SIGQUIT, or SIGKILL. This may be why using FreeBSD's rc system caused badness on my system. Thu Aug 31 2023 11:10:32 EDT from rss <> Subject: Removed background and restart from citserver. These functions should be managed by an external supervisor program.
[Citadel Development] I'm in ur VM, running FreeBSD
Latest code isn't building on FreeBSD. I has no idea how to fix this one. So, here's a quick guide to set up a FreeBSD VM.
I'm going to assume you already have VM softwares and know how to use it.
Config:
CPU: 1 or 2
Memory: 8GB minimum for ZFS, if you do a UFS2 install you can get away with 2GB.
Disk: 50GB is moar than sufficient, use a SATA drive
Network: Just make sure it has webternet access, depending on how you do things it should also be able to access stuffs on your local network (like your VM host). vtnet works, so use that as the network device
UEFI firmware: enable this if your VM software supports it
CD-ROM/DVD-ROM drive: use a SATA drive, this is where you'll attach the install image.
First, download the install image. You'll want the memstick image for your hardware. For most of you this will be FreeBSD-13.2-RELEASE-amd64-mini-memstick.img.xz
https://download.freebsd.org/ftp/releases/ISO-IMAGES/13.2/
Unpack the image and attach it as an optical media drive to your VM and boot.
Install should be self-explanatory. This is not linux so there's no GUI installer. Use auto-defaults for disk.
When creating your user account, you can do what I do and set the home folder inside /usr/home, this is where it's gonna go anyway due to a compatibility symlink.
Booting into your freshly installed system:
Log in as root. I'm not even joking.
edit /etc/rc.conf and make sure the following lines are indeed present:
sshd_enable="YES"
If you had to add this, save the file and then: service sshd start
Now for your first installs: pkg install doas vim git git-extras gitty gitui
edit /usr/local/etc/doas.conf and make sure it has the following:
permit :wheel
permit nopass keepenv as root
If you wanna use ZSH, do this: pkg install zsh zsh-{antigen,autosuggestions,completions,navigation-tools,syntax-highlighting} git-prompt.zsh
Use the pw command to make ZSH the default shell for you user account (only if you wanna use zsh):
pw usermod -n -s zsh
Now you can logout, and then like a psychopath log in again, this time using your normal user account. TEST TO MAKE SURE DOAS WORKS. Should work with the above config, but just to test: doas cat /var/log/auth.log
Configure everything else as you would normally, keeping in mind everything will be installed under /usr/local instead of wherever the fuck linux installs stuff. FreeBSD likes to keep softwares separate from base.
Now install Citadel's deps:
doas pkg install gmake gcc auto{conf,make} binutils, gsed, {lib,intl}tool openssl curl expat lib{ical,iconv,nghttp2,ssh2} icu gettext gettext-runtime openldap24-client readline shared-mime-info pkgconf
Whew, that was a long one. It's okay if you decided to SSH into the VM to copy-paste that one. Congrats, you now have a development VM to play with.
[Citadel Development] (null)
The email verification thingy doesn't seem to be workering. Fri Aug 11 2023 08:26:21 EDT from IGnatius T Foobar I already did the account creation thingy. Ok, I approved the account. I guess I need to set up more alerts on that sort of thing.
[Citadel Development] (null)
CMake does do a bit of automagic resolution of certain resources. It's a common tie-in with many ports/pkg systems, so in 3 ways CMake makes building official packages on OS project build servers much easier. I also limit myself to only using CMake 3.3, which is the last version made for IRIX. Unless you're using a pretty extensive build system, building for IRIX is non-trivial. CMake makes it trivial by automagically bringing in all the various library and include locations.
[Citadel Development] (null)
With CMake, all the dependencies are listed right there in CMakeLists.txt, so someone creating a port for their OS of choice can damn near copy-paste the dependencies into the file that describes the port in whatever ports/pkg system is in use. I've never seen CMake be an issue for compatibility across multiple systems, and since it's a ready-made package for basically every OS, build time for CMake should be a non-issue. Even those who build everything from ports wouldn't have much issue because CMake is already used for much of the other infrastructure apps already in use.
[Citadel Development] (null)
I already did the account creation thingy. Thu Aug 10 2023 18:46:53 EDT from IGnatius T Foobar Absolutely! Just go to https://code.citadel.org and create an account. You will need to use your @uncensored.citadel.org address, at least initially. I'll approve it in short order.
[Citadel Development] (null)
Can I pwease be granted access to the lab of git? I promise not to hock up too many hairballs.
[Citadel Development] (null)
It is nice and clean. Additionally, it makes creating a FreeBSD port dead-simple: https://github.com/freebsd/freebsd-ports/blob/main/security/diffcode/Makefile The line that causes it to be built and packaged purrfectly for insertion into the FreeBSD Ports repos is line 12, "USES = cmake". No other magic is needed. A lot of Makefile-based projects have moar complercated ports because the locations of the output files varies widely. Thu Aug 10 2023 15:19:38 EDT from IGnatius T Foobar If it were up to me, the entire build system would be using CMake. That does look nice and clean. My objective wasn't to stop using Make, but to stop using the GNU Autotools stuff. Those tools are built to handle all sorts of obscure edge cases that don't exist anymore. There just isn't that much variation in our target platforms anymore, so a few simple tests are all we need. As long as it builds on BSD and Linux, I'm fine with it; we no longer have to check for oddball systems with oddball libraries because no one is running Citadel on those systems anymore (and we don't have enough developer time to support them anyway). I probably reinvented a wheel somewhere ... but it was a very basic, very simple wheel. All of the usual criticisms of Richard Marx Stallman are included herein by reference.
[Citadel Development] (null)
If it were up to me, the entire build system would be using CMake. That's what I use in all my projects: https://github.com/LadySerenaKitty/klaculate https://gitlab.com/LadySerenaKitty/diffcode https://gitlab.com/LadySerenaKitty/hexcode https://gitlab.com/LadySerenaKitty/rotate
[Citadel Development] (null)
I'm a fan of SQL stuffs, like MariaDB. It's all I use.
[Citadel Development] Re: Master: citserver coredump on Shutdown
MariaDB and/or MySQL Sun Jul 09 2023 09:46:26 EDT from IGnatius T Foobar Subject: Re: Master: citserver coredump on Shutdown Grrr. Unrecoverable error on my database this morning. I think maybe it is time to put Berkeley DB out to pasture after all. Let's think about what we want the next to look like.
[Citadel Development] Re: Daily commit digest for Citadel
wtf, why do you want to break the universe? Mon Mar 13 2023 06:27:26 EDT from noreply Subject: Daily commit digest for Citadel cff70a055 Art Cancro 2023-03-12 Implemented an algorithm that can correctly divide by zero.
[Citadel Development] (null)
AHAHAHAHAHAHA! Wed Nov 23 2022 16:44:22 EST from IGnatius T Foobar I just removed the "check for new mail in the inbox" function out of the server's GOTO command and moved it into a new command called BIFF. I wonder how many people will know why I named it that?
[Citadel Development] Re: Notes on new build system
Sat Jun 11 2022 13:12:07 EDT from IGnatius T Foobar Subject: Re: Notes on new build system I'm going to need some help with QA on FreeBSD and I'm hoping that LadySerenaKitty is willing to continue offering a bit of time in that area. The running theory is that I am going to continue halping, but that's just a theory, A CODE THEORY!
