[Citadel Development] (no subject)
GGGHHH "ctdlmigrate" has probably been broken for several years, another victim of untested "optimization". Every time I find these "buffered I/O" routines, something was broken by it. In this case, a migration of anything other than the smallest test system would make the importer get stuck. I believe I have it fixed and people will be able to migrate their systems across different hardware platforms again. (Including me -- this system is sooo overdue for a migration to 64-bit.)
[Citadel Development] (no subject)
I'm working with someone via e-mail. They're seeing an issue. It seems like responses get addressed "to:" the original composer and also to the account responding to the mail. This may be related to using external clients to compose/read/respond to messages. I'm having the user run some tests to see if we can recreate. I've got his version numbers, hardware, distro, and all of that gathered.
[Citadel Development] (no subject)
I don't know much about such things. :) I thought maybe there was some backdoor way. Mon Jan 18 2021 23:37:05 EST from IGnatius T Foobar Unfortunately a Mac port has been elusive, but we're gradually working it out. We have at least one person who has been interested in following the progress on that. It should be clear, however, that AppImage doesn't run on Mac ... only Linux.
[Citadel Development] (no subject)
Unfortunately a Mac port has been elusive, but we're gradually working it out. We have at least one person who has been interested in following the progress on that. It should be clear, however, that AppImage doesn't run on Mac ... only Linux.
[Citadel Development] (no subject)
Is it compiled for Mac? I can test there too. There is a Linux part on MiSTer FPGA. :) If you want *weird* - the Intel core on MiSTer FPGA could run Linux - but it is a 486sx at 100mhz that runs more like a 386. :) Mon Jan 18 2021 14:53:47 EST from IGnatius T Foobar We probably need some extra diagnostics and stuff. Again, however, right now I'm looking for people to just test that it works on various systems ... the weirder the better.
[Citadel Development] (no subject)
The final package will probably have syntax something like this... citadel.AppImage run [-h dir] [-p port] [-s port]-h : Operate with data directory dir-p : Run the HTTP server on port port-s : Run the HTTPS server on port port citadel.AppImage stop(stops a running instance of the Citadel system) citadel.AppImage install [-h dir](installs systemd unit files for automatic startup/shutdown) citadel.AppImage configure We probably need some extra diagnostics and stuff. Again, however, right now I'm looking for people to just test that it works on various systems ... the weirder the better.
[Citadel Development] (no subject)
Oh, if it works - we're going to see a lot more people using Citadel - because it was dead-easy - I was just overthinking it. I think it will reduce the questions and issues, too. It is pretty spiffy. I spent today moving my Cit to the default HTTPS port. I'm still getting some errors, but everything seems to be settling and running faster than on the non-standard fault. But I haven't had a lot of time to play with the AppImage today. Who knows, my calendar may be opening up, this week, though. ;) Sun Jan 17 2021 17:53:34 EST from IGnatius T Foobar Of course. No hurry. While the lot of you are testing for compatibility, I will be writing all of the installation and maintenance scripts around it. My hope is that this will work so well that it becomes the primary way people install and run Citadel. "Even Easier Install" or something like that.
[Citadel Development] (no subject)
Of course. No hurry. While the lot of you are testing for compatibility, I will be writing all of the installation and maintenance scripts around it. My hope is that this will work so well that it becomes the primary way people install and run Citadel. "Even Easier Install" or something like that.
[Citadel Development] (no subject)
Ok. I wasn't sure. In that case - I did not notice any issues. I'll run it again and put it through some more complex testing - creating accounts, rooms - inserting images and doing attachments. I've got an ARM copy of my real DB around here somewhere, so I'll take a wack at getting it to install with my production DB in place too. That may take some time. Sun Jan 17 2021 00:53:31 EST from IGnatius T Foobar What you experienced is exactly what it is supposed to do at this stage, no more, no less. There is not yet an option to "install" it, nor is there an option to go with anything other than the default configuration and location. When you run the appimage, the entire system starts up; when you ctrl-c out of it, the entire system shuts down. We will build all of that dressing before it's released to the general public. Right now I just want to get some people testing it, making sure it's compatible with their systems and working as expected.
[Citadel Development] (no subject)
What you experienced is exactly what it is supposed to do at this stage, no more, no less. There is not yet an option to "install" it, nor is there an option to go with anything other than the default configuration and location. When you run the appimage, the entire system starts up; when you ctrl-c out of it, the entire system shuts down. We will build all of that dressing before it's released to the general public. Right now I just want to get some people testing it, making sure it's compatible with their systems and working as expected.
[Citadel Development] (no subject)
So, to launch Citadel, I need to execute sudo ./Citadel-armhf.AppImage every time I reboot or shut it down with a CTRL-C. Is that by design, or did something go wrong with the install?
[Citadel Development] (no subject)
That worked like a breeze. When I ran the AppImage, once it was done, it didn't go back to a prompt. It wasn't clear that it had finished installing and executing. As soon as I connected to 127.0.0.1, it came right up in Chromium, and I saw that an update from citserver in the terminal window. I never got any configuration input. It just ran the script and brought the Citadel up... so, no option to select ports or create the initial admin. I'm not sure if I'm describing it clearly. It wasn't apparent to me that the script had finished and it was returning logging to me in the terminal session. If I hit ctrl-C in the terminal window, it reloads, too... Another Ctrl-C and it seemed like it was shutting down, and now when I try to open the browser, the service can't be reached. Is that expected behavior now? Do you run AppImage in a terminal every time to bring it up? Going to try to reboot the Pi right now to see if it autolaunches after install. Sat Jan 16 2021 00:06:44 EST from IGnatius T Foobar All righty then! It's time for round 2 of testing the Citadel appimage. 64-bit x86: https://easyinstall.citadel.org/Citadel-x86_64.AppImage 32-bit ARM: https://easyinstall.citadel.org/Citadel-armhf.AppImage (A note on the ARM build: the Raspberry Pi OS is 32-bit, even on 64-bit hardware. If anyone is running 64-bit Ubuntu, go ahead and try it, let me know how it works.) Once again, the installation process is as follows: 1. Download the x86-64 or ARM image 2. mkdir /usr/local/citadel (or make a copy of your existing database, if it's on the same CPU type as your test machine) 3. Make the appimage executable, and run it. WebCit ought to be working properly this time, HTTP on port 80 and HTTPS on port 443. All of the other services will come up on their default ports., and as always, you can change or disable them at runtime. Once we go through a few rounds of testing, I will be adding all the typical bells and whistles, such as setup and diagnostic tools, the ability to run with any data directory, etc. Looking forward to hearing your test results.
[Citadel Development] (no subject)
All righty then! It's time for round 2 of testing the Citadel appimage. 64-bit x86: https://easyinstall.citadel.org/Citadel-x86_64.AppImage 32-bit ARM: https://easyinstall.citadel.org/Citadel-armhf.AppImage (A note on the ARM build: the Raspberry Pi OS is 32-bit, even on 64-bit hardware. If anyone is running 64-bit Ubuntu, go ahead and try it, let me know how it works.) Once again, the installation process is as follows: 1. Download the x86-64 or ARM image 2. mkdir /usr/local/citadel (or make a copy of your existing database, if it's on the same CPU type as your test machine) 3. Make the appimage executable, and run it. WebCit ought to be working properly this time, HTTP on port 80 and HTTPS on port 443. All of the other services will come up on their default ports., and as always, you can change or disable them at runtime. Once we go through a few rounds of testing, I will be adding all the typical bells and whistles, such as setup and diagnostic tools, the ability to run with any data directory, etc. Looking forward to hearing your test results.
[Citadel Development] (no subject)
I'm excited to have something to do with the Pi400. It will be ironic if the Pi 400 turns out to be as suitable as the Intel i5 for hosting my Citadel. They're cheap. Thu Jan 14 2021 19:05:55 EST from IGnatius T Foobar Yeah. Hang tight, I've already figured out the webcit problem in the current build, and I just need to make sure I didn't break Easy Install or manual builds with it. I probably broke the Debian build, but as explained in a previous post I can't commit to supporting that style of installation anymore. A new AppImage build will be forthcoming in the next couple of days, and this time an ARM version will be included.
[Citadel Development] (no subject)
Yeah. Hang tight, I've already figured out the webcit problem in the current build, and I just need to make sure I didn't break Easy Install or manual builds with it. I probably broke the Debian build, but as explained in a previous post I can't commit to supporting that style of installation anymore. A new AppImage build will be forthcoming in the next couple of days, and this time an ARM version will be included.
[Citadel Development] (no subject)
Ah, the ARM version isn't available yet. Ok. Well, I'll try to get to testing the Intel version this weekend. Thu Jan 14 2021 17:56:36 EST from ParanoidDelusions Oh. I got a 128gb SD for the Pi400... so I'll try installing there too
[Citadel Development] (no subject)
Oh. I got a 128gb SD for the Pi400... so I'll try installing there too - as Pi users are really the sweet spot for the Appimage install. Thu Jan 14 2021 17:46:50 EST from ParanoidDelusions I've got another 240gb SSD...
[Citadel Development] (no subject)
I've got another 240gb SSD, so I'm going to image the production this weekend, and will be able to do testing on the AppImage next week. I may have quit today... so I might have a lot of time, and little money. :)
[Citadel Development] (no subject)
So, I feel like I’m the model of badly reported issues - but my gift is a dogged determination to force my way through - and in the process I tend to reveal how the documentation could be clearer - and I’m actually pretty good at clarifying the documentation in language that average people understand (e.g., people who don’t code or architect networks for a living). You might find me useful, but the signal to noise ratio will be pretty chatty. The text-client install is an example. You’ve got to really understand permissions, file locations, and the relationship of How you installwhere you installWhere accounts launch programs from. And with Linux, this isn’t as cut and dry as with Windows. There are a hundred different ways it could happen, and as many different locations. I’m still wrapping my mind around this - but my basic problem was not understanding that citadel.rc is critical to the Citadel text client launching. It has to be in a path the citadel binary can find, and the account that citadel launches under has to have permissions to it. It *says* this in the text document, and I read it several times - but it was understood the way a kid from the Peanuts understands a teacher talking. I’m pretty sure ALL of my issues were Operator Error and not RTFM... But part of it was RTFM but not comprehending what was said. I had the ability to comprehend it - because when the lightbulb lit up, it instantly fixed the problem - it was just that the lightbulb took a long time to warm up and glow. Which isn’t necessarily a documentation problem. It is an *audience* problem. The documentation is written for people who *get it* - but documentation is most successful when people can just follow the steps and succeed, even if they have no idea why. At least, that is my approach, as a Windows career professional. Sat Dec 05 2020 13:45:14 EST from IGnatius T Foobar @ Uncensored I've created a private wiki called "Citadel Issue Tracker". It is a hidden room. In the past we've had a terrible record with bug trackers because they tend to fill up with badly reported issues, badly tested issues, and feature requests. So I'm making the audience limited for this one because I only want it to carry two types of issues: 1. Issues which have been confirmed and we intend to fix 2. Reported issues which look plausible enough to look into (and if confirmed, fix)
[Citadel Development] (no subject)
I've created a private wiki called "Citadel Issue Tracker". It is a hidden room. In the past we've had a terrible record with bug trackers because they tend to fill up with badly reported issues, badly tested issues, and feature requests. So I'm making the audience limited for this one because I only want it to carry two types of issues: 1. Issues which have been confirmed and we intend to fix 2. Reported issues which look plausible enough to look into (and if confirmed, fix)
[Citadel Development] (no subject)
Regarding the proper RFC 2047 encoding of Subject and other lines in RSS
feeds...
I saw the "fixed_01" version of serv_rssclient.c and I don't understand why
it makes calls to html_to_ascii. HTML is not allowed in subject fields,
is it?
I fixed the problem (or at least I think I did) with just the following code:
else if (!strcasecmp(el, "title")) {// item subject (rss
and atom)
if ((r->msg != NULL) && (CM_IsEmpty(r->msg, eMsgSubject))) {
//CM_SetField(r->msg, eMsgSubject, ChrPtr(r->CData), StrLength(r->CData));
StrBuf *encoded_field = NewStrBuf();
StrBufRFC2047encode(&encoded_field, r->CData);
CM_SetAsFieldSB(r->msg, eMsgSubject, &encoded_field);
}
}
(Excuse the formatting, please). So basically that one line that is commented
out is replaced with the following three lines, which RFC2047-encode the field
before saving it. Leading
and trailing whitespace are now stripped elsewhere, so it doesn't have to
be done here.
I'm committing this change so please let me know if I'm missing something.
[Citadel Development] (no subject)
I know you're going to hate this suggestion, that it is difficult or impossible to implement, and it makes Citadel *closer* to what you're trying to get away from - but I think if it could happen - it would generate more traffic and encourage more repeat traffic. First - the ability to "like" posts. I know... I know... but what happens with likes is it encourages lurkers to interact. They don't have to say anything, but they can go, "I agree with this post," or "I like what this post is saying," without going through making a whole post to say, "Yeah... what he said..." Second, it encourages content producers to create more content. Often, if I get several likes on a post, I'll expand on my original post, and that creates a positive feedback loop where I post more. Often, that will get people who agree to speak up and elaborate on their read of my post, stimulating conversation. Or, people will see the post getting likes, disagree, and feel like they have to speak up their opposition to what I am saying. Either way - it encourages conversation. The other would be a mobile app, or at least mobile sharing. The ability to take a photo, or something else saved on your mobile device, add a little note, and share it - without first sending it to a PC... this encourages repeat usage - it creates discussion. I know we can easily do most of this now and that the way it is creates a natural barrier to entry that actually keeps the userbase higher quality and more technical. But maybe if something like this could be switched on or off, depending on what kind of userbase the sysop wants? Just throwing ideas out there. I have no actually coding ability outside Access VBA - so it is all just a big pipe dream for me.
[Citadel Development] (no subject)
At this time the revisions to the wire protocol, configuration storage, and user interface have been completed for the transition from Sieve to a rules-based filter. Now I just have to write the filter itself :)
[Citadel Development] (no subject)
FINALLY. After a lot of work and even more procrastination, the new http://www.citadel.org site has been completed and published. This is kind of a big deal because I've been trying to have the self-discipline to finish the site before I go back to writing any more code. Documentation is important and this is the first step to having an updated set of documentation. It's far from being as complete as it ought to be, but it's enough to get to the point where I can write code again.
[Citadel Development] (no subject)
After some consideration I think I agree. Hopefully the number of people using custom Sieve scripts is very small. When someone uses the rules-based filter available in WebCit, here's what's actually happening: 1. The rules are converted to a Sieve script 2. The rules are then encoded into comments attached to the Sieve script 3. When someone edits their rules again, it reads the encoded rules in the comments, not the script. This saved us from having to write a reverse-parser. This means that during an upgrade we can just throw away the generated scripts, turn the comments into the "real" rules, and now the majority of the users who didn't even know about Sieve will still have their rules intact and working. Thoughts?
[Citadel Development] (no subject)
In my external perspective of looking forward, I would vote #2. Give the most control with the least reliance on legacy code.
[Citadel Development] (no subject)
I am having a first look at the conversation in Citadel Support (will look closer at it later) and noticing that libSieve is involved in someone's build troubles. libSieve [https://github.com/sodabrew/libsieve] hasn't been updated in 8 years. It is a small library. I am starting to think that maybe we should do one of several things: 1. Fold it into the Citadel Server source, eliminating the build dependency 2. Switch from Sieve to a rules-based filter I wonder if any sites are actually *using* Sieve instead of just the webcit-generated rules.
[Citadel Development] (no subject)
Heh. I miss Eudora. It worked. I actually did remove that mention in the new web site, which will be launched very soon. You can preview it at http://wwwdev.citadel.org The new web site is what I've been spending my time on lately. Even though I've got BIG plans for WebCit-NG and Citadel-on-Docker and apparently Citadel-on-MacOS, I've been trying to maintain the self-discipline to finish the new web site with its improved look and updated documentation before I go back to coding. It's miserable! I want to write code!! :) After the new web site goes up, ping me again and we'll resume the MacOS effort. In the meantime, please make yourself at home as a regular on Uncensored ... things are slow right now and we need a few more friendly voices.
[Citadel Development] (no subject)
>I'm going through the Citadel Server documentation and it mentions that >the IMAP client works with Eudora. Then it should probably also say that it is compatible with Pine and Elm. :)
[Citadel Development] (no subject)
Bump on the "Citadel on macOS" initaitive, now that Apple has deactivated all functional groupware and web serving from their Server.app product. I've still got my same macOS environment online, I just need to know if/when you will need access to it again so I can re-open firewall ports and cretae accounts and such. At least Easy Installer is smart enough to now error out and tell me that the OS is not supported, so that's something for the plus column. :)
[Citadel Development] (no subject)
I'm going through the Citadel Server documentation and it mentions that the IMAP client works with Eudora. LOL
[Citadel Development] (no subject)
>Though just one question, is a strong knowledge of Citadel required >for the task? >I must admit I have not played/experimented with Citadel for the last >months... No, but you might need to have a working Citadel system that you're willing to play around with, to verify that what you see in the documentation is still correct. I'm in the process of cleaning up the FAQ (Knowledge Base) section. I'm dreading the Documentation section, so some help would be awesome.
[Citadel Development] (no subject)
Sun Nov 24 2019 23:10:08 EST from IGnatius T Foobar @ Uncensored Is anyone interested in helping to clean up our documentation for the new-and-improved Citadel website? We have 145 documents from the FAQ / Knowledge Base / Documentation portion of the old site. I have bulk-converted them as best as I could from wiki markup to "real" HTML, which you can see here: [ http://wwwdev.citadel.org/documentation.html ] I could really use some help from someone who would be willing to: * Read each document and point out the ones that are obsolete and need to be deleted * ...or updated * Clean up any remaining wiki markup and make it look like (very simple) HTML Any takers? I would really like to bring the new site online. >*Raises hand* Though just one question, is a strong knowledge of Citadel required for the task?I must admit I have not played/experimented with Citadel for the last months...
[Citadel Development] (no subject)
Is anyone interested in helping to clean up our documentation for the new-and-improved Citadel website? We have 145 documents from the FAQ / Knowledge Base / Documentation portion of the old site. I have bulk-converted them as best as I could from wiki markup to "real" HTML, which you can see here: [ http://wwwdev.citadel.org/documentation.html ] I could really use some help from someone who would be willing to: * Read each document and point out the ones that are obsolete and need to be deleted * ...or updated * Clean up any remaining wiki markup and make it look like (very simple) HTML Any takers? I would really like to bring the new site online.
[Citadel Development] (no subject)
Actually yes, that's exactly what it does. Easy Install was designed to be run over and over again, updating the software and keeping your data intact. I've been running Uncensored using the Easy Install distribution for years and I've never had a problem. When people do have problems, it's usually because they introduced some sort of incompatibility with their underlying operating system, or somehow they got two Citadel servers running at the same time and it corrupted the db (but the software is supposed to prevent that from happening, so I don't know how they actually pulled that off). So yes, you can just keep running Easy Install over and over again to get the latest version. But if you do manage to break it, I always advise reading this FAQ article: [ http://citadel.org/doku.php?id=faq:troubleshooting:no_backups ]
[Citadel Development] (no subject)
Fri Sep 06 2019 12:56:11 EDT from IGnatius T Foobar @ Uncensored Don't you have a snapshot feature on your VM hosts? It's common practice to snapshot a virtual machine before performing any sort of major upgrade, for exactly that reason. If it totally goes south you just revert, and it's far easier than restoring from a backup. LOL. I'm running on bare metal on the Pi. I can image the SD that runs the machine, of course - same net effect. But even then, do I just run the latest easy-install - and it keeps the same databases, pointers, messages, users, rooms, etc.?
[Citadel Development] (no subject)
Don't you have a snapshot feature on your VM hosts? It's common practice to snapshot a virtual machine before performing any sort of major upgrade, for exactly that reason. If it totally goes south you just revert, and it's far easier than restoring from a backup.
[Citadel Development] (no subject)
I have a problem with sticking with a solid, stable production environment as development continues because I like what I have and don't want to risk losing things in the upgrade process. Often this results in a new feature I want showing up, and there being a tremendous upgrade process between where I am and where the feature exists that wouldn't be there if I had just bit the bullet when I should have. It is one of my liabilities as a systems admin/engineer... whatever the company wants to call my role supporting their servers and platforms.
[Citadel Development] (no subject)
An airplane of a room, to be sure. Welcome. As you can see, the current Shiny Thing is getting Citadel to run inside a container. That ought to make deployment easier for a lot of people. I'm getting tired of listening to support requests from people running the outdated .deb packages. There's an effort in progress to change the way we index user records. The new index will make all non-alphanumeric characters insignificant, so when you do (for example) an XMPP login, mine for example will be "ignatiustfoobar@" (plus the domain name) and it won't matter if you try "ignatius.t.foobar@" or "igna_tiust_foobar@" or "ig_na.tius_t-foo_bar@" it'll all be the same. This has always been a sticking point with XMPP, and as we begin to use these tokens with other protocols it will matter. One of those other protocols will be ActivityPub (the protocol used by Mastodon) so we can join the fediverse. Most of this stuff will require webcit-ng, because the legacy WebCit code has become unmaintainable and I'm not adding any new features to it.
[Citadel Development] (no subject)
767 new of 767 messages.
[Citadel Development] (no subject)
Whoa, great progress! I had read before that Citadel had pendant issues with Debian and derivatives, including of course Raspbian.But now fixed, and also included systemd support. Great work. Thanks.
[Citadel Development] (no subject)
What originally manifested as "Citadel no longer runs on Raspberry Pi" is turning into a pretty big deal. As was discovered in the support room, it crashes in the same place when running on the newest versions of Debian. As far as I can tell, the issue developed some time *during* the Buster train. It currently manifests on fully-updated installations of Buster, and on Sid. A bit of experimentation later, what I'm finding is that the server crashes on the very first database read. That happens to be a read of some configuration values, but the behavior doesn't change if I budge in some dummy reads of other tables before that. This is a weird one. This code hasn't changed in years. Something on the underlying platform is making it b0rk. I even downloaded the very latest Berkeley DB and linked it in as a static library, to make sure it wasn't a problem that has been discovered and fixed by Oracle. No dice.
[Citadel Development] (no subject)
A few things are going on behind the scenes. Right now, I've got Easy Install running on my Raspberry Pi and I was able to reproduce the problem where the database is hosed from the very first install (manifests as "citadel-admin.socket" missing). Now to figure out what is going on, and to fix it. This is weird because Raspbian is basically just Debian, and it works fine on Debian. I have another thing going on in my brain right now that's going to fix a minor data model issue that's been poking me for years. I've decided to make Citadel user names not only case insensitive, but punctuation insensitive. So for example: IGnatius T Foobar , ignatius.t.foobar , ig natius tfoobar , ignatiust_foobar ...are all the same user. Hopefully this will not create a mess for anyone out there when we reindex their databases. The reason I want to do this, is because we need the indexed name as a primary identity for several protocols. It's always been a mess with XMPP, for example. My identity for XMPP is "ignatius_t_foo...@uncensored.citadel.org" but right now, it converts the underscores to spaces and assumes that my screen name is "IGnatius T Foobar". That's correct, but it's an assumption, and we don't like assumptions. I'd like to be "ignatiustfoo...@uncensored.citadel.org" and it Just Works (tm). Why this, why now? Because I also want to be: http://uncensored.citadel.org/~ignatiustfoobar (home page, etc.) @ignatiustfoo...@uncensored.citadel.org (ActivityPub) ActivityPub seems to be winning as the federation protocol of choice (it's what Mastodon and Gab use, for example) so it seems sensible to prepare for it.
[Citadel Development] (no subject)
Makes sense. :) Now to find a Linux variant that doesn't suck... I never should have sold off my SunFIRE server. Something tells me that with the addition of a 10tb RAID array, that could have made a very practical application server... I have so many regrets...
[Citadel Development] (no subject)
Microsoft, Apple, and the Linux Foundation have all been tainted by a lot of the same problems, such as SJW's and "flat" user interfaces. They all suck now. There's no escape. But ... if you ran a Linux VM on a Mac, your data files would probably migrate to a future native build without exporting them first, since it's the same CPU architecture (64-bit x86). As I said before ... getting Citadel to run on both FreeBSD and OpenBSD cleanly will be a good step towards getting it running on a Mac. There's probably not much to it, but I'm not clear on any of their build systems, and that's probably the biggest obstacle.
[Citadel Development] (no subject)
But...but... Windows is now at one with the open source community, so what is to stop someone like me from feeling like it's the only viable choice when Apple's "Just works" model is so obviously a lie? Hmmm, maybe there is a middle ground... Wonder if anyone came up with anything that is not bound by Micro$oft or Apple? ;) macOS as a VM host for the Citadel appliance seems like it has potential. See? Just a little deductive reasoning and suddenly options appear out of nowhere! :) Ok, soapbox demolished.
[Citadel Development] (no subject)
Easy there, chief. I said it's ironic, not that it's the preferred way to run ... well, pretty much anything, actually.
[Citadel Development] (no subject)
> The biggest irony is that it's running fine on Windows. Well crap, why didn't I think of that? Time to sell all my Apple hardware, get a $500 server and spend 50 times that much on licenses for things that I already have! Paying for software I get get free or bundled without license seems standard to Micro$oft, after all! End rant...
[Citadel Development] (no subject)
And now for the answers to some questions fleeb asked in email. * Yes, ctdlsh uses libreadline. It also ... doesn't do much yet. Someday it will replace the text client as a place to do system configuration from the command line. But it doesn't yet do enough useful things for it to assume that role (or any role), which is why it hasn't been packaged. * Yes, it's possible to build a WebCit package that ties into your Apache or nginx installation. I'd avoid doing it though, because there are s many variables. Remember, there are an infinite number of monkeys using their infinite number of typewriters to generate support inquiries. The original version of WebCit used a cgi-bin to bind incoming HTTP connections to WebCit sessions, and the *vast* majority of people having installation problems were reporting issues in this area. When we gave WebCit its own built-in HTTP server, those support inquiries just stopped completely. (The snarky version of this is that every time we make the installation easier, we open up another stratum of users underneath the previous one, who have even *less* of a clue how to install software, so the goal is to make it *infinitely* easy.) If you want to experiment though, maybe build another package like "citadel-webcit-nginx-integration" or something like that, which finds your WebCit and Nginx installations and tries to tie them together. But maybe you want to wait until webcit-ng is in production, since that one *strictly* uses the /ctdl prefix for all transactions.
[Citadel Development] (no subject)
Every revision gets us closer to OSX compatibility, so there's a good chance we'll get there around version 1000. As we continue cleaning up old code and removing obsolete features, the build gets cleaner and cleaner on things like FreeBSD and OpenBSD. My eventual goal is to streamline it so much that we can remove GNU Autotools entirely because we're sticking to standards and not relying on platform-specific hacks. The biggest irony is that it's running fine on Windows.
[Citadel Development] (no subject)
Any chance that the OSX port can be 1000? :)
[Citadel Development] (no subject)
By the way ... in case I haven't mentioned this ... you can go to http://code.citadel.org to look at the git repository, read the commit history, browse the code, etc.
[Citadel Development] (no subject)
>On a RPM-based system, uninstalling would also need to initiate a few >commands to disable/remove the startup scripts: > > # service citadel stop > # service citadel remove True, but the next time I update the setup program, I'm probably going to switch it to use systemd instead of sysvinit scripts anyway.
[Citadel Development] (no subject)
So I just read this:https://www.fastmail.com/help/technical/ssltlsstarttls.html Quite helped me to understand the differences, but one of the parts that caught my attention was this:"At some point, it was decided that having 2 ports for every protocol was wasteful, and instead you should have 1 port that starts off as plaintext, but the client can upgrade the connection to an SSL/TLS encrypted one. This is what STARTTLS was created to do." In my particular case, I also think it'd be better to go back having only one port for each protocol. But, from what I could overall understand, they still cannot reach a "global" agreement, old software is too conservative, etc, etc, which has the effect of keeping more than one port for each protocol for good... And I'd like to ask, what do you think in general?In the case you think as well only one port would be enough, which case would you prefer to stay for good? The more recent TLS implicit ports, or the old ports just with STARTTLS? Thanks again.
[Citadel Development] (no subject)
On a RPM-based system, uninstalling would also need to initiate a few commands to disable/remove the startup scripts: # service citadel stop # service citadel remove I could be wrong on the precise options here, I do not have a system to confirm these against, but this is to the best of my recollection from many years back when I did run on Linux. FreeBSD and other OS's will have their own commands. I believe the setup binary creates these, I can't imagine it would be too difficult to add an --disable parameter to remove the startup scripts, and potentially to do the file system cleanup with an --uninstall option.
[Citadel Development] (no subject)
Hello.First time in the Development room. First, I'm not really a developer, but FWIW I do have some time for reading, and testing <-- hope this serves. Actually, even after reading IG's comments regarding XMPP and IRC protocols, I remained a bit insecure about which would be better. Also I thought, if Citadel project decided to favor one of them for good, wouldn't it be better to drop the other one for good as well? Is it worth to maintain *two* different protocol implementations? After taking a look at RFCs about IRC protocol and serv_xmpp.c, I'm still clueless about how a "mapping of IRC protocol to the existing Citadel internals" would be done. And in the way, while reviewing serv_xmpp.c, I thought "what if it was just a patching matter after all"? According tohttps://tools.ietf.org/html/rfc6120#section-7.6https://xmpp.org/rfcs/rfc3920.html#bindit would seem the function void xmpp_stream_start (starting at line 180) is totally lacking at its end the needed IQ stanza of type "result" that must be sent back to the client. While currently XMPP seems to somewhat "work" on Webcit, this would explain why it doesn't work with XMPP clients. In the meanwhile I'll try imagining how to do this "patch".. Sorry for my indecisiveness beforehand; if you say "forget about XMPP, better focus on IRC", I'll take the advise.
[Citadel Development] (no subject)
Sounds like a plan. MySQL has a similar mechanism, probably for similar functionality (though also for much more).
[Citadel Development] (no subject)
If both programs are running on the same server, and WebCit is running under the same uid as Citadel Server (or as root), it will be able to access citadel-admin.socket, which offers super-privileged access to the server. I'm inclined to use this as the conduit for certificate installation, since the security model is already in place and already effective, and for sites that have Citadel Server and WebCit running on different hosts, we will just have to say sorry, you have to do certificate management manually.
[Citadel Development] (no subject)
What if citserver listened on an auxiliary port in order to perform this kind of transaction, in a space that would not have access to any other data than what is absolutely needed? Webcit already needs to know where the citserver process is running from, so it would not mean any additional inconvenience for the admin setting it up. Since it's a one-time setup, I think security impacts of using a non-secured TCP connection for this exchange could be overlooked. A case where 'localhost' is used would also need to be looked into, since that is the third type of FQDN that could be used to connect Webcit to citserver.
[Citadel Development] (no subject)
WebCit and Citadel Server are not guaranteed to be on the same host system. And there might be multiple instances of WebCit on different host systems. Also ... the reason WebCit needs to be the agent which configures the SSL elements is because I want to do one-button SSL enrollment. If the site operator wants to use the Let's Encrypt free certificates, for example, it should say "you are accessing this site as https://ctdl.example.com , do you want to obtain a certificate for this name?" and then it generates a key and CSR, and uses ACME protocol to contact the Let's Encrypt servers, which then connect back to https://ctdl.example.com to perform domain validation, and the certificate is generated. That key and cert then need to be pushed down into Citadel Server so that it can be used for other protocols, and then pushed back up to WebCit for future startups. I suppose we could also do your idea where both programs point to the same keystore, but it breaks the data model. However, I'd be willing to bet that 99.99% of Citadel installations have both programs running on the same host system anyway, so we could just be lazy and say "sorry you have to do certificate installation manually" to the other 0.01% , who are likely to be advanced administrators, and not the type who will just run Easy Install and then whine in the Lobby in a foreign language when it doesn't work.
[Citadel Development] (no subject)
I see your point, IG. Java applications that access services over SSL use the JVM's keystore and keep everything self contained. When WebCit is installed, couldn't an external call to the citserver binary be used to generate the certs/keys for WebCit to import and serve? Ok, I'm not great when it comes to knowing about this kind of stuff, but just trying to offer some food for thought.
[Citadel Development] (no subject)
Well, there are plenty of applications where you can perform key and certificate maintenance from a web interface. There are typically buttons to generate or import keys, to generate and export certificate signing requests, and to import certificates and chains. That much is not in question. To do this with the Citadel system, the agent acting on behalf of the site would be WebCit, not Citadel Server. This breaks our data model somewhat because we would require a true keystore *in* Citadel Server, which would then have to deliver the private key to WebCit when it starts up, so that WebCit can serve up SSL sessions. The reason this breaks our data model is because WebCit is currently not trusted -- it only has the security level of the user who is logged into it. A central design tenet has always been that the client on the other end of the 504 protocol is no more trustworthy than an end user; a protocol connection has the same security level as a telnet connection. Key management from WebCit will require the addition of some sort of trust model. I suppose each instance of WebCit will have to generate its own identity the first time it starts up, and register itself with Citadel Server the first time an Administrator logs in. Then on subsequent startups it would use that registration to download the private key and the certificate. I hate this but I'm unable to think of any other way.
[Citadel Development] (no subject)
For SSL certs, especially ones that are not self-signed, the provider packages the cert(s) as files, with the assumption that if a cert is for multiple subdomains or web sites within the same domain, that it would need to exist as a file for servers such as Apache HTTPD. The most obvious benefit of this is so that backup solutions (most notably enterprise tape backups) can benefit from having a static file rather than having to continually treat the database as 'dirty' because other non-SSL certification content was modified. This also makes recovery much simpler, if necessary. (FWIW, I loathe recovery from tape, especially when the full backup set is off site and I only have access to incrementals. Grrr.)
[Citadel Development] (no subject)
I have officially Done The Needful. AdjRefCount() is now a synchronous operation. If there happens to be a refcount_adjustments.dat lying around from a previous version, it is ingested at startup and then deleted. I *may* come back later and have zero-refcount messages deleted by TDAP instead of by AdjRefCount() itself, but for now it seems to perform well enough as is. Deleting a room is a deferred operation (from the user's perspective) anyway, so that's not something they'll have to sit around and wait for. I haven't yet tried a bulk deletion in IMAP, so when that's tested it might tell me that I have to do the deferred delete sooner than later. Internal server version is now 922. That's the last of the server state that was directly in the filesystem, that has now been moved into the database. A packager can now build Citadel and only needs to resolve the paths to the following three directories: data/ files/ keys/ We can have a discussion about whether SSL keys and certificates should be in-db or remain in the filesystem. I'm not thrilled about the way we handle key and certificate management. Right now it has to be handled manually because the client (i.e. WebCit) is not trusted. Because of this, there is no way to set up SSL from the client side, even if you are an administrator. From a security perspective, this separation of trust is outstandingly good. However, I've got Let's Encrypt in my sights, and there's no way that Citadel Server will ever be able to speak ACME protocol without exposing a web service. This means WebCit has to be trusted, even when there is not an administator logged in.
[Citadel Development] (no subject)
Yeah, the server didn't answer at all. :(
[Citadel Development] (no subject)
I believe the odd characters have something to do with images contained within messages input from WebCit. I've seen several messages by multiple users (including yourself in your last post in Small Achievements> which you indicated a photo of a cylindar. Hope that helps track it down a little more. As for the macOS port, just let me know if you have any issues logging into the server, I've been slowly migration my macOS environemnt to the macOS Server add-on application, which adds a whole new set of layers to authentication and securitry.
[Citadel Development] (no subject)
I don't see the stray characters on my terminal. Maybe this is a Mac thing. I'll check it out when I log in to your system. Speaking of which, before I do that, I've been cleaning up some random bits of code and data that seem to serve no purpose in the current build. For example, the "upload_type" field in CitContext only applies to API calls that have been gone for about a year. So that's gone. I also found a 64-byte field in struct MetaData called "mimetype" which seems to be a duplicate of meta_content_type that isn't used anywhere. I don't know the intentions of whoever put it there, but since it isn't used I took that out too. My short-term intention is to remove the refcount_adjustments.dat flat file, and make reference count adjustments directly to the message metadata records. This of course makes it a Good Thing to have those records as small as possible. So why is IG getting hung up on this seemingly little thing? The answer is that refcount_adjustments.dat is the VERY LAST bit of system state that is not stored in the main database. Once it's gone, ALL data will be in the data/ directory (and also in files/ if you have upload rooms, but that's ok). Over the last year or so I've been working towards this goal, because it will allow us to put the Citadel system into a container. This neatly solves almost all of our build and packaging challenges, at least on Linux. All you have to do is install the container point it back to off-container data/ and files/ directories. When it's time to upgrade, simply delete the container, install one with the new version, and hitch up your existing data/ and files/ directories. And no I don't consider myself to be part of the cargo cult of containers. I do want to try it and see if it is a win. The way I see it, this kind of modularization is a win regardless of the final packaging format.
[Citadel Development] (no subject)
Feel free to move this to Save the Text Client> if you feel it more appropriate there. I thought a more limited audience was better for this. :) Not sure if this has been reported, or if it is a byproduct of how SSH is pasing between the daemon and local text client, but I've spotted a couple of oddities that I wondered about. 1. I see the letters 'ta' in high ASCII form while reading messages that include line feeds. 2. The .ead io feature appears broken if you enter ? for a list. The reponse is "Unrecognized or unsupported command." I know it's been a long time since I've been online, loads of personal issues that needed attention, but in case it helps explain my own interest in the betterment of the text client, I am legally blind, and while I do not need screen readers, I do need large text, and the web interface is not overly practical for me due to limited space in each frame with larger text (through no fault of webcit's own). I started on BBS's back in the 80's, and the keyboard is where I am most at home, and remains the simplest way to stay connected with everyone, and still exchange e-mail and all that jazz. Just thought I'd pass that along to see if anyone else has encountered these, and if a fix is warranted. Thanks!
[Citadel Development] (no subject)
It would have been Wednesday and Thursday. That same 70+ MB email getting pushed and indexed over and over again was slowing everything else down, exacerbated by hardware that is still showing some issues. I think we're good now, unless the hardware quits entirely. Let me know if you still see anything.
[Citadel Development] (no subject)
Hiya Ig.. I think that I was seeing the same problem that was experienced by fleeb the other day.
[Citadel Development] (no subject)
Ugh. My phone has been killing Uncensored. Over and over again it's been trying to push a 70+ MB email from one folder to another, and timing out before Citadel delivers the reply saying the operation was completed. I think the message was local on the phone. Not quite sure. I finally noticed when my Trash folder kept filling up with new copies of this message. And they were individual copies, not linked clones, so it kept trying to add it to the fulltext index over and over again. TDAP wasn't helping either. I still think I've got a hardware problem somewhere, and this just exacerbated it a bit. The indexer has always been configured to stop after 2500 messages so that it can flush its buffers, let other housekeeping tasks take place, and do a database checkpoint. We were getting biglybytes of database logs filling up the disk. As an emergency measure I changed it to 25 messages and it started keeping up. I wonder if this is a problem some of the unhappier Citadel users were having a few months ago. I think I'm going to change the indexer to flush after "x messages or y amount of time" which should give it a good balance.
[Citadel Development] (no subject)
(I did it ... and it works)
[Citadel Development] (no subject)
> Ugly, I know. This is the one downside to using libcurl for outbound >SMTP: it doesn't have the ability to fetch back the *actual* error >message sent back by the remote server; it only gives back the numeric >code (550). But it reduced thousands of lines of code, that we were >having trouble maintaining, down to a few screenfuls, and added a bunch >of features that were built into the library. While re-reading the libcurl documentation to see if there is any way around this omission in their API, I came across a particularly ugly way of capturing the remote server's error messages, and am giving it some consideration. The libcurl API has a debug mode you can turn on, which spits all of the protocol chatter out to stderr. It also has a callback hook that lets you do send the debug messages to a user-supplied function instead of stderr. We could theoretically abuse this function to capture the responses to SMTP commands, ignore all transmitted data (so we don't re-capture the entire message being sent out), and then filter for lines beginning with the numeric SMTP response (such as 550) that the library *did* return to us. Like I said ... it's ugly. But it's 100% permitted by the API, doesn't resort to using undocumented calls, and it's way better than maintaining our own protocol handler. "Believe me. I know protocol handlers. libcurl has the best protocol handlers."
[Citadel Development] (no subject)
And furthermore... I just did an awful lot of hacking and slashing to the text client. Billions, perhaps trillions, of lines of legacy and experimental code were removed. It's all back in a single directory with a single include file. Like ctdlsh and webcit-ng, it no longer uses GNU Autotools, but instead uses a script I'm calling "conf-IG-ure" for now, which works perfectly on the 99.999% of the target hosts that run modern operating systems. No, we don't need to support SGI IRIX running on a modified TI-99/4A with Cygwin for TMS9900, even if Autotools knows how to do that. It's pretty clever, actually. For most installations you just type "make" and it figures everything out. Makefile requires config.mk which is generated by configure, and it runs configure if it doesn't have config.mk. Quadrillions of lines of m4 macros are replaced by about two screenfuls of shell script. I am WAY smarter than Richard Marx Stallman.
[Citadel Development] (no subject)
Based on the kind of feedback we've been getting on the Easy Install system, here's what we're learning: * People really do read the "how to install the dependencies for your OS" section * But they're not smart enough to figure out how to fix it if it's slightly wrong I've removed the bits of Easy Install that make it send the compiler output to a log file instead of the screen. Everything goes to the screen now. Messages and prompts that are generated by Easy Install now show up in color. Yes, I'm making the assumption that everyone's got an ANSI-like terminal at this point. The compiler output will output in the default screen color. Next, I'm going to try to put a few lines of shell script in that will detect the host OS and offer to run the "apt install" or "yum install" command to install the dependencies, instead of relying on the user to put them in first. This is probably a big can of worms to open and I'm going to regret it.
[Citadel Development] (no subject)
In case anyone is interested ... I am once again working hard on WebCit-NG (which will become simply "WebCit" when it replaces the old code). Progress is slow because no shortcuts are being taken; every layer needs to be perfect. The current code supports various dialects of DAV (even enough of CalDAV to work with Lightning), HTTPS, and a client-side renderer that works with both big screens and mobile. The "forum" view (bbs view) is starting to take shape. You can log in (but it won't mark any messages as read, so feel free to move from room to room). No there isn't a demo site yet. That will come at some point. In the mean time you can download the code and run it. :)
[Citadel Development] (no subject)
Many years ago, someone on Uncensored was playing around with SCO and asked, "What the heck is 'micnet' and what is it for? As far as I can tell, all it does is duplicate the functionality of UUCP." And they were correct: the micnet code continued to exist in the descendents of Xenix long after no one was using it. Everyone had long since implemented UUCP, even within the halls of Microsoft and SCO. This is the feeling I am now getting as I remove IGnet from the Citadel system. It served its purpose, but aside from Uncensored and Dogpound, no one was using it. And it raised the question: if Citadel speaks all of the standard protocols, why do Citadel sites need a special protocol to connect to each other? The answer, quite simply, is that they don't. Giant hunks of code are now removed. An entire layer of complexity is gone. All sorts of cruft that made assumptions about dialup BBS and UUCP networking from 30 years ago has finally been removed. There is only one network now: the standard TCP/IP Internet. Mail is always SMTP, syndication is always RSS, and we might throw in more robust NNTP eventually. I'm sad to see IGnet go away, but the gigantic reduction of complexity in the code more than makes up for it.
Re: [Citadel Development] (no subject)
On Thu, Jan 11, 2018 at 10:12 AM, IGnatius T Foobar wrote: > Citadel 917, WebCit 917, libcitadel 917, textclient 917 all made it into > Debian > Testing after clean auto-builds. Nice. Indeed. Now that those made it to Testing, I'm setting up a new Debian Testing system in order to be able to review the rest of the Debian Bugs in light of the new versions. Jame
[Citadel Development] (no subject)
Citadel 917, WebCit 917, libcitadel 917, textclient 917 all made it into Debian Testing after clean auto-builds. Nice.
[Citadel Development] (no subject)
DAMMIT ... I *fixed* the problem, upgraded Uncensored *again* , and *still* got the directory wiped out. Fortunately I took an image-level backup of the system immediately before running the upgrade.
[Citadel Development] (no subject)
Thu Dec 14 2017 07:26:17 PM EST from IGnatius T Foobar @ Uncensored How did it go? Very badly. It wiped out most of my Internet email address directory. I had to restore from a backup. Oy gevalt!
[Citadel Development] (no subject)
>How did it go? Very badly. It wiped out most of my Internet email address directory. I had to restore from a backup.
[Citadel Development] (no subject)
Sat Dec 09 2017 03:48:11 PM EST from IGnatius T Foobar @ Uncensored Ok, I'm gonna take the plunge here, and upgrade both Uncensored and Easy Install to the current git-master. This will either be a disaster or totally awesome. How did it go?
[Citadel Development] (no subject)
Ok, I'm gonna take the plunge here, and upgrade both Uncensored and Easy Install to the current git-master. This will either be a disaster or totally awesome.
[Citadel Development] (no subject)
I've got the entire Citadel system running smoothly on the latest Debian. I think the changes we made were all good, but I must have missed something in the backporting. So it's in our best interest to get a new release of Citadel published as soon as possible. I think it's all working properly, but I need someone with an LDAP system (bennabiy) to give it a good solid testing. The current git master code synchronizes the entire user list from LDAP, at startup and every 5 minutes thereafter. It also has the undocumented (we'll fix that) setting that lets you configure your users' email addresses from LDAP instead of from Citadel, and this is disabled by default. It's working with OpenSSL 1.1, libical 2.0, and Berkeley DB 5.x, which are all shipping with the latest Debian. In other news, when you look at things like Docker, it seems that the rest of the world is following Citadel's lead and simply bringing along the versions of the libraries the package needs. Because of this, efforts will continue to remove any system state from anywhere other than /usr/local/citadel/data/ so we can put everything else in a container, or maybe even a "Really Easy Install" with precompiled binaries and self-contained libraries. We've *really* got to get the webcit-ng effort going again.
[Citadel Development] (no subject)
Yup. Now I am going to be switching locations in a week or two, and testing will probably have to stop at that point for a while ( I am taking a technical break ) so if we can get some things hammered out and solid, it will free you to get the new version out :) > Fri Nov 17 2017 10:57:03 AM EST from IGnatius T Foobar @ Uncensored > >That's fine, so far. Right now I'm only looking to see if it picks up >all of your users during what will become the directory sync process. > > > >
[Citadel Development] (no subject)
That's fine, so far. Right now I'm only looking to see if it picks up all of your users during what will become the directory sync process.
[Citadel Development] (no subject)
Logging CN and DN in debug output like so (names changed...) citserver[68186]: ldap: found uid=someone-mn,ou=Mn,ou=People,dc=servername,dc=net citserver[68186]: ldap: cn = Someone of Mn citserver[68186]: ldap: display name: , uid = <10073> I can log into my aide fine, with uid, but to log in to LDAP users, I would need to type the full CN, which simply will not work. but this is a step in the right direction. I need users to be able to use uid for their login name, not CN (as some of our CN's are VERY LONG) > Sun Nov 05 2017 10:34:18 PM EST from IGnatius T Foobar @ Uncensored > >Lots of work is going into this LDAP Sync thing. Once again it has >caused me to go back through a bunch of modules and solidify some pieces of >the system that were a bit dated because they mostly-worked and didn't really >call for much attention. > >bennabiy: you might want to run the current git master against a TEST COPY >of your production system, or some other test system that has an existing >database. If you watch the logs at maximum debug (citserver -x9) you should >see it scan your directory, producing screen names and uid's for everyone -- >but it won't actually sync the directory yet. You should also be able to log >in as any user without difficulty, which is an important test since it's >hitting a newly created index now. > > > >
[Citadel Development] (no subject)
Will test it. Glad we can solidify the system while at it :) > Sun Nov 05 2017 10:34:18 PM EST from IGnatius T Foobar @ Uncensored > >Lots of work is going into this LDAP Sync thing. Once again it has >caused me to go back through a bunch of modules and solidify some pieces of >the system that were a bit dated because they mostly-worked and didn't really >call for much attention. > >bennabiy: you might want to run the current git master against a TEST COPY >of your production system, or some other test system that has an existing >database. If you watch the logs at maximum debug (citserver -x9) you should >see it scan your directory, producing screen names and uid's for everyone -- >but it won't actually sync the directory yet. You should also be able to log >in as any user without difficulty, which is an important test since it's >hitting a newly created index now. > > > >
[Citadel Development] (no subject)
Lots of work is going into this LDAP Sync thing. Once again it has caused me to go back through a bunch of modules and solidify some pieces of the system that were a bit dated because they mostly-worked and didn't really call for much attention. bennabiy: you might want to run the current git master against a TEST COPY of your production system, or some other test system that has an existing database. If you watch the logs at maximum debug (citserver -x9) you should see it scan your directory, producing screen names and uid's for everyone -- but it won't actually sync the directory yet. You should also be able to log in as any user without difficulty, which is an important test since it's hitting a newly created index now.
[Citadel Development] (no subject)
> 2017-11-05 19:44 from kinetix @uncnsrd > Hey Citadmins / Devs / Packagers: > > There's a couple of funny errors in the webcit templates that produce >the "who's online" page. I noticed that when someone's idle, the alt >text shows "Idle sinces XX minutes" (with XX being replaced >appropriately). Both the templates who/section.html and >summary_section.html have stray "s"'s after their "idle since" text >piece. > > One other webcit quirk that I've noticed is that webcit isn't >overriding files from the static root folder itself, just from the >static/t folders and files. For example, if I put my own favicon.ico >in static.local, webcit doesn't ever use it. I may have been under the >mistaken impression that anything in static.local would override things >in static. > > Thanks! > >
Re: [Citadel Development] (no subject)
posixAccount contains the uidNumber which would work. I agree, Full DN would be bad idea... some UID generation would be better. > Mon Oct 30 2017 10:11:52 AM EDT from IGnatius T Foobar @ Uncensored >Subject: Re: [Citadel Development] (no subject) > > >> >> >>so either ldap:uid=blah,dc=base,dc=name or some uid which (already) exists >>anyway in the system? I just wonder if there would be more confusion if >>someone switched their LDAP basename etc on a running system if the full DN >>is used... even if the users are the same, would their mail store disappear? >> >> >> > > >It would be the DN of the user, not the Base DN of the search scope. And >yes, if someone changed their DN in this case, Citadel would consider it to >be a different user and the old account would be purged. > >However, I don't think that's the direction we're going to take. Indexing >by UID (or in the case of Active Directory, a synthetic UID derived from the >ObjectGUID) makes more sense. That's how we join the accounts today, but it >uses a sequential search. If we put something like "uid:12345" in the >extauth table, it will eliminate the sequential search without a major >refactoring of the LDAP logic, and it should also be compatible with sites >using system auth (which is apparently nobody, as far as I can tell; every >site I've ever heard of is using either native or LDAP). > > > >
Re: [Citadel Development] (no subject)
so either ldap:uid=blah,dc=base,dc=name or some uid which (already) exists anyway in the system? I just wonder if there would be more confusion if someone switched their LDAP basename etc on a running system if the full DN is used... even if the users are the same, would their mail store disappear? It would be the DN of the user, not the Base DN of the search scope. And yes, if someone changed their DN in this case, Citadel would consider it to be a different user and the old account would be purged. However, I don't think that's the direction we're going to take. Indexing by UID (or in the case of Active Directory, a synthetic UID derived from the ObjectGUID) makes more sense. That's how we join the accounts today, but it uses a sequential search. If we put something like "uid:12345" in the extauth table, it will eliminate the sequential search without a major refactoring of the LDAP logic, and it should also be compatible with sites using system auth (which is apparently nobody, as far as I can tell; every site I've ever heard of is using either native or LDAP).
Re: [Citadel Development] (no subject)
where ldap:dc=foo,dc=bar is the DN or just the basename for lookups? so either ldap:uid=blah,dc=base,dc=name or some uid which (already) exists anyway in the system? I just wonder if there would be more confusion if someone switched their LDAP basename etc on a running system if the full DN is used... even if the users are the same, would their mail store disappear? > Sat Oct 28 2017 12:54:33 AM EDT from IGnatius T Foobar @ Uncensored >Subject: Re: [Citadel Development] (no subject) > > >Ok, getting back to LDAP Sync. This opened a big can of worms but it's going >to end up a lot cleaner. > >Currently we map LDAP entries to Citadel accounts by using (or deriving) a >uid, and then passing that through the same way we would if we were using >Unix authentication. This requires a sequential search of the user table, >which is ugly. > >Rather than build yet another index, I'm goingf to to make use of the >CDB_OPENID table, which I've internally renamed to CDB_EXTAUTH. You can see >where I'm going with this. Right now, the key is an OpenID URI. But there is >no reason we can't put other types of keys in there. We can put fake URI's >like "uid:123456" or "ldap:cn=foo,dc=bar" (and I'm currently trying to decide >which makes more sense). > >Later on we can add other auth protocols like SSO ( >SAML) or OAuth or whatever and still use the same table. But for now, if >we're going to scan LDAP every five minutes and map user ID's, we can't be >doing all those sequential searches. > > > >
Re: [Citadel Development] (no subject)
Ok, getting back to LDAP Sync. This opened a big can of worms but it's going to end up a lot cleaner. Currently we map LDAP entries to Citadel accounts by using (or deriving) a uid, and then passing that through the same way we would if we were using Unix authentication. This requires a sequential search of the user table, which is ugly. Rather than build yet another index, I'm goingf to to make use of the CDB_OPENID table, which I've internally renamed to CDB_EXTAUTH. You can see where I'm going with this. Right now, the key is an OpenID URI. But there is no reason we can't put other types of keys in there. We can put fake URI's like "uid:123456" or "ldap:cn=foo,dc=bar" (and I'm currently trying to decide which makes more sense). Later on we can add other auth protocols like SSO ( SAML) or OAuth or whatever and still use the same table. But for now, if we're going to scan LDAP every five minutes and map user ID's, we can't be doing all those sequential searches.
Re: [Citadel Development] (no subject)
Grr. In addition to OpenSSL, the new Debian also brings along libical2, which has an API change which is neither forward nor backward compatible.
Re: [Citadel Development] (no subject)
ummm ... It looks like the patch submitted by a user back in August, in commit e42596a94e31ac9f5106f62d83dc56ddbd779b6a , may have already fixed the libssl issue. I'm going to investigate some more!
Re: [Citadel Development] (no subject)
NP :) I just read some of the bug reports and responses from debian side of things, and it seems that buster is going to drop support for 1.0 API, so the temporary fix is going to disappear after stretch. Hopefully the API changes are not too difficult to rework back in. Shabbat Shalom! > Fri Oct 20 2017 04:48:25 PM EDT from IGnatius T Foobar @ Uncensored >Subject: Re: [Citadel Development] (no subject) > > >>Seems the temporary solution would be to put libssl1.0-dev as a >>requirement > > >>instead of libssl-dev until the code is updated for the 1.1 api. > >Actually it seems that you've done the hardest part of the work already, by >helping to arrive at the likely root cause: that something in Citadel is >compatible with the OpenSSL 1.0 API but not the 1.1 API. > >Thank you! > > > >
Re: [Citadel Development] (no subject)
>Seems the temporary solution would be to put libssl1.0-dev as a requirement >instead of libssl-dev until the code is updated for the 1.1 api. Actually it seems that you've done the hardest part of the work already, by helping to arrive at the likely root cause: that something in Citadel is compatible with the OpenSSL 1.0 API but not the 1.1 API. Thank you!
Re: [Citadel Development] (no subject)
Seems the temporary solution would be to put libssl1.0-dev as a requirement instead of libssl-dev until the code is updated for the 1.1 api. > Thu Oct 19 2017 04:11:40 PM EDT from "Robert J. Clay" >Subject: Re: [Citadel Development] (no subject) > > On Thu, Oct 19, 2017 at 1:47 PM, IGnatius T Foobar >wrote: > >>Some are still having trouble, and evidently it compiles when they *remove* >> libssl-dev. This seems to imply that there's something b0rked in the new >> OpenSSL, and compiling Citadel without SSL support fixes it. That's a >>workaround >> but not one I'm willing to live with. >> > Which version of OpenSSL was it last written against? Because while > in jessie libssl-dev was v1.0.x, starting with stretch it's v1.1.0. > And in fact there's a transition underway for getting openssl1.0 > removed, in favor of the existing openssl1.1, and an open bug in > Debian [1] about it. > > > > -- > Robert J. Clay > rjc...@gmail.com > j...@rocasa.us > [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=851086 > > >
