I'm really pleased with how libical does the heavy lifting for me here. It
has improved greatly since the old days when we were its caretakers. Comparing
two timestamps that are in different time zones actually works. I don't know
how it works, but it works. Just to make sure it wasn't just
Well that's a fine howdy-do, I tagged the release without also mentioning why I pushed it at this time. This version has the config option to advertise STARTTLS on the public SMTP server.
Me too, which is why it took me so long to get out of that headspace. The
truth is, however, Citadel is extremely popular with the operators of small
autonomous sites, not so much the "Exchange killer" I once thought it could
become. That crap all moved to the cloud anyway.
Even now as I am
Understood, was just a passing thought since i live in the 'big world' and first thought is always scale.. Many of our apps get 100k + connections all day long.
>That is a hard one. But keeping them separate might help with
>scalability?Â
That was true for WebCit-classic which contains a server-side user interface.
And until a few years ago they could run on separate machines, so you could
have multiple WebCit instances running with a load
That is a hard one. But keeping them separate might help with scalability?
Mon Jan 08 2024 12:42:10 EST from IGnatius T Foobar Subject: Re: START_CHAT_MODE is renamed to SEND_THEN_RECV
I've been thinking lately that it might begin to make sense to put the web server (really, the web API)
I've been thinking lately that it might begin to make sense to put the web
server (really, the web API) directly into Citadel Server. Since it's now
a web API and no longer a user interface, it could be an improvement.
*shrug*
While most of us like the old interface just fine ( unless we are stuck with a phone, and dont feel like doing imap ), i know it does turn off some new comers. Might not hurt to do something like that so they know the future is not vaporware... They might stick around, or at least make plans
I'm starting to be quite happy with this. WebCit-NG is starting to take
the shape of a very competent webmail client. Very clean, too, since it doesn't
need to push a bunch of other garbage in your face the way commercial offerings
do: "Here's a feature we really want you to use and we won't
>git push from an airplane? showoff
Yup. :) Unfortunately I was en route to World Of Crap (AWS re:Invent) while
on the plane.
As a t-mobile customer I get a lot of free wifi benefits. I think they provide
the ground links or something.
lol
git push from an airplane? showoff
Mon Nov 27 2023 11:43:38 EST from rss <> Subject: serv_extensions.c: style cleanup from 30,000 feet
No, I'm not takin a broader view of the code. I'm actually on an airplane.
So here's what is on my mind regarding this. Suggestions are welcome.
`/var/run` is a good place to put the PID file, which as previously mentioned
is also the lock file. Mounting the same database from two different instances
of `citserver` would be A Bad Thing. (Even though I'm still
Whoa, what font is that ?!!
I am aware that `/var/run` is the conventional location, and may yet move
it there. Doing so would, however, prevent multiple instances of Citadel
Server from running on the same host. There were definitely people doing
that in the past, but I wonder if
PID files are usually in /var/run and are suffixed with "pid" as seen in the attached screenshot of my /var/run
Mon Sep 18 2023 10:21:10 EDT from rss <> Subject: Write server PID to citadel.lock in addition to locking it.
This will be useful for systems like FreeBSD that need to know a PID to
While i am super rusty, if we need a non-prod FBSD install to beat on, i can always prop-up a VM.
I installed it on that 'box of crap' not long ago out of desperation, and enough memories did come back so i was able to get it to install ok, and some add-ons. Tho i wouldn't be able to use it for
I am not. What is the secret incantation that makes it build on FreeBSD?
Fri Sep 08 2023 22:41:47 EDT from IGnatius T Foobar Subject: Re: Removed background and restart from citserver.
Kitty -- I hope you're having good results on FreeBSD -- please let me know if it works for you.
Geez. I can't believe how much more stable Citadel Server is with this new improved backend. it was worth spending most of the summer working on this.
Kitty -- I hope you're having good results on FreeBSD -- please let me know if it works for you.
I've got a test running right now with the
The above commit has to do with the experience a few of us had with rooms
suddenly containing all new messages.
There was no database corruption. THE DREADED AUTO-PURGER did this, and
it appears to have believed that it was supposed to do that. When we purge
`visit` records, we first
Phooey. This was a bit of a brown-paper-bug. I have to do a full release
of the software to correct it.
Dumping and loading a test system was fine, but when I tried to dump a clone
of Uncensored it showed the memory leak.
No pressure at all! Just looking out to see if you are ok and still with
us. But after a break you'll notice that there is likely some tweaking to
do. Specifically, if we are to have multiple versions of the visit record
(and I do think that's a good way to go) we'll need to tweak `ctdldump` to
Sorry. I have been sidetracked for about a month with work and life.
Going from upgrading the database to supporting multiple versions of the
Visit records required some thought and will require rework of a lot of the
implementation. I had optimized a lot assuming the database was at one
>The rc system fully detaches it from the entire call stack, in the
>end it just looks like init ran the daemon. The rc system is really
>just a series of shell scripts, nothing too fancy. That's why PID
>files are so important, the rc system uses those to figure out which
>PID to
The rc system fully detaches it from the entire call stack, in the end it just looks like init ran the daemon. The rc system is really just a series of shell scripts, nothing too fancy. That's why PID files are so important, the rc system uses those to figure out which PID to send signals to.
FreeBSD's in-base (default) compiler is clang. Moving all of base from gcc to clang was a monumental effort.
Fri Sep 01 2023 23:30:53 EDT from IGnatius T Foobar Subject: Re: Removed background and restart from citserver.
All righty then. The current code in git master is running on FreeBSD.
All righty then. The current code in git master is running on FreeBSD.
Here's the deal:
1. You need ldap client library (`pkg install openldap26-client`) even if
you aren't using LDAP
2. `gmake` and `gcc` are required. It won't build with whatever FreeBSD
is using natively.
3. The
UUU---
Drive 0 on my desktop is fux0red.
This was my main workstation and also where I had my FreeBSD vm installed.
Time to rebuild.
nd just like that, my main machine at home bit the dust. *grumble*
>So FreeBSD's rc system sends a SIGTERM when you do "doas service
>citserver stop". This should run the same code that cleanly shuts
>down citserver when you do a ".ATN y" from the text client. All the
Perfect. That's exactly the desired behavior, and it will be "even more
that
So FreeBSD's rc system sends a SIGTERM when you do "doas service citserver stop". This should run the same code that cleanly shuts down citserver when you do a ".ATN y" from the text client. All the BSDs opurrate this way, and since it is a standard way of doing things, SMF
>citserver should exit cleanly when it receives a SIGINT, SIGQUIT, or
>SIGKILL.
>
>This may be why using FreeBSD's rc system caused badness on my
>system.
And it so happens that FreeBSD is the reason I pulled that section of code
out.
This is an opportunity to remove all
citserver should exit cleanly when it receives a SIGINT, SIGQUIT, or SIGKILL.
This may be why using FreeBSD's rc system caused badness on my system.
Thu Aug 31 2023 11:10:32 EDT from rss <> Subject: Removed background and restart from citserver.
These functions should be managed by an external
yay
>When I build from master now, when I shutdown citserver, it core dumps right
>after closing the 0d database. Since I had an issue where webcit was core
>dumping and was not re-creatable by anyone else, I am wondering if this
is
>just me.
Remember that discussion, from early
Yet another status update!
With some sleep under my belt and a thankfully slow day at work I've been
able to poke around a little more.
Uncensored crashed a couple of times last night but I think it was because
of a null pointer bug I somehow introduced. Turning off the indexer
Yeesh. It's nearly 2:00am and I still haven't figured out the deadlock
problem.
But I can make it appear pretty reliably by doing these things together:
1. loadtest, with 3 threads
2. while true ; do ./sendcommand tdap ; sleep 1 ; done
I also have a room on the test system that
Yecch. The new code has a deadlock problem. The DB is rock solid though.
On it.
Oh, and in case I didn't make it abundantly clear: the remaining backend
decoupling work is complete, and work on more modern backends can begin soon.
LMDB? MariaDB? SeattleCloud Hosted DB? You name it, we'll be able to do
it.
Ok, I've got my FreeBSD system installed and have begun making changes to
fix any problems that came up.
I am also now running Uncensored on 'git master' (commit
6960933711b8353ad5cddcc17994db670911a1dc)
as of right now. When it comes to dealing with database corruption, in the
words of
Ok, I've got my FreeBSD VM set up and will give the Citadel build a shot today.
\
I'm having *really* good results with this latest commit. I haven't gotten
it to corrupt the db no matter how badly I abuse it. This is the one.
The only issue so far is that under multiple very heavy load tests, I have
gotten it to deadlock. But I don't know whether that's a new
Geez. The more I look into the Berkeley DB back end the more fucked up I
find it is. Disregard the previous comment about the locking subsystem.
I changed dbenv->open() flags again, this time
DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN |
DB_RECOVER
| DB_THREAD
Some interesting developments here.
I disabled Berkeley DB's locking subsystem, because Citadel handles locking
and concurrency on its own. This seems to have fixed the segfault when it
closes the database environment. I also have not managed to corrupt the
database
no matter how badly
Clarification of the above commit -- because it's important.
When traversing an entire table (as opposed to just fetching one row for
which you already know the key) the backend API now returns both a key and
a value. This is done by returning a "struct cdbkeyval" which simply contains
a
>as for creating a patch, i would but if you wanna switch to cmake
>anyway it might not be necessary anymore. a word of caution though
>regarding cmake:
CMake is interesting and I'm happy to learn about it, but we've got too much
going on right now to think about switching to a new
Thu Aug 10 2023 15:02:19 EDT from IGnatius T Foobar Subject: Re: Build System
CPPFLAGS is not used anywhere because there is no C++ anywhere in the system.
CPPFLAGS are not related to c++ but are preprocessor flags (like: -I/opt/something/include). c++ compiler options would be CXXFLAGS.
UFarx: your suggestions for the build system are most welcome.
I don't suppose we could ask you to submit a patch? We have a GitLab instance
at [ https://code.citadel.org ] which accepts merge requests to the repository.
(You'll need to sign up for it using uf...@uncensored.citadel.org
I could be wrong, but when you first get here, it defaults to the last room seen by a non-logged in person. When you login, it stays there.
i always just go to the room list, to see what is new, for me.
i know, it was an accident. i actually was going to post there but logging in changed the current room.
I was mostly teasing on that. "get off my lawn" sort of thing :)
Tho as i scanned thru it seemed like one of the no-SQL sorts of things.
Tue Aug 08 2023 09:23:09 AM EDT from IGnatius T Foobar Subject: Re: y creashRe: Database Recovery Tools
The only thing that's "all the rage with the kids"
Ok, here's what I'm starting to do. You can totally follow the commit logs
to stay up to date.
database.c has been renamed to database_bdb.c for the obvious reason. (We've
been here before, but it was about 25 years ago when we switched from GDBM
to Berkeley DB.) The database interface
The only thing that's "all the rage with the kids" is *calling* it "NoSQL".
Berkeley DB has been a simple key/value data store for 30 years. And until
recently it's worked fine for us.
One of those new fangled no-sql things that is all the rage with the kids these days?
Mon Aug 07 2023 10:44:40 AM EDT from IGnatius T Foobar Subject: Re: y creashRe: Database Recovery Tools
To be honest I'm leaning towards LMDB,
To be honest I'm leaning towards LMDB, and for the people running on Raspberry
Pi we will have to say "look, you're just going to have to install the 64-bit
OS before you upgrade"
Read through this presentation on LMDB. It's practically designed for our
use case:
[
The options currently under consideration are SQLite, MariaDB, and LMDB.
SQLite: well known, well respected, embeddable. To do it right, however,
would require adding a schema layer. The idea of just throwing away the SQL
semantics and storing all records as blobs kind of bothers me
With as long as it was gone, i figured it was not intentional. :(
I know i cant contribute code from lack of time and i'm so out of practice, but SQLite might be an easy and safe replacement? ( unless you go with the total abstraction model then its academic )
Mon Aug 07 2023 09:56:00 AM
Ok, I think I found it.
[ https://docs.oracle.com/cd/E17276_01/html/installation/build_unix_small.html
]
Easy Install now uses the "--enable-smallbuild" flag, which disables verify.
I am going to change it to
--with-cryptography=no --disable-hash
I changed your project role from Developer to Maintainer, so hopefully you
can do merges now. I'm also going to try to get GitLab to send more "news"
to this room.
Ok, here is the DB installation:
$MAKE $MAKEOPTS || die
$MAKE install_lib || die
$MAKE install_include || die
$MAKE install_utilities || die
So they *should* be there. Are your tools not on the system, or did it throw
an error message when you tried
Did your db get corrupted because you were playing around with a development
system, or did it happen on its own on a production system?
I've just about had it with Berkeley DB, which is why I've spent a large
number of hours on ctdldump/ctdlload. This is our escape hatch once we have
an
Probably a role thing.
Crap. Ok, I'll do the merge, but I really need to figure out what the heck
is happening.
I do not have permissions to check in. No button to merge and my icon shows
"Cannot merge". I am logged in.
> Mon Jul 31 2023 06:25:58 PM EDT from IGnatius T Foobar Subject: Re: Merge
>Request Approval: IMAP memory issues with use of ConstStr
>
>Sorry about that, it just got away from
Sorry about that, it just got away from me. I approved it. Looks good.
This is still waiting for approval. Do you want me to bring the branch up
to the head or anything else?
I agree. The developer who introduced StrBuf and ConstBuf was helpful in
giving us an elastic string class, but I wish he hadn't gone back through
old code and converted stuff without a really deep inspection of how it was
used.
While looking at the use of ConstStr, it was interesting how it was being
used. "Const" is definably not the description to use. It really is just
a StrBuf where the pointer does not need to be freed. All sorts of code
change the pointer string and really did not check what it is doing.
>Not sure why the parameter was empty, but a lot of code accessed ConstStr
>pointers without checking for zero length and pointer was null and could
>coredump, so went through and added checks.
*sigh*
The IMAP server was written completely with straight C pointers and string
>I am changing to updating the records on they fly rather than upgrading,
so
>the version number is really not that important. New records use a version
>signature and don't compare to the citserver version so no conflict.
Sounds good. We are officially changed over from
Forgot to say that I was not logged in either and that was what was causing
the check-in. I did not notice since I had access to everything.
Strange
I am changing to updating the records on they fly rather than upgrading, so
the version number is really not that important. New records use a version
signature and don't compare to the citserver version so no conflict.
Oh, and I pushed the release button a few times, so when the visit merge is
done we'll have to adjust the version numbers being applied.
> If not then I'll merge and then troubleshoot. I think when we tell it
>not to squash the commits, it brings in your entire commit history
>followed by a merge commit.
Ok, I went ahead and tried it. It did exactly that. The commits that make
up the merge are written to the log
I had the same problem, and then I realized I was not logged in. Any chance
that's happening to you?
If not then I'll merge and then troubleshoot. I think when we tell it not
to squash the commits, it brings in your entire commit history followed by
a merge commit.
Tried to create a new Merge Request for a different branch and when I click
to create it, the page just spins and never creates it. Something is up
with GitLab
> Thu Jul 20 2023 02:28:36 PM EDT from HarlowSolutions Subject: Re:
>Approval Request: Memory_Leak_SmtpClient
>
>
>
>Either I
Either I do not have permission to merge, or something is wrong with
GitLab. When I bring up the Merge Request, the left menu has a bunch of
spinning updates that never complete. I was able to assign the merge to
myself, but that did not change anything, except there is a warning on my
icon that
Geez. Nice find. Approved, of course.
All good, take as much time as you need on the IMAP flags stuff. We'll make
sure we do it right.
As you've undoubtedly noticed if you follow the commit logs, over the last
couple of weeks I've written "ctdldump" and "ctdlload" utilities that can
be used to dump a Citadel database into a flat
After I finish up the IMAP Flags, I should have some time to contribute to
the database. I was initially thinking of SqlLite. The trick is what
database schema we want to support. I was looking at keeping with the
key/value concept and just replace database.c. I have an idea of the schema
for
To get things started, I have begun work on `ctdldump` and `ctdlload`
frameworks.
I've also improved the Makefile for Citadel Server so that it doesn't require
compiling the entire server every time even one file is touched. I got lazy
when I removed the GNU Autotools and just had it
Potentially, yes. I'm thinking more in terms of how to refactor the code.
The database access stuff is all in `database.c` because there *was* a somewhat
pluggable architecture in the past -- from flat files to gdbm and later to
Berkeley DB.
Since we've been using Berkeley DB exclusively
MariaDB and/or MySQL
Sun Jul 09 2023 09:46:26 EDT from IGnatius T Foobar Subject: Re: Master: citserver coredump on Shutdown
Grrr. Unrecoverable error on my database this morning. I think maybe it is time to put Berkeley DB out to pasture after all. Let's think about what we want the next to
Grrr. Unrecoverable error on my database this morning. I think maybe it
is time to put Berkeley DB out to pasture after all.
Let's think about what we want the next to look like.
>Pretty consistent for me. I put a check in the Berkley code to work around,
>but not sure of the root cause either.
Something must be getting double freed, but I can't figure out what.
>I just realized that a Seen message set of *:2099348686 means all messages
>have been seen. Were all your messages unseen before the upgrade? I
do
A seen message set of *:2099348686 means all messages have been seen up to
and including 2099348686.
We are using what RFC 3501
Pretty consistent for me. I put a check in the Berkley code to work around,
but not sure of the root cause either.
Duh. Button next to Comment button. I closed.
>I was trying to figure out how to reject the request myself. Does not
look
Do you have the button "Close merge request" on the bottom of the screen?
>It looks like the Berkley code is passing a NULL to a memory free routine
>that does not check for NULL. Anyone else having a problem? If not,
I
>will look into how I build the server.
It happens to me too, some of the time, and I have been busting my butt trying
to figure it
I just realized that a Seen message set of *:2099348686 means all messages
have been seen. Were all your messages unseen before the upgrade? I do
ignore Visit records when upgrading that look bad. If they got skipped, the
message set would be blank and all messages would show as unseen after
I was trying to figure out how to reject the request myself. Does not look
like you can. I Googled and others were talking about how GitLab was
lacking that option. If you can figure out how to reject or delete it, I
will not be offended unless you don't tell me how :-) Otherwise I will just
Ok then, I think the proper workflow is to cancel the merge request and start
a new one? I could reject it but that seems rude :)
It isn't just "a seen message set" -- it was ALL of them, across all rooms,
at least for me. But if you want an example, my current seen-set for this
room
Thanks for the feedback. My testing only had a small database, so 10 min is
rather bad. I think switching to doing the conversion when accessed is the
best idea. It is simple since that is how I did it before I switched to the
upgrade. I only have to add some code to one function. I am going
You mean like this? ;)
[ https://code.citadel.org/citadel/citadel-docker ]
I'd like to get things trending towards more deployment of the Docker image
(or technically an OCI compatible container, since you don't have to run it
under Docker, blah blah blah)
The current distribution is
>That seems to imply that Sqlite might be a better option than
>MySQL/MariaDB because we do *not* want to require people to install a
>working database server before Citadel can be installed.
Devil's Advocate from an Armchair Programmer:
these days, if you bundle an app as a
Ok, I tried out the proposed merge. Here are the results.
1. I made a copy of the Uncensored database and ran it on the main line code
to verify that the copy is functioning identically to the production database.
2. I switched over to the merged branch and built everything from
>If you want to stay with index/data for the database, I think I might go
back
>and see if I can finish up a database.c replacement with MySql. What I
Hey, I'm totally cool with moving to a SQL database, if someone else is doing
to be doing the bulk of the work. Seriously. I am just
No rush on the review. I had to put the code aside for a while (pesky day
job :-) and had to write some tests to try to hit most code paths.
I tried not to change too much existing code since the change is pretty big
to begin with. I only tried to rewrite things that affected performance or
If you want to stay with index/data for the database, I think I might go back
and see if I can finish up a database.c replacement with MySql. What I
don't have is the statistics for a large email server (mine is very small).
I wrote a small C program that went through all the Berkley databases
>Just to chime in, the database.c code is very clean. The only problem
I
>have had with Berkeley is that if the code or system crashes, most of the
>time, the database is corrupted and recover does not help sometimes and
I
>have to revert to a backup. The code has become a lot more
1 - 100 of 1528 matches
Mail list logo
