Re: assertion failed: imap/mboxevent.c: 743: filled_params(type, event)
Bug filed:
https://github.com/cyrusimap/cyrus-imapd/issues/3115
On 7/17/20 1:52 AM, Matthew Schumacher wrote:
HI Ellie,
I agree that it's probably a bug. I'll open a github issue.
I'll report back with the issue number.
Thanks,
Matt
On 7/16/20 5:47 PM, ellie timoney wrote:
Hi,
I've seen something like this before, and my gut feel is that this is
going to turn out to be a bug in Cyrus.
I think what's happening is that, somewhere in Cyrus, an event is
being generated with a type that's supposed to contain a
serverAddress field, but the serverAddress field is not being
initialised.
Before a generated event actually gets sent out to the notifier, we
validate that all the required parameters have been filled
("filled_params()"), and the fatal assertion is telling us that this
one has not been, even though it should have been.
Would you like to open a GitHub issue at
https://github.com/cyrusimap/cyrus-imapd/issues ? If you don't, I
will. But if I do it, then you won't get automatic notifications
about updates, so if you can, it's better if you do it. Feel free to
just paste your previous email as the issue text. :)
Cheers,
ellie
On Thu, Jul 16, 2020, at 11:23 AM, Matthew Schumacher wrote:
I'm trying to use external notifications on 3.2.2 but it doesn't work.
If I define
event_notifier: external
notify_external: /usr/cyrus/bin/cyrus_notify
event_groups: access
event_extra_params: clientAddress timestamp service
Then the imapd thread dies with this assertion:
Jul 15 18:01:54 snow imaps[28108]: Cannot notify event Login: missing
parameters: serverAddress clientAddress
Jul 15 18:01:54 snow imaps[28108]: Fatal error: Internal error:
assertion failed: imap/mboxevent.c: 743: filled_params(type, event)
If I remove event_groups and event_extra_params then notify never calls
my external script and notify breaks.
If I define "event_groups: access" and omit event_extra_params then I'm
back to:
Jul 15 18:14:26 snow imaps[28934]: Cannot notify event Login: missing
parameters: serverAddress
Jul 15 18:14:26 snow imaps[28934]: Fatal error: Internal error:
assertion failed: imap/mboxevent.c: 743: filled_params(type, event)
Anyone know where this serverAddress is coming from and how to fix it?
schu
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: assertion failed: imap/mboxevent.c: 743: filled_params(type, event)
HI Ellie,
I agree that it's probably a bug. I'll open a github issue.
I'll report back with the issue number.
Thanks,
Matt
On 7/16/20 5:47 PM, ellie timoney wrote:
Hi,
I've seen something like this before, and my gut feel is that this is going to
turn out to be a bug in Cyrus.
I think what's happening is that, somewhere in Cyrus, an event is being
generated with a type that's supposed to contain a serverAddress field, but the
serverAddress field is not being initialised.
Before a generated event actually gets sent out to the notifier, we validate that all the
required parameters have been filled ("filled_params()"), and the fatal
assertion is telling us that this one has not been, even though it should have been.
Would you like to open a GitHub issue at
https://github.com/cyrusimap/cyrus-imapd/issues ? If you don't, I will. But
if I do it, then you won't get automatic notifications about updates, so if you
can, it's better if you do it. Feel free to just paste your previous email as
the issue text. :)
Cheers,
ellie
On Thu, Jul 16, 2020, at 11:23 AM, Matthew Schumacher wrote:
I'm trying to use external notifications on 3.2.2 but it doesn't work.
If I define
event_notifier: external
notify_external: /usr/cyrus/bin/cyrus_notify
event_groups: access
event_extra_params: clientAddress timestamp service
Then the imapd thread dies with this assertion:
Jul 15 18:01:54 snow imaps[28108]: Cannot notify event Login: missing
parameters: serverAddress clientAddress
Jul 15 18:01:54 snow imaps[28108]: Fatal error: Internal error:
assertion failed: imap/mboxevent.c: 743: filled_params(type, event)
If I remove event_groups and event_extra_params then notify never calls
my external script and notify breaks.
If I define "event_groups: access" and omit event_extra_params then I'm
back to:
Jul 15 18:14:26 snow imaps[28934]: Cannot notify event Login: missing
parameters: serverAddress
Jul 15 18:14:26 snow imaps[28934]: Fatal error: Internal error:
assertion failed: imap/mboxevent.c: 743: filled_params(type, event)
Anyone know where this serverAddress is coming from and how to fix it?
schu
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
assertion failed: imap/mboxevent.c: 743: filled_params(type, event)
I'm trying to use external notifications on 3.2.2 but it doesn't work. If I define event_notifier: external notify_external: /usr/cyrus/bin/cyrus_notify event_groups: access event_extra_params: clientAddress timestamp service Then the imapd thread dies with this assertion: Jul 15 18:01:54 snow imaps[28108]: Cannot notify event Login: missing parameters: serverAddress clientAddress Jul 15 18:01:54 snow imaps[28108]: Fatal error: Internal error: assertion failed: imap/mboxevent.c: 743: filled_params(type, event) If I remove event_groups and event_extra_params then notify never calls my external script and notify breaks. If I define "event_groups: access" and omit event_extra_params then I'm back to: Jul 15 18:14:26 snow imaps[28934]: Cannot notify event Login: missing parameters: serverAddress Jul 15 18:14:26 snow imaps[28934]: Fatal error: Internal error: assertion failed: imap/mboxevent.c: 743: filled_params(type, event) Anyone know where this serverAddress is coming from and how to fix it? schu Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Unable to subscribe to folders.
List, Anyone see this before? For some reason I simply can't subscribe to my folders. Looking at it from the protocol level: a list (subscribed) "" "*" returns . * LIST (\Subscribed \HasNoChildren) "/" Vendors/Tools * LIST (\Subscribed \HasNoChildren) "/" Vendors/Travel * LIST (\Subscribed \HasNoChildren) "/" cron a list "" "*" returns . * LIST (\HasNoChildren) "/" Vendors/Tools * LIST (\HasNoChildren) "/" Vendors/Travel * LIST (\HasNoChildren) "/" cron * LIST (\HasNoChildren) "/" fred So then I: a subscribe "fred" and get: a OK Completed But then my folder is still not subscribed to. Like above. a list (subscribed) "" "*" returns . * LIST (\Subscribed \HasNoChildren) "/" Vendors/Tools * LIST (\Subscribed \HasNoChildren) "/" Vendors/Travel * LIST (\Subscribed \HasNoChildren) "/" cron It does the same thing with cyradm: localhost> lm fred fred (\HasNoChildren) localhost> sub fred localhost> lm fred fred (\HasNoChildren) localhost> lm Vendors/Travel Vendors/Travel (\Subscribed \HasNoChildren) Even my sub file shows I'm subscribed: grep fred /var/spool/imap/user/s/schu.sub user.schu.fred Anyone know what to look at next? Thanks, schu Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Preventing users from deleting a spam folder.
On 12/13/2016 02:08 PM, Patrick Boutilier via Info-cyrus wrote: > On 12/13/2016 05:23 PM, Matthew Schumacher via Info-cyrus wrote: >> Is there a way to change this? > > What version of Cyrus? In 2.4.18 I can set to just lrs . Do you have > implicit_owner_rights defined in imapd.conf? > > > implicit_owner_rights: lkxa > The implicit Access Control List (ACL) for the owner of a > mailbox. > That was it, I didn't realize I could set the implicit_owner_rights. So now I can create a protected spam mailbox. Next is to have sieve sort the spam into the spam folder. Thanks! Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Preventing users from deleting a spam folder.
Forgot to sent to the list: On 12/13/2016 12:02 PM, Patrick Boutilier via Info-cyrus wrote: > On 12/13/2016 04:56 PM, Matthew Schumacher via Info-cyrus wrote: >> Expiring the email looks simple enough, but I can't quite seem to figure >> out how to create a spam mailbox they can't delete. If I create a >> user.fred.spam mailbox and set fred to only have read permissions, it >> gives the user admin permissions which allows the user to delete the >> mailbox. > > Remove the x right. > > http://www.cyrusimap.org/~vanmeeuwen/imap/admin/access-control/rights-reference.html > Thanks for the help, but I can't seem to do that: localhost> sam user/schu/Spam schu lrs localhost> lam user/schu/Spam schu lrskxca The user always ends up with the kxca rights if the mailbox lives under their INBOX. Is there a way to change this? Thanks again, schu Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Preventing users from deleting a spam folder.
Hello, I would like to start moving spam into a spam folder for each of my users instead of dropping it, then deleting it after a few weeks. Expiring the email looks simple enough, but I can't quite seem to figure out how to create a spam mailbox they can't delete. If I create a user.fred.spam mailbox and set fred to only have read permissions, it gives the user admin permissions which allows the user to delete the mailbox. What is a good way to go about this? Re-create the mailbox if the user deletes it? Any suggestions really appreciated! schu Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Bug or feature: (Too) Many imapd processes hanging around?
On 10/08/2012 12:48 PM, Matthew Schumacher wrote:
I have exactly the same issue:
I found on my system that this file:
/cyrus/mailboxes/e/user/escalations/cyrus.index
Was being held open by 6423 children processes which all reported:
fcntl(17, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}
The process that was holding the lock reported:
futex(0x7f6d86e0f720, FUTEX_WAIT_PRIVATE, 2, NULL
After I killed it, everything else seems to be closing out, but slowly.
As a data point I'm running a fairly different config as Andreas:
Ext4 file system on native partition (though it's a vmware disk) so no drdb
Cyrus 2.4.16 with the following patches:
cyrus-imapd-2.4.4-autocreate-0.10-0.patch
cyrus-imapd-2.4.12-autosieve-0.6.0.patch
This just happened to me again:
root@mail:/home/schu# lsof | grep
/cyrus/mailboxes/s/user/support/cyrus.index | head
imapd 2013 cyrus mem REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
imapd 2013 cyrus 17u REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
imapd 2929 cyrus mem REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
imapd 2929 cyrus 17u REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
imapd 3180 cyrus mem REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
imapd 3180 cyrus 17u REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
imapd 3444 cyrus mem REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
imapd 3444 cyrus 17u REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
imapd 4600 cyrus mem REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
imapd 4600 cyrus 17u REG8,8 26432
4034886 /cyrus/mailboxes/s/user/support/cyrus.index
A total of 282 processes are hung on this file
This process has the lock:
root@mail:/home/schu# strace -p 27029
Process 27029 attached - interrupt to quit
futex(0x7f6ff0a7a720, FUTEX_WAIT_PRIVATE, 2, NULL
When I kill it with TERM I get:
) = ? ERESTARTSYS (To be restarted)
--- SIGTERM (Terminated) @ 0 (0) ---
Process 27029 detached
Then my process list starts shrinking as the other 281 processes close out.
I should note that the mailboxes this happens on are being monitored by
multiple users so there are multiple connections to this mailbox and the
others that have had this problem.
schu
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Bug or feature: (Too) Many imapd processes hanging around?
On 10/04/2012 04:41 AM, Andreas Haumer wrote:
I checked about 20 processes and all of them hang on the F_SETLKW fcntl()
call on fileid #16 which always points to the same file
(/cluster/var/imap/user/o/office.seen)
Using lsof I could identify the process which was holding the
lock on /cluster/var/imap/user/o/office.seen:
ravel:/var/log # lsof /cluster/var/imap/user/o/office.seen | grep 16uW
imapd 27522 cyrus 16uW REG 147,057804 3686853
/cluster/var/imap/user/o/office.seen
This process itself was waiting in a futex() call!
A SIGHUP did not help, but a SIGTERM made the process terminate:
ravel:~ # strace -p 27522
Process 27522 attached
futex(0x7f5c8b149620, FUTEX_WAIT_PRIVATE, 2, NULL) = ? ERESTARTSYS (To be
restarted if SA_RESTART is set)
--- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=20435, si_uid=0} ---
rt_sigreturn() = 202
futex(0x7f5c8b149620, FUTEX_WAIT_PRIVATE, 2, NULL) = ? ERESTARTSYS (To be
restarted if SA_RESTART is set)
--- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=20435, si_uid=0} ---
+++ killed by SIGTERM +++
I would rather not upgrade to 2.4 bypassing the official OpenSUSE
packages (well, if absolutely necessary, I could, but I would like
to avoid this path)
I have exactly the same issue:
I found on my system that this file:
/cyrus/mailboxes/e/user/escalations/cyrus.index
Was being held open by 6423 children processes which all reported:
fcntl(17, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}
The process that was holding the lock reported:
futex(0x7f6d86e0f720, FUTEX_WAIT_PRIVATE, 2, NULL
After I killed it, everything else seems to be closing out, but slowly.
As a data point I'm running a fairly different config as Andreas:
Ext4 file system on native partition (though it's a vmware disk) so no drdb
Cyrus 2.4.16 with the following patches:
cyrus-imapd-2.4.4-autocreate-0.10-0.patch
cyrus-imapd-2.4.12-autosieve-0.6.0.patch
lessLogs.patch:
diff -ruN cyrus-imapd-2.4.16.orig/imap/userdeny_db.c
cyrus-imapd-2.4.16/imap/userdeny_db.c
--- cyrus-imapd-2.4.16.orig/imap/userdeny_db.c 2012-04-19
01:39:51.0 -0800
+++ cyrus-imapd-2.4.16/imap/userdeny_db.c 2012-08-24 17:01:40.276482897
-0800
@@ -81,7 +81,6 @@
if (!deny_dbopen) return 0;
/* fetch entry for user */
-syslog(LOG_DEBUG, fetching user_deny.db entry for '%s', user);
do {
r = DENYDB-fetch(denydb, user, strlen(user), data, datalen, NULL);
} while (r == CYRUSDB_AGAIN);
Here is my database config:
duplicate_db: skiplist
tlscache_db: skiplist
Linux is 3.2.29 (Slackware 14.0)
So unless one of my patches is breaking something, I suspect that there
is a bug in cyrus-imap.
Thanks,
schu
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
POP3d won't timeout, seems stuck on a write.
List, I have the latest cyrus 2.3.16 but I run into frequent pop lock issues where the pop3 server process just won't timeout and it locks the account. I have used a patch followed on this list that allows me to set a pop3timeout less than 10 minutes and currently it's set at 3 minutes but that didn't help because the server doesn't seem to honor any timeout. To test that the client isn't sending anything, I connected to the process with strace and it shows absolutely nothing going: r...@server:/var/log# time strace -p 16996 Process 16996 attached - interrupt to quit write(1, 3nI8\212S%\315\n\221\363m\213\203\377\3101\30(\334\332..., 2762 unfinished ... Process 16996 detached real17m0.327s user0m0.000s sys 0m0.004s It looks like it's stuck on a write of some kind, but that is blocking the timeout. Anyone have any ideas? This problem is causing upper management to want to switch to exchange and I really don't want to deal with that. schu Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Create subfolder permissions.
List, I have a folder with the following permissions: lam user/matt/Folder matt lrswited Which omits both the administrative (a) and the create (s) sub-folder permissions. I am using these settings because I want to provide users with a folder that they can't delete or create subfolders in, but can do everything else. It seems the administrative part works fine, but I am able to create sub-folders. What is really interesting is that I can't delete them. Does anyone know why this doesn't work. I'm sure I'm missing something obvious. Info: Cyrus 2.3.16 with autocreate patches. schu Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Connection throttling POP3.
David S. Madole wrote: If you are talking about the suggestion I made, which looked like this: iptables -A INPUT -p tcp --dport 22 \ -m state --state NEW \ -m recent --update --seconds 60 -j DROP iptables -A INPUT -p tcp --dport 22 \ -m state --state NEW \ -m recent --set -j ACCEPT then you did not read it right. It limits to one connection per IP address per minute. Each source address is kept track of in enforcing the limit. Using the --hitcount option in addition to the --seconds option, you can also create limits such as a maximum of four connections in two minutes, etc. David Wow, I never played with recent before but it's quite handy. Thanks for pointing this out. I'm already added a number of rules to protect various things. schu Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Connection throttling POP3.
David S. Madole wrote: From Matthew Schumacher on Monday, May 21, 2007 6:35 PM I agree with Blake, while I can do it with IPtables it's not a good solution. The first iptables suggestion blocked the offending IP, which is fine, but also requires me to babysit the server. The second suggestion would correctly limit connections, but if I'm reading it right, would lump all connections together, not just connections per originating IP address. If you are talking about the suggestion I made, which looked like this: iptables -A INPUT -p tcp --dport 22 \ -m state --state NEW \ -m recent --update --seconds 60 -j DROP iptables -A INPUT -p tcp --dport 22 \ -m state --state NEW \ -m recent --set -j ACCEPT then you did not read it right. It limits to one connection per IP address per minute. Each source address is kept track of in enforcing the limit. Using the --hitcount option in addition to the --seconds option, you can also create limits such as a maximum of four connections in two minutes, etc. David This must be something that the recent module does. I'll do some testing Thanks for the helpful suggestion, it looks very promising. schu Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Connection throttling POP3.
List, I'm getting some spammer trying to guess usernames and passwords: May 21 11:01:55 larry pop3[5845]: badlogin: [83.209.35.32] plaintext bob SASL(-13): authentication failure: checkpass failed May 21 11:01:54 larry pop3[5860]: badlogin: [83.209.35.32] plaintext complaints SASL(-13): authentication failure: checkpass failed May 21 11:01:56 larry pop3[5922]: badlogin: [83.209.35.32] plaintext diablo SASL(-13): authentication failure: checkpass failed May 21 11:01:58 larry pop3[5924]: badlogin: [83.209.35.32] plaintext darren SASL(-13): authentication failure: checkpass failed May 21 11:02:00 larry pop3[5927]: badlogin: [83.209.35.32] plaintext dallas SASL(-13): authentication failure: checkpass failed May 21 11:02:00 larry pop3[5939]: badlogin: [83.209.35.32] plaintext edgar SASL(-13): authentication failure: checkpass failed May 21 11:02:01 larry pop3[5945]: badlogin: [83.209.35.32] plaintext cristopher SASL(-13): authentication failure: checkpass failed May 21 11:02:02 larry pop3[5965]: badlogin: [83.209.35.32] plaintext easter SASL(-13): authentication failure: checkpass failed May 21 11:02:10 larry pop3[5964]: badlogin: [83.209.35.32] plaintext felicia SASL(-13): authentication failure: checkpass failed And this spammer is racking up a zillion processes which is killing my machine. I need a way to throttle this somehow where he is only allowed one connection per IP at a time, or perhaps a way to ignore them after so many invalid passwords. Anyone know of a way to do this? schu Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Connection throttling POP3.
Blake Hudson wrote: These types of threats are becoming more and more common and in reaction awareness is increasing and more software seems to be implementing mechanisms to cope. I would personally love to see Cyrus implement some sort of connection limit or throttling per IP/network/user. The current process limits do help ensure that one daemon does not make the machine unusable, but does nothing to prevent a DoS attack. -Blake I agree with Blake, while I can do it with IPtables it's not a good solution. The first iptables suggestion blocked the offending IP, which is fine, but also requires me to babysit the server. The second suggestion would correctly limit connections, but if I'm reading it right, would lump all connections together, not just connections per originating IP address. The pam suggestion doesn't really free up processes since the connections would still be made, not to mention that I'm not using pam, so that is pretty much out. Fail2ban is interesting (I could whip this up in perl in 10 minutes) but it's kind of a hack. In the end it would be best to have this part of cyrus. That way we can do different things based on number of connections in a time period, number of simultaneous connections, or password failure. Perhaps someone can add it to the wish list, I would write it myself except my C skills are lacking. Perhaps I'll just write some perl hack to scan the logs until there is a better way to do it. Thanks, schu Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus, clusters, GFS - HA yet again
Janne Peltonen wrote: But I still seem to get some weird DB errors, the same I used to: if I log in and out on the node on which Cyrus was started first, the imapd process that accepted my connection complains about DBERROR on exit: --clip-- Oct 30 09:21:19 lcluster2 imap[10378]: login: localhost.localdomain [127.0.0.1] cyrus plaintext User logged in Oct 30 09:22:21 lcluster2 imap[10378]: DBERROR db4: PANIC: fatal region error detected; run recovery Oct 30 09:22:21 lcluster2 imap[10378]: DBERROR: critical database situation Oct 30 09:22:21 lcluster2 master[10368]: process 10378 exited, status 75 Oct 30 09:22:21 lcluster2 master[10368]: service imap pid 10378 in READY state: terminated abnormally --clip-- Janne, I can't make bdb work reliably on a single host let alone the config your trying to run. I would try converting that database to skiplist and try again. schu Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Configuring cyrus imap with postfix and mysql database
Sanchez Nicolas wrote: Hi ! First, I'm sorry but I'm french and I don't speak english very well, so if you don't understand all I write, say it to me. So I would like to install on a server, postfix+cyrus-imap+saslauth+mysql database for authentification. I've found this tuto but I think my server don't work: http://www.campworld.net/thewiki/pmwiki.php/LinuxServersFC5/FC5VirtMailServer I explain... In first, I've installed the packages cyrus-imapd cyrus-imapd-utils perl-Cyrus pam_mysql and pam_devel. Then,I've installed postfix with mysql support from here : http://www.campworld.net/downloads/postfix-2.2.8-1.2.i386.rpm After : fetchmail mdadm(don't know why, but it was in the tuto...). Then, I've created my database with the files in attachment. Then, lots of config files modifications I have done, but not really understand for pam and saslauthd(see the end of the mail for config files). After, the Cyrus IMAP configuration... I've created a cyrus user (adduser+passwd) and modified the cyrus conf files. After , I've configured postfix and sasl files. Then i've launched saslauthd and cyrus-imapd. Now the pb (Ouch!): I don't know how to test my conf. I've send a mail to [EMAIL PROTECTED] and an other to [EMAIL PROTECTED] Where can I found the mails i've send on my server? How can I configure thunderbird to get the mails? Let's see my account config: -Server type : imap -address : cyrus or admin @sd-910.dedibox.fr -server name : sd-910.dedibox.fr -account name : cyrus or admin -port: 143 -no secure connexion -smtp: ??? (smtp.gmail.com cause I don't know what I have to write) I'm sorry for this long, long mail, but It's my first mail server installation and I don't know at all where is my prblem in configuration. Please, help me! Thanks in advance, Nico Nico, First of all your English isn't as bad as you say. Your request made perfect sense to me. Anyway, I don't run this setup so I'm not going to be able to walk you though it, but I can offer some insight on how to troubleshoot and how this works. The first thing I would do is setup mysql, pam_mysql, pam, and confirm that is all working. So make sure that your data is in your database, then make sure that pam_mysql can query this information, then setup pam to use pam_mysql as it's authentication module. Once you are there you can test everything by trying to chown a file as a user in your mysql database. Also, try to troubleshoot in order. PAM uses pam_mysql as it's authentication module, which queries mysql. Once you have your mysql users available in pam then work on then configuring cyrus to use saslauthd with this syntax in your imapd.conf: sasl_pwcheck_method: saslauthd Once cyrus is configured to check users against saslauthd, then you need to start the saslauthd daemon and tell it you want it to resolve users against pam: # saslauthd -a pam Now that saslauthd is checking against pam, you need to create a pam config file to tell pam which modules to use when saslauthd trys to authenticate: Put this into your /etc/pam.d/imap auth sufficient pam_mysql.so user=mail passwd=secret host=localhost db=mail table=accountuser usercolumn=username passwdcolumn=password crypt=1 logtable=log logmsgcolumn=msg logusercolumn=user loghostcolumn=host logpidcolumn=pid log timecolumn=time accountrequired pam_mysql.so user=mail passwd=secret host=localhost db=mail table=accountuser usercolumn=username passwdcolumn=password crypt=1 logtable=log logmsgcolumn=msg logusercolumn=user loghostcolumn=host logpidcolumn=pid log timecolumn=time Make sure you use the correct host, user, database, table, columns, etc. So in order: Your cyrus server talks to the saslauthd daemon, which talks to the pam abstraction later, which talks to the pam_mysql module, which makes queries against mysql. Hope that helps, schu Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: High availability email server...
Andrew Morgan wrote: On Fri, 28 Jul 2006, Rich Graves wrote: My question: So is *anyone* here happy with Cyrus on ext3? We're a small site, only 3200 users, 246GB mail. I'd really rather not try anything more exotic for supportability reasons, but I'm getting worried that our planned move from Solaris 9/VxFS to RHEL4/ext3 on significantly newer and faster hardware is going to be a downgrade. We run Cyrus on ext3 under Debian Linux without complaints here. We have approximately 35000 mailboxes/users split between 2 backend servers. Each backend server is connected to an EMC Cx500 SAN (no shared access or anything fancy) with 800GB of mail spool each. The commands used to build the filesystems were: mkfs -t ext3 -j -m 1 -O dir_index /dev/sdb1 tune2fs -c 0 -i 0 /dev/sdb1 The filesystem is mounted like so: /dev/sdb1/private ext3defaults,data=ordered,noatime 0 2 If you want more information, just ask. :) Andy We also use ext3 not because I think it's the fastest or has the most features but because it just works. We do volume management with EVMS and I had a lot of trouble getting XFS and other file systems to snapshot correctly under heavy load without the box eventually running into a situation where all processes started to hang waiting for IO eventually causing a system crash. Ext3 worked every time so the choice was obvious. I figured if it would survive a snapshot while I'm hitting it very hard with postal then odds of having problems in prod are going to be pretty slim. One thing that ext3 does have going for it is the fact that it is the most tested and most common file system on linux. schu Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Migrate QMail to Cyrus
Patrick T. Tsang wrote: Read this link, and then you know why mailutil is NOT good enough. http://www.webservertalk.com/message1280487-1.html I am also having this problem. To improve the speed, I have to tar the mail to the new server and do reconstruct, and then use imapsync to re-build the imap flag and re-subscrib the imap folder. However, I am still not satisfied by this method. Please kindly share me a better way. BTW, people using SAN for clustering solution is welcome, too. Patrick When We did this we needed to loop though our ldap database so I wrote a perl script that read the uw-imap mbox files and copied them to cyrus via imap. I found that it was slow so I rewrote it to spool a new child for each mbox file, with 10 threads running at a time. This made it much faster, but it still wasn't that fast. After some tweaking I got it to go at a reasonable speed, but at some point I think I was hardware bound. My script didn't block up the messages so consumed a ton of memory (which was recovered when each child died) however it did work. I used up every minute of my maintenance window to get it moved, but it did work and there where minimal issues the following day. If I had a bigger site, I would probably use some form of imap proxy to aggregate imap traffic to the old and new server while I migrated a couple thousand accounts at a time. In the end mail migrations are a pain, but cyrus is worth it. Performance and features are better than any other imap server I have found. Add this to mimedefang/sendmail and you have a very good mail solution. schu Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Berkeley DB upgrade?
Paul Boven wrote: Hi everyone, I'm planning an upgrade from a Cyrus-2.2.8 to Cyrus-2.2.12. However, we will also be upgrading Berkeley from 4.1.25 to 4.4.20 because of some performance issues with the first. So now I'm wondering: should I call db_upgrade from Berkeley on each of these databases? And which databases exactly would that be? This is a stock standard 2.2.8, with no specific backends mentioned in the imapd.conf or compiled in. What are the filenames/locations where I should find Berkeley db files? Some candidates: /var/imap/mailboxes.db - but Berkeley-db fails to recognise that file, so I can't dump it. /var/imap/annotations.db - same problem. /var/imap/tls_sessions.db is a Berkeley db-file, but one I could afford to wipe during the upgrade. /var/imap/deliver.db is a Berkeley db-file, but also one I could affort to wipe. Paul, If it where me I would downgrade your bdb back to 4.1.25 so that you can read the files then export them to text format. Once you upgrade cyrus, import them into skiplist format and be done with bdb. I have a fairly large system and was always bumping into bdb problems so about a week ago I finally decided to go to skiplist format and haven't looked back. Skiplist should also make things easier next time you upgrade since it doesn't depend on external libraries and their versions. HTH, schu Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Berkeley DB upgrade?
Paul Boven wrote: Thank you for your reply. I was already considering changing to skiplist, but at the moment the problem is that I don't know which database is where, and what kind of backend it is. On my test-machine, I think I've located all Berkeley's, but Cyrus is completely dead, failing with a 'DB4' error. So first of all I'm trying to find which ones I'm missing - which is getting a bit urgent, given the planned upgrade for tomorrow :-/ Regards, Paul Boven. If you use the command `file` you can figure out what is what: # file mailboxes.db mailboxes.db: Apple QuickTime movie file (skip) That is a skiplist file... # file /etc/mail/aliases.db /etc/mail/aliases.db: Berkeley DB (Hash, version 7, native byte-order) That is a bdb file. On your production system you should be able to use: # su cyrus -c /usr/cyrus/bin/ctl_mboxlist -d mailboxes.txt To dump the mailboxes to a file, then later after you change your imapd.conf to use skiplist, you can import it with: # su cyrus -c /usr/cyrus/bin/ctl_mboxlist -u mailboxes.txt HTH, schu Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
BDB problems with lockers.
Cyrus users, I have been having a problem with my cyrus 2.2.13 install over the last couple of days. First I keep getting this DBERROR log: Jul 1 15:37:55 server lmtpunix[18456]: DBERROR db4: 49980 lockers Several messages I read on the list stated to ignore it, it wasn't a big deal, however this doesn't seem to apply on larger installations that eventually fill the lock table: Jul 1 15:42:50 server imaps[20139]: DBERROR db4: Lock table is out of available locker entries Jul 1 15:42:50 server imaps[20139]: DBERROR: opening /var/spool/imap/tls_sessions.db: Cannot allocate memory Jul 1 15:42:50 server imaps[20139]: DBERROR: opening /var/spool/imap/tls_sessions.db: cyrusdb error Jul 1 15:47:05 server lmtpunix[21385]: DBERROR db4: Lock table is out of available locker entries Jul 1 15:47:05 server lmtpunix[21385]: DBERROR: opening /var/spool/imap/deliver.db: Cannot allocate memory Jul 1 15:47:05 server lmtpunix[21385]: DBERROR: opening /var/spool/imap/deliver.db: cyrusdb error So now we have a problem, the database quits and eventually becomes corrupted: Jul 1 15:47:06 server lmtpunix[20989]: DBERROR db4: 5 lockers Jul 1 15:47:07 server lmtpunix[21391]: DBERROR db4: Lock table is out of available locker entries Jul 1 15:47:07 server lmtpunix[21391]: DBERROR db4: PANIC: Cannot allocate memory Jul 1 15:47:07 server lmtpunix[21391]: DBERROR: critical database situation Jul 1 15:47:07 server lmtpunix[21407]: DBERROR db4: PANIC: fatal region error detected; run recovery So now the question is how do I fix this? I am running BDB-4.2.52 with all 4 patches so the only thing I can do with bdb is to upgrade to the 4.4.20 release, but I'm unsure if this release is any more stable. My mailboxes file seems safe since it is in skiplist format, so maybe I should convert the deliver database to skiplist as well? What about cyrus 2.3.6? Is this stable enough for a large server? Does it work around any of the BDB problems? Thanks, schu Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
