Re: cyrus-imapd-2.4.17-caldav-beta5 released
Hi Ken, Out of curiosity, is there support for replication in the caldav release? Thanks, Rudy
Re: Special-Use and Cyrus 2.5 git
Hi Bron, We are not using it. And have no plans in rushing any upgrades :) Quoting Bron Gondwana br...@fastmail.fm, Fri, 26 Apr 2013: Is anybody actually using cyrus 2.5 git in production other than FastMail? I'm planning to do the mailboxes.db format changes before releasing 2.5, so that we don't have to support a stupid intermediate format. It's been on my todo list for AGES as one of the final blockers to 2.5 being released, and I've finally got some spare cycles away from our internal search code. But it would simplify things considerably if I don't have to write an importer that goes upstream. I'd just manually script a conversion of the FastMail servers, and then toss away the intermediate code. Of course - I can't do that if a bunch of you are already using it - so speak now (in the next week or so) or forever deal with your specialuse fields going away on upgrade. (converting from 2.4 xlist_foo: config options is always going to be messy, but that's a given) Bron. -- Bron Gondwana br...@fastmail.fm -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert e-mail: rudy.geva...@ugent.be Directie ICT, Afdeling Infrastructuur Groep Systemen tel: +32 9 264 4750 Universiteit Gent fax: +32 9 264 4994 Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Underscore in SERVICES-names can cause deadlocks.
On 03/20/2012 06:42 PM, Henrique de Moraes Holschuh wrote: On Tue, 20 Mar 2012, Robert Linden wrote: The tricky part is, that not all names that look distinct really are treated differently by cyrus. Notable example: imap and imap_remote result in the same lockfile (/var/imapd/socket/imap-0.lock), because the name is truncated at the '_'. The '_' doesn't seem to be illegal per se, because cyrus does not complain and the service runs ok most of the time, i.e. as long as no concurrent connections are created on the 2 interfaces to trigger the race- condition. Maybe other characters cause this problem as well, so if one is unsure about this it would be best to just stick to alphanumeric characters. IMHO we really should refuse service names that will cause issues, which probably means we should enforce that services be named [a-z][a-z0-9]+ to avoid potential pitfalls. This is the second report about the permissive acceptance of service names causing bad surprises... I think _ isn't allowed (but it isn't detected) because in imapd.conf we use the _ to configure per service overrides. I'm all for tightening up the allowed characters.
Re: What do we want from the Changelog?
Hello, I have a comment that (in my opinion) fits in this thread. I would like to see a good way of pointing out new imapd.conf options. Now you have to look at bug reports, changelog, install-upgrade to find them. Sometimes they are not documented even, but that is a different problem. Regarding install-upgrade.html, I think that this file is not read by default by most sysadmins. E.g. I just look in the changelog. If it mentions install-upgrade.html I go look at it. (Unless I know there are big changes.) Rudy
Re: FOSDEM 2012 Meet Greet?
On 11/04/2011 06:56 PM, Jeroen van Meeuwen (Kolab Systems) wrote: Hi there, It looks like both myself as well as Bron Gondwana are going to attend FOSDEM[1] early next year, February 4-5 in Brussels, Belgium. In fact, it might be as bad as the two of us giving a talk on Cyrus - watch for further announcements while we try to work out the details. If you love Free Software, FOSDEM is undoubtedly *the* must-attend event in Europe. If you love Belgium beers, Brussels is by far the must-be place in Europe. So, if you like Cyrus, or Bron, or me, or Free Software, or Belgium beers, or any or all of the above, well then your first weekend in February should be spoken for by now. I'm curious who else would be interested in attending! Without making any promises, let me try to point out that with enough attendees, but moderate amount, we might also try and arrange a hackaton on, in and/or around FOSDEM. I should normally be there too. I live not far from Brussels. (In Belgium nothing is far from Brussels.) Rudy
Re: Cyrus 2.4.9 Released
Hi Bron, Thanks for you reply. On Fri, Jun 24, 2011 at 06:10:15PM +0200, Bron Gondwana wrote: Yes, that is quite possible! I recommend running quota -f twice to fix it, possibly even with the server down if you're worried about mail bouncing during the short time that quota usage is incorrectly doubled. Taking it offline is impossible. Luckily we increased everybodies quota to 5G some time before. I don't think many will hit that limit. If I understand you correctly, if I run the quota -f twice the probably will be totally fixed? Or will it show again, till the bug is fixed. I really hope to have a fix to the quota issue soon, and to ship 2.4.10. Unfortunately it really needs a complete reworking of the quota tool, and also some changes to how lib/cyrusdb_quotalegacy.c handles foreach. Then I can fix ALL the outstanding quota related bugs in one big fix :) I'm sure I'm not the only one that is appreciating the hard work Opera is doing on cyrus. Rudy
2.4.8 sync_client crashes
Hi,
After upgrading to 2.4.8 the sync_client crashes regularly:
Fatal error: Internal error: assertion failed: seen_db.c: 127:
*seendbptr == NULL
Please see the attachment for a strace.
I tried reconstructing the mailbox but that didn't help...
On the syncserver I don't see any messages :(
Rudy
open(/mail/mail16/var/imap/lock/domain/u/ugent.be/v/user/victoria^montesrestrepo.lock,
O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
fcntl(8, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=0, len=0}) = 0
fcntl(3, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=0, len=0}) = 0
fstat(3, {st_mode=S_IFREG|0600, st_size=2315764, ...}) = 0
stat(/mail/mail16/var/imap/mailboxes.db, {st_mode=S_IFREG|0600,
st_size=2315764, ...}) = 0
fcntl(3, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0
open(/mail/mail16/imap/domain/u/ugent.be/v/user/victoria^montesrestrepo/cyrus.index,
O_RDWR) = 9
fstat(9, {st_mode=S_IFREG|0600, st_size=30752, ...}) = 0
mmap(NULL, 40960, PROT_READ, MAP_SHARED, 9, 0) = 0x7f74c7581000
fcntl(9, F_SETLKW, {type=F_RDLCK, whence=SEEK_SET, start=0, len=0}) = 0
stat(/mail/mail16/imap/domain/u/ugent.be/v/user/victoria^montesrestrepo/cyrus.header,
{st_mode=S_IFREG|0600, st_size=219, ...}) = 0
open(/mail/mail16/imap/domain/u/ugent.be/v/user/victoria^montesrestrepo/cyrus.header,
O_RDONLY) = 10
fstat(10, {st_mode=S_IFREG|0600, st_size=219, ...}) = 0
mmap(NULL, 219, PROT_READ, MAP_SHARED, 10, 0) = 0x7f74c7694000
munmap(0x7f74c7694000, 219) = 0
fcntl(9, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0
close(10) = 0
close(9)= 0
munmap(0x7f74c7581000, 40960) = 0
close(8)= 0
munmap(0x7f74c750, 528384) = 0
write(1, META tine^lievr...@ugent.be\n, 28META tine^lievr...@ugent.be
) = 28
sendto(7, 182Apr 21 16:47:33 mail16/sync..., 74, MSG_NOSIGNAL, NULL, 0) = 74
write(6, GET META tine^lievr...@ugent.be\r..., 33) = 33
read(6, * LSUB (ugent.be!INBOX^Femke \ug..., 4096) = 592
write(2, Fatal error: Internal error: ass..., 82Fatal error: Internal error:
assertion failed: seen_db.c: 127: *seendbptr == NULL
) = 82
sendto(7, 179Apr 21 16:47:33 mail16/sync..., 128, MSG_NOSIGNAL, NULL, 0) =
128
exit_group(75) = ?
cyrus@cyrprd3:~$ /usr/cyrus-2.4.8/bin/sync_client -C
/etc/cyrus-ugent/conf/mail16/imapd.conf -l -v -f
/mail/mail16/var/imap/sync/log-6813 -r -v -v
MAILBOXES ugent.be!user.victoria^montesrestrepo
META tine^lievr...@ugent.be
Fatal error: Internal error: assertion failed: seen_db.c: 127: *seendbptr ==
NULL
cyrus@cyrprd3:~$ echo $?
75
Re: 2.4.8 sync_client crashes
On 04/21/2011 06:30 PM, Simon Amor wrote: On 21 Apr 2011, at 16:51, Rudy Gevaert wrote: As some extra info if I run sync_client -u for that user I get the crash too. I have attached a gdb backtrace. I'm not sure if it will be handy. Rudy Hi Rudy, After our discussion on IRC, I would suggest the attached patch as a fix - it seems to follow the same syntax as the other places seen_open() is called. Disclaimer: My C is very rusty so it might fail horribly. The patch is against clean 2.4.8 sources. Perhaps Bron could give it a quick check and if it's correct? Simon Hi Simon, I see this is fixed? on the master branch, commits: 5de9eee60d947243a4b4b2f4eccc63cff2771b30 9dacbfcd6ee077a850570508d0d97f30bfe96640 Bron? :)
cyr_expire[14157] trap divide error
Hi, I'm seeing the following message in my kern.log every morning when cyr_expire is run: cyr_expire[14157] trap divide error ip:4432d5 sp:7fff96f38c90 error:0 in cyr_expire[40+136000] When I run gdb with the core file I see: Core was generated by `/usr/cyrus-2.4.6/bin/cyr_expire -C /etc/cyrus-ugent/conf/mail21/imapd.conf -D 3'. Program terminated with signal 8, Arithmetic exception. #0 0x004432d5 in hash_lookup (key=0x7f330b5050a9 Address 0x7f330b5050a9 out of bounds, table=0x7fff96f38fd0) at hash.c:163 163 hash.c: No such file or directory. in hash.c (gdb) backtrace #0 0x004432d5 in hash_lookup (key=0x7f330b5050a9 Address 0x7f330b5050a9 out of bounds, table=0x7fff96f38fd0) at hash.c:163 #1 0x00420a56 in prune_p (rock=0x7fff96f38d70, id=0x7f330b505098 Address 0x7f330b505098 out of bounds, idlen=value optimized out, data=0x7f330b5050c4 Address 0x7f330b5050c4 out of bounds, datalen=value optimized out) at duplicate.c:281 #2 0x0043ccda in myforeach (db=0x16b2b90, prefix=value optimized out, prefixlen=0, goodp=value optimized out, cb=value optimized out, rock=value optimized out, tidptr=0x0) at cyrusdb_skiplist.c:1101 #3 0x004209c2 in duplicate_prune (days=8, expire_table=0x7fff96f38fd0) at duplicate.c:321 #4 0x00407662 in main (argc=value optimized out, argv=0x0) at cyr_expire.c:496 I provide the full core dump uppon request. Thanks! Rudy
rename 2.4.5 assertion failed
Hi, I'm testing the rename functionality on 2.4.5: Dec 7 12:11:20 cyrdev1 maild1/imaps[24583]: starttls: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits new) no authentication Dec 7 12:11:20 cyrdev1 maild1/imaps[24583]: login: pimp.ugent.be [157.193.44.68] cyrus plaintext+TLS User logged in SESSIONID=maild1-24583-1291720280-1 Dec 7 12:11:20 cyrdev1 maild1/imaps[24583]: Deleted mailbox mail.ugent.be!user.test2 Dec 7 12:11:20 cyrdev1 maild1/imaps[24583]: error renaming /mail/maild1/sieve/domain/m/mail.ugent.be/t/test2 to /mail/maild1/sieve/domain/m/mail.ugent.be/r/rename2: No such file or directory Dec 7 12:11:20 cyrdev1 maild1/imaps[24583]: cannot unlink /mail/maild1/var/imap/domain/m/mail.ugent.be/user/t/test2.mboxkey: No such file or directory Dec 7 12:11:20 cyrdev1 maild1/imaps[24583]: Fatal error: Internal error: assertion failed: cyrusdb_skiplist.c: 727: db-current_txn == NULL imap commands debug: 12917202802 RENAME user/te...@mail.ugent.be user/rena...@mail.ugent.be 1291720280* OK rename user/te...@mail.ugent.be user/rena...@mail.ugent.be 1291720280* BYE Fatal error: Internal error: assertion failed: cyrusdb_skiplist.c: 727: db-current_txn == NULL I am not seeing this on an other server also running 2.4.5 Thanks! Rudy
Re: rename 2.4.5 assertion failed
On 12/07/2010 12:24 PM, Rudy Gevaert wrote: On 12/07/2010 12:19 PM, Rudy Gevaert wrote: I am not seeing this on an other server also running 2.4.5 correction: I am seeing this on an other host too. Removing the mailbox through cyradm works but closes the connection with error. In log I see: Dec 7 12:26:58 cyrprd2 mail8/imap[31089]: Deleted mailbox mail.ugent.be!user.rename2 Dec 7 12:26:58 cyrprd2 mail8/imap[31089]: cannot unlink /mail/mail8/var/imap/domain/m/mail.ugent.be/user/r/rename2.mboxkey: No such file or directory Dec 7 12:26:58 cyrprd2 mail8/imap[31089]: Fatal error: Internal error: assertion failed: cyrusdb_skiplist.c: 727: db-current_txn == NULL Dec 7 12:26:58 cyrprd2 mail8/imap[31089]: skiplist: closed while still locked
expire 2.4.5 on replica?
Hello, With the recent changes in 2.4.X it isn't clear to me if I need to run cyr_expire (to expunge mails) on the replica too. Now I am running delprune cmd=/usr/cyrus-2.4.4/bin/cyr_expire -C /etc/cyrus-ugent/conf/mail8/imapd.conf -D 8 -E 8 -X 8 at=0400 on master and replica... Thanks! Rudy
Re: 2.4.5 ?
On 11/26/2010 03:06 AM, Bron Gondwana wrote: hen will 2.4.5 be released? I'm aiming for Tuesday next week if I can get Jeroen on board. I'm planning to spend Monday working with Greg to get fixes for all the known bugs and test them. Good luck!! :) Rudy
Re: Cyrus IMAPd 2.4.0 Released
On 10/11/2010 11:48 PM, Ken Murchison wrote: I am pleased to announce the release of Cyrus IMAPd 2.4.0. This is really great news! I'm going to test asap. Thank you to everyone who contributed. - Replication code largely rewritten Bron, two specific questions for you about the replication. I saw that this code base can recover from a split brain. What is the procedure? Is there in any documentation about it? Previously I was using 0019-GUID-IMAP-COMMANDS.patch. Is there any hope in getting this updated? Or is it already applied? I found this a great way to check if the master and replica are in sync. Thanks! Rudy
Re: Cyrus IMAPd 2.4.0 Released
On 10/13/2010 02:30 PM, Bron Gondwana wrote: On Wed, Oct 13, 2010 at 09:23:11AM +0200, Rudy Gevaert wrote: On 10/11/2010 11:48 PM, Ken Murchison wrote: I am pleased to announce the release of Cyrus IMAPd 2.4.0. This is really great news! I'm going to test asap. Thank you to everyone who contributed. - Replication code largely rewritten Bron, two specific questions for you about the replication. I saw that this code base can recover from a split brain. What is the procedure? Is there in any documentation about it? It's automatic. The replication code notices that just applying the changes from the master to the replica doesn't generate the same SYNC_CRC value, and it falls back to a full mailbox fetch and comparison. GUID mismatch messages are promoted to new UIDs and flag changes are made one direction or the other depending who has the most recent modified time. Obviously if there are flag change clashes (like you set \Flagged at one end and \Seen at the other) then one end will lose detail - we don't have change history per flag in the underlying format. Interesting. Would this mean that one could set up to run sync client on both master and replica and set sync_log to 1 on both? So that one could failover one user to the replica e.g. and get his changes synced back to the 'old master'? Previously I was using 0019-GUID-IMAP-COMMANDS.patch. Is there any hope in getting this updated? Or is it already applied? I found this a great way to check if the master and replica are in sync. It still exists :) Check github.com/brong/cyrus-imapd/branches/fastmail - which is what we apply on top of 2.4 for the FastMail packages. It's not something we want to push upstream because it's non-standard IMAP, and the eventual plan is to add the per-message ANNOTATION RFC and expose the values as read-only annotations. That would indeed by to way to go. But for now it's ok with me :) I already applied your patch and I'm compiling.. :) Rudy Btw, I got a lot of inspiration from your mkdebian.pl :)
Re: cyrus 2.4 upgrade path?
On 09/18/2010 04:25 PM, Bron Gondwana wrote: On Sat, Sep 18, 2010 at 08:52:01AM +0200, Rudy Gevaert wrote: Dear list, I've seen on the new website that 2.4 would be released soon. We are really looking forward to the new improvements. However, at this moment I'm busy with setting up our new cyrus infrastructure. I will need to migrate 4TB of data to this new infrastructure. Will updates from the current latest release be easy? Just install the new software and start up? Or will more things need be taken care of? Pretty much. I will double-triple-check that of course. It is an index upgrade, so it's hard to downgrade again afterwards. You probably don't want to put the initial testing release onto your main production systems straight away. We did, but that's because we had me watching, and it had to be put through fire somewhere. If it will not be easy then maybe I should consider waiting some time? (But I'm already overdue... :) It will be easy - easy upgrade is important :) Replication between old and 2.4 doesn't work - but we're going to make sure murder does! Bron. Thanks for the info Bron. :)
cyrus 2.4 upgrade path?
Dear list, I've seen on the new website that 2.4 would be released soon. We are really looking forward to the new improvements. However, at this moment I'm busy with setting up our new cyrus infrastructure. I will need to migrate 4TB of data to this new infrastructure. Will updates from the current latest release be easy? Just install the new software and start up? Or will more things need be taken care of? If it will not be easy then maybe I should consider waiting some time? (But I'm already overdue... :) Thanks! -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert e-mail: rudy.geva...@ugent.be Directie ICT, Afdeling Infrastructuur Groep Systemen tel: +32 9 264 4750 Universiteit Gent fax: +32 9 264 4994 Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: sync_authname usability
On Wed, Jul 14, 2010 at 01:45:12PM -0400, Wesley Craig wrote: Would more privilege separation actually improve the security model in the cases above? Hi Wesley, I think it surely would. Rudy
sync_authname usability
Dear list, I want to share the following remarks concerning sync_user configuration and usage. In a setup where the master and replica are on different servers and you don't mix master and replica's you can set up rather good security model. master imapd.conf: admin: cyrus sync_authname: syncuser replica: imapd.conf admins: cyrus, syncuser However if you are running replica's and masters on the same server (of different instances) you'll have your sync_password on the server in plain text. And thus the possibility of getting it abused (only on the replica). However, if you want to be able to failback, and you'll need to add your syncuser to the admins of the master server. In the end, your are just easier and better of in using one user for replication and admin. However I like to possibility to have a different user for replication. It would maybe be nice to have some more privilege separation between the replication and admin users. E.g. the replication user don't have to be in the admin list. Wouldn't it? I agree one could use some kind of password (Kerberos or md5 or ssl certs.) less authentication, but that adds extra complexity if you don't have (had) that already running. Kind regards, Rudy
Re: Modifying cyrus.index to add message flags
Quoting Ken Murchison mu...@andrew.cmu.edu: I haven't looked at your code, but its a lot easier to do what you want using the IMAP protocol using a tool such as mailutil(1). I believe there are others, but I don't recall their names at the moment. imapsync -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert rudy.geva...@ugent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: ANNOTATEMORE = METADATA and rfc 5464
On Tue, Nov 17, 2009 at 11:28:49PM +1100, Bron Gondwana wrote: Does anybody out there use annotations much? Does anybody know any code that would be broken by changing the way annotations are done? I'm the only one who uses it here ;) -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert rudy.geva...@ugent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: [patch] New option: proxy_passfile
Øyvind Kolbu wrote: Hi We use cyrus 2.3.12p2 as standalone nodes behind a nginx proxy, and found that we needed to migrate users from one node to another. The solution was the options proxy_authname, proxy_password and proxyservers. However we have our configfiles in CVS and distribute the commit diffs on a mailinglist, and hence we don't wan't the cleartext password in the configfile. The simple attached patch reads the password from a given file. Hi, Nice effort, but imapd.conf also supports an include paramter. I do: include: /file/syncpass and then /file/syncpass has sync_password: 1234 It is not the same as your patch but it works :) Rudy
Re: 2.3.12 statuscahe off_t size bug
Hi,
Could this be revised and imported in CVS please?
John Capo wrote:
Found a bug handling the off_t index_size value in the statuscache
code. The code assumes off_t is a 32 bits and the message count gets
written as 0.
(gdb) p *scdata
$22 = {statusitems = 63, index_mtime = 1208893338, index_ino = 1974639, index_size = 6581528,
messages = 74789, recent = 174, uidnext = 1547527, uidvalidity = 1125360596, unseen = 74743,
highestmodseq = 0}
(gdb) n
247 r = DB-store(statuscachedb, key, keylen, data, datalen, NULL);
(gdb) p data
$23 = 3 63 1208893338 1974639 6581528 0 74789 174 1547527 1125360596 74743\000
The attached patch assumes off_t is 64 bits. Are there any 32 bit
off_t systems left?
John Capo
--
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734
Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office
Groep SystemenSystems group
Universiteit Gent Ghent University
Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Cyrus 2.3.12 RC2
Ken Murchison wrote: I just put together a second release candidate for Cyrus 2.3.12. I'd appreciate any independent testing before I release this to the masses. http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.12rc2.tar.gz I have tested the following: * Added option to unexpunge to restore messages by time interval -- courtesy of David Carter Works very nicely! * Added serverinfo option to control the information displayed in banner greetings and capability responses. Works * Implemented incremental squat updates (see squatter.8) -- courtesy of David Carter Seems to work, (running it works and it doesn't barf on virtual mailboxes) I also tested if reconstruct does his recurstion alright on new virtual mailboxes. (With a cyrus.header file only). That works too. * Added option to mbexamine to compare quota usage in cyrus.index to the actual message file sizes. This works, but has a bug. More specific. If I do an unexpunge on a mailbox the quota on a replica mailbox isn't any more correct. Quota mailbox on master: [EMAIL PROTECTED]:~$ quota -d dict.ugent.be user/rudy Quota % Used Used Root 25 1948071 user/[EMAIL PROTECTED] Quota mailbox on replica: [EMAIL PROTECTED]:~$ quota -C /etc/imapd-replica.conf -d dict.ugent.be user/rudy Quota % Used Used Root 25 1948071 user/[EMAIL PROTECTED] Both are in sync. I do an unexpunge: [EMAIL PROTECTED]:~$ unexpunge -a -d user/[EMAIL PROTECTED] restoring all expunged messages in mailbox 'dict.ugent.be!user.rudy' restored 14 out of 14 expunged messages Quota on master: [EMAIL PROTECTED]:~$ quota -d dict.ugent.be user/rudy Quota % Used Used Root 25 1948117 user/[EMAIL PROTECTED] Quota on replica not in sync: [EMAIL PROTECTED]:~$ quota -C /etc/imapd-replica.conf -d dict.ugent.be user/rudy Quota % Used Used Root 25924028 user/[EMAIL PROTECTED] mbexamine says it's all ok [EMAIL PROTECTED]:~$ mbexamine -C /etc/imapd-replica.conf -q user/[EMAIL PROTECTED] Examining user/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Examining user/rudy/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Examining user/rudy/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Examining user/rudy/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Examining user/rudy/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Examining user/rudy/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Examining user/rudy/rudy/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Examining user/rudy/rudy/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Examining user/rudy/rudy/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Examining user/rudy/rudy/[EMAIL PROTECTED] Mailbox has CORRECT total quota usage Fix the quota and it now it's in sync again: [EMAIL PROTECTED]:~$ quota -C /etc/imapd-replica.conf -d dict.ugent.be -f user/rudy dict.ugent.be!user.rudy: usage was 24605660, now 49272171 Quota % Used Used Root 25 1948117 user/[EMAIL PROTECTED] -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Cyrus 2.3.12 RC1
Ken Murchison wrote: I just put together a release candidate for Cyrus 2.3.12. I'd appreciate any independent testing before I release this to the masses. http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.12rc1.tar.gz Noteworthy changes: * Added statuscache.db to cache IMAP STATUS data which significantly reduces the amount of I/O necessary when neither the mailbox nor \Seen state has changed -- courtesy of Fastmail.fm * Added option to unexpunge to restore messages by time interval -- courtesy of David Carter * Implemented undocumented IMAP SCAN extension, which allows Pine/Alpine to do cross-mailbox searches -- based on work of David Carter * Implemented incremental squat updates (see squatter.8) -- courtesy of David Carter * Fixed major bugs in reconstruct -k implementation -- courtesy of David Carter Check doc/changes.html for a complete list of changes. If there are any outstanding issues that you believe still need to be addressed in 2.3.12, please let me know. Hi Ken, I have tested 2.3.12rc1 and didn't notice any problems. However I would like to see bug https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3028 fixed. Thanks in advance, Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
quota and replication
Hi, One of our users managed to share her mailbox with somebody else, and then revoked her own access. After fixing that I had a look at how the mailbox got replicated. It was out of sync. I resynced it. But whatever I try, the quota won't get in sync! On the master and replica 'du -s' reports: 58144 On the master 'lqr' reports 53781/256000, while on the replica it reports 53737/256000 Running quota/reconstruct on both sides hasn't got me any further. Removing quota and setting it again didn't too. My script that checks if the messages on the master and replica are the same reports that it is in sync. Does anyone have any hints? Thanks! -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
about the replication
Hi, The past week I finally set my mind to make some scripts to check if a mailbox on the master was the same as on the replica. My back-ends started running 2.3.7 and I upgraded to 2.3.10 some time ago. I ran my script last night on one of the back-ends. This one has 20928 mailboxes. And 1067 are not in sync. From those, 897 users didn't have all their mailboxes replicated to the replica. The others had the same folder count, but one or more messages were not the same (or weren't there). To check if the folder count was the same I just counted the folders on the master and the replica. To see if the messages were the same, I used the GUID of each mail message. So, if you are using replication, you should really have a look at it, if you have haven't started already :) Also, I would really like to see the GUID commands taken up in upstream! Thanks to fastmail.fm for hacking that into cyrus and releasing there patches. Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: about the replication
Rob Mueller wrote: I ran my script last night on one of the back-ends. This one has 20928 mailboxes. And 1067 are not in sync. From those, 897 users didn't have all their mailboxes replicated to the replica. The others had the same folder count, but one or more messages were not the same (or weren't there). We also have a script which does this for all users every week. It checks for each user that the following match: 1. Folder list 2. Folder subscriptions 3. Quota (total + used) 4. Sieve scripts 5. For every folder, status output for (messages uidnext unseen recent uidvalidity) 6. For every message in every folder, flags + message sizes + GUID I'm moving to a mix of thoses. Once you have the framework adding an extra check is only a little more work. FYI, unfortunately at the moment this script is fairly closely tied to our system (it uses a bunch of other modules to work out master/replica servers for a user). It probably would be nice to factor that stuff out so others could use it... Mine is fairly generic, when it's more robust I'll put it online somewhere. Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
condstore
Hello, Cyrus has support for condstore. However when using replication condstore doesn't work. Could someone explain to me what the downside is when have condstore enabled on a mailbox and using replication? Thanks in advance, -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: guid mismatch
Bron Gondwana wrote: On Thu, Nov 08, 2007 at 05:04:59PM +0100, Rudy Gevaert wrote: Hi, So, I'm running 2.3.10 here in production, I've also added some patches of fastmail (thanks guys!). E.g. the guid mismatch patch (it's reports when the master and replica are out of sync for a message). I was running 2.3.7 and I'm sure that my replica wasn't worth a thing. Now that I see tons of guid mismatch errors I'm certain it isn't. What would be the way of getting my replica in sync? I could delete the replica and resync it, but I would prefer not to do that. I think I could delete the offending user on the replica and issue a sync_client -u. Or I could delete the offending message on the replica and issue a sync_client -u (or -m the mailbox). Am I missing any other (not to difficult) options? Have you reconstructed both ends (with -G) to ensure that the different is actually the message and not the index record? Yep We just delete the entire user on the replica and sync_client -u generally in this case (after trying reconstruct and sync_client -u first) That is indeed the easiest. Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Fwd: mech_step takes long to return
Aditya Khasnis wrote: Hello, We have a LDAP server that uses Cyrus SASL library v 1.5.27. On AIX 5.2, we observe that the SASL searches take long to return. The behavior is such that the first SASL search that we fire returns fast but the subsequent search takes long time to return. I have tried to debug SASL library and in the place where it takes long is the function sasl_server_start(), and exact location is line 1205. It will be great if you great if you could provide us any guidance to debug the problem. The mechanism we are using in the search is DIGEST-MD5. Slowdown in Sasl is most of the time related to the lack of entropy. Q: I'm having performance problems on each authentication, there is a noticeable slowdown when sasl initializes, what can I do? A:libsasl reads from /dev/random as part of its initialization. /dev/random is a secure source of entropy, and will block your application until a sufficient amount of randomness has been collected to meet libsasl's needs. To improve performance, you can change DEV_RANDOM in config.h to be /dev/urandom and recompile libsasl. /dev/urandom offers less secure random numbers but should return immediately. The included mechanisms, besides OTP and SRP, use random numbers only to generate nonces, so using /dev/urandom is safe if you aren't using OTP or SRP. (http://www.sendmail.org/~ca/email/cyrus2/sysadmin.html) -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Cyrus 2.3.10 RC3
Ken Murchison wrote: I just put together a third release candidate for Cyrus 2.3.10 which fixes a couple of bugs, and hopefully resolves outstanding build issues with com_err and xversion. I'd appreciate any independent testing before I release this to the masses. If there are any outstanding issues that you believe still need to be addressed in 2.3.10, please let me know ASAP. If you see the time to fix https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2999 (sync_client and dot in username with unixhierarchysep: yes) I would be great. For the rest I haven't see any other problems. Thanks, Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: delayed delete bug?
Ken Murchison wrote: So, is it your opinion that this doesn't need to be fixed, or shouldn't be fixed? Te be honest, I would like to see it fixed. Thanks in advance, Rudy
delayed delete bug?
Hi, I'm busy looking at delayed delete. I'm using unix hierarchy seperator. I deleted a mailbox and see it like this: kavula.ugent.be lm DELETED/user/rudy.gevaert/Foo/[EMAIL PROTECTED] (\HasNoChildren) user/rudy.gevaert/[EMAIL PROTECTED] (\HasNoChildren) user/rudy.gevaert/[EMAIL PROTECTED] (\HasNoChildren) user/rudy.gevaert/[EMAIL PROTECTED] (\HasNoChildren) user/[EMAIL PROTECTED] (\HasChildren) However the file system location is: /var/cyrus/imap/domain/u/ugent.be/u/DELETED/user/rudy^gevaert/Foo/471364C1/ I would have thought it would be under domain/u/ugent.be/DELETED/user/... Is something going wrong here? I've run cyr_expire and it cleans it up nicely! Also, is it correct to undelete a deleted folder I need to rename it? (This works! And gets replicated). Thanks in advance, Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
addition for changes.html file
Hi, Maybe it would interesting to update changes.html so that it mentions that unexpunge now takes the mailboxes in the form of user/[EMAIL PROTECTED] (I can only that the unix hierarchy seperator here) in stead of the internal mailbox name. I just noticed this on my test server. Unfortunately it took some time till I figured out it. Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Cyrus 2.3.10 RC2
Rudy Gevaert wrote: I'll try to upgrade from 2.3.7, that is our production version, to 2.3.10 on my test machine. Together with replication. Just to confirm. I didn't see any problems (yet). Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Cyrus 2.3.10 RC2
Ken Murchison wrote: I just put together a second release candidate for Cyrus 2.3.10 which hopefully fixes a couple of build issues with RC1. I'd appreciate any independent testing before I release this to the masses. If there are any outstanding issues that you believe still need to be addressed in 2.3.10, please let me know. http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.10rc2.tar.gz Hi, It's still not really clear to me what to do to use replication with GUID. E.g. I'm running a version 2.3.10 and upgrade to 2.3.10 I stop master and slave. Upgrade software. Remove provide_uuid=X from cyrus.conf. Add guid_mode: sha1 to imapd.conf on both master and replica. Start replica, start master. install-upgrade says: Upgrading from 2.3.9 * The method used for generating Globally Unique IDentifiers used for replication has been changed to be the SHA1 hash of the messages. If you wish to upgrade the existing GUIDs in particular mailbox(es) or the entire server, perform the following steps in the listed order. Note that is is NOT REQUIRED that existing GUIDs be upgraded. 1. Zero GUIDs on the replica (reconstruct -g) 2. Regenerate GUIDs on the master (reconstruct -G) 3. Regenerate GUIDs on the replica (reconstruct -G) But it doesn't mention my case. Should I run -G on the master and replica? Or isn't it necessary? Or must I do it. Does it matter if the master/replica are(n't) running? What is the consequence if I don't zero the GUIDs? Or don't regenerate them? Thanks in advance, Rudy PS I don't have any trouble of summarizing the answers to something that can be added to install-upgrade. -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Cyrus 2.3.10 RC2
Bron Gondwana wrote: On Wed, Oct 10, 2007 at 06:26:43PM -0400, Ken Murchison wrote: I just put together a second release candidate for Cyrus 2.3.10 which hopefully fixes a couple of build issues with RC1. I'd appreciate any independent testing before I release this to the masses. If there are any outstanding issues that you believe still need to be addressed in 2.3.10, please let me know. http://www.contrib.andrew.cmu.edu/~murch/cyrus-imapd-2.3.10rc2.tar.gz Initial rebuild and apply of our patches is promising. After changing from uuidmode: md5 to guid_mode: sha1 anyway! I'm wondering if uuidmode: md5 is gone, how can I get I md5 hash of a message, so I can compare a message on the master and the replica? Or am I missing something? -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Cyrus 2.3.10 RC2
David Carter wrote: On Thu, 11 Oct 2007, Rudy Gevaert wrote: I'm wondering if uuidmode: md5 is gone, how can I get I md5 hash of a message, so I can compare a message on the master and the replica? Or am I missing something? make_md5 still works: it doesn't use the UUIDs/GUIDs. There is also: http://www-uxsup.csx.cam.ac.uk/~dpc22/cyrus/patches/2.3.8/sha1_make_sha1.patch which should probably be merged if we are working towards SHA1. Hello David, I've looked at make_md5, but could it be that it isn't aware of virtual domains and the unix hierarchy seperator? Thanks in advance, -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Ready for 2.3.10?
Hi, I would really appreciate if someone could give me some pointers to get the cvs trunk compiled. I eager to test :) Thanks in advance, Rudy Rudy Gevaert wrote: Ken Murchison wrote: Folks, I *think* 2.3.10 is just about ready. Does anyone feel that there are any showstopper bugs that need to be addressed before I make a release candidate? To help a bit, I have tried build from CVS. I checkout out the trunk, and I thought I should sh SMakefile. That complains because it doesn't have the 'sys' command. I commented that out, and then it complained about my perl version. I went to cyrus and cyrus-devel archives, and found a message from Bron that said something like, get the aclocal part and autoconf and autoheader parts out. So I ran: aclocal -I cmulocal (Do I actually need this? When I autoconf autoheader ./configure ... make depend make all CFLAGS=-O2 -g It then complained I didn't have 'yacc', I'm doing this in the same chroot that compiled my 2.3.7 and 2.3.9 from tarball. Why does it need yacc now? Anyway I installed bison. Now it breaks on: gcc -L/usr/lib -Wl,-rpath,/usr/lib -L/usr/lib/ -Wl,-rpath,/usr/lib/ -o sievec sievec.o libsieve.a ../lib/libcyrus.a ../lib/libcyrus_min.a libsieve.a -lsasl2 -lresolv -lresolv -L/usr/lib/ -Wl,-rpath,/usr/lib/ -ldb-4.3 -lssl -lcrypto /usr/lib/libcom_err.a libsieve.a(script.o)(.text+0x311): In function `sieve_script_parse': /tmp/cyrus-imapd/sieve/script.c:175: undefined reference to `yylineno' libsieve.a(sieve.o)(.text+0x1518): In function `yyparse': /tmp/cyrus-imapd/sieve/y.tab.c:1384: undefined reference to `yylex' libsieve.a(sieve.o)(.text+0x1555): In function `sieve_parse': ./sieve.y:677: undefined reference to `yyrestart' libsieve.a(sieve.o)(.text+0x15b6): In function `yyerror': ./sieve.y:694: undefined reference to `yylineno' libsieve.a(addr.o)(.text+0x669): In function `addrparse': /tmp/cyrus-imapd/sieve/y.tab.c:963: undefined reference to `addrlex' collect2: ld returned 1 exit status make[1]: *** [sievec] Error 1 make[1]: Leaving directory `/tmp/cyrus-imapd/sieve' make: *** [all] Error 1 Actually I don't think I need the m4 files in the cmulocal dir, it cmulocal stuff. But if I only do: check out cvs aclocal pimp:/tmp/cyrus-imapd# autoconf configure.in:110: error: possibly undefined macro: AC_DEFINE If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. autoheader ./configure ... ./configure: line 5662: CMU_SOCKETS: command not found ./configure: line 5666: syntax error near unexpected token `getaddrinfo,' ./configure: line 5666: `IPv6_CHECK_FUNC(getaddrinfo, IPv6_CHECK_FUNC(gai_strerror,' So I then tried with a ugent directory that only contained the macro's that where needed. But make all still stops at the same place as above. Any hints would be appreciated to compile from CVS. Thanks in advance, Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Making Replication Robust
Hello, I agree with Bron. However I do think some parts are more important than others. I'll try to explain my point of view. Note, we are running 2.3.7, I'm going to upgrade when 2.3.10 is out. We have replication in place, but daren't use it. If I have a method to check if the replica is in sync then I'll dare to do a fail over. For me points a, e and f are most important, but the others are also important. Bron Gondwana wrote: So I'd like to start a dialogue on the topic of making Cyrus replication robust across failures with the following goals: a) MUST never lose a message that's been accepted for delivery except in the case of total drive failure. b) MUST have a standard way to integrity check and repair a replica-pair after a system crash. Do you mean that if the replica crashes it should be able to catch up again? c) MUST have a clean process to soft-failover to the replica machine, making sure that all replication events from the ex-master have been synchronised. In deed this is nice, but it would still need a lot of site specific tools. E.g. I know (I think I do) that Fastmail runs master/replica in the same subnet. We don't. So soft-failover isn't that easy. For us it's more important that all mail that isn't delivered gets queued at the MTA (it's not on the same machine as cyrus). All delivered mails are replicated. We then still need to update the DNS or /etc/hosts file. d) MUST have replication start/restart automatically when the replica is available rather than requiring it be online at master start time. This would be great if there are some tools available for doing automatic failover, recovery, ... e) SHOULD be able to copy back messages which only exist on the replica due to a hard-failover, handling UIDs gracefully (more on this later), alternatively as least MUST (to satisfy point 'a') notify the administrator that the message has different GUIDs on the two copies and something will need to be done about it (to satisfy point 'd' this must be done without bailing out replication for the remaining messages in the folder) f) SHOULD keep replicating in the face of an error which affects a single mailbox, keeping track of that mailbox so that a sysadmin can fix the issue and then replicate that mailbox hand. g) MAY have a method to replicate to two different replicas concurrently (replay the same sync_log messages twice) allowing one replica to be taken out of service and a new one created while having no gaps in which there is no second copy alive (we use rsync, rsync again, stop replication, rsync a third time, start replication to the new site - but it's messy and gappy) Is again a good idea, and would be very usable. But this is depending what you will be doing with the second replica. If it would be possible to take out the second replica, to make it conssistent and back it up, and then make it up to date it would be a neat way have consistent backup. Kind regards, Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
[Fwd: suggestions]
Hello, I sent this mail regarding http://www.uit.co.uk/mail-conference/ Rudy Original Message Subject: suggestions Date: Mon, 08 Oct 2007 10:14:35 +0200 From: Rudy Gevaert [EMAIL PROTECTED] Organization: Universiteit Gent To: [EMAIL PROTECTED] Hello, I would like to see the following topics on the conference on email: - Zimbra - Cyrus Zimbra is a replacement of Exchange that is partially Free Software. It's a bit of hot in some places looking to replace exchange or place looking for something like exchange. Regarding Cyrus, there have been some very good improvements in Cyrus, and there a couple of really big organisations running Cyrus and that are helping doing active development. It would be a great opportunity to meet these people and hear some talks from them. E.g. Fastmail.fm is doing great work on the replication engine of Cyrus. David Carter from Cambridge university is also doing excellent work on it. The email conference could be a great opportunity to get some people who are running Cyrus in a big environment around the table. Kind regards, Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Ready for 2.3.10?
Ken Murchison wrote: Folks, I *think* 2.3.10 is just about ready. Does anyone feel that there are any showstopper bugs that need to be addressed before I make a release candidate? To help a bit, I have tried build from CVS. I checkout out the trunk, and I thought I should sh SMakefile. That complains because it doesn't have the 'sys' command. I commented that out, and then it complained about my perl version. I went to cyrus and cyrus-devel archives, and found a message from Bron that said something like, get the aclocal part and autoconf and autoheader parts out. So I ran: aclocal -I cmulocal (Do I actually need this? When I autoconf autoheader ./configure ... make depend make all CFLAGS=-O2 -g It then complained I didn't have 'yacc', I'm doing this in the same chroot that compiled my 2.3.7 and 2.3.9 from tarball. Why does it need yacc now? Anyway I installed bison. Now it breaks on: gcc -L/usr/lib -Wl,-rpath,/usr/lib -L/usr/lib/ -Wl,-rpath,/usr/lib/ -o sievec sievec.o libsieve.a ../lib/libcyrus.a ../lib/libcyrus_min.a libsieve.a -lsasl2 -lresolv -lresolv -L/usr/lib/ -Wl,-rpath,/usr/lib/ -ldb-4.3 -lssl -lcrypto /usr/lib/libcom_err.a libsieve.a(script.o)(.text+0x311): In function `sieve_script_parse': /tmp/cyrus-imapd/sieve/script.c:175: undefined reference to `yylineno' libsieve.a(sieve.o)(.text+0x1518): In function `yyparse': /tmp/cyrus-imapd/sieve/y.tab.c:1384: undefined reference to `yylex' libsieve.a(sieve.o)(.text+0x1555): In function `sieve_parse': ./sieve.y:677: undefined reference to `yyrestart' libsieve.a(sieve.o)(.text+0x15b6): In function `yyerror': ./sieve.y:694: undefined reference to `yylineno' libsieve.a(addr.o)(.text+0x669): In function `addrparse': /tmp/cyrus-imapd/sieve/y.tab.c:963: undefined reference to `addrlex' collect2: ld returned 1 exit status make[1]: *** [sievec] Error 1 make[1]: Leaving directory `/tmp/cyrus-imapd/sieve' make: *** [all] Error 1 Actually I don't think I need the m4 files in the cmulocal dir, it cmulocal stuff. But if I only do: check out cvs aclocal pimp:/tmp/cyrus-imapd# autoconf configure.in:110: error: possibly undefined macro: AC_DEFINE If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. autoheader ./configure ... ./configure: line 5662: CMU_SOCKETS: command not found ./configure: line 5666: syntax error near unexpected token `getaddrinfo,' ./configure: line 5666: `IPv6_CHECK_FUNC(getaddrinfo, IPv6_CHECK_FUNC(gai_strerror,' So I then tried with a ugent directory that only contained the macro's that where needed. But make all still stops at the same place as above. Any hints would be appreciated to compile from CVS. Thanks in advance, Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
use of guid instead of uuid
Hello, While I was going through the sources in CVS at the moment I saw that imapd.conf gained the guid_mode option, and lost the sync_machineid option. The provide_uuid paramater in cyrus.conf has been removed too. If people are going to upgrade from a configuration that uses sync_machineid= X and provide_uuid=1, and don't configure guid_mode = sha, what will happen? (Not that I'm going to do that) Could some documentation be missing? I saw some code being removed from master.c about uuid's. It al leads me to think that if you don't configure guid_mode to sha and assume the uuid way of working will not stop, you may be wrong. Am I right or wrong? Thanks in advance, Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Re: Cyrus IMAP Readonly partitions
Ken Murchison wrote: Michael D. Sofka wrote: I am working on a way to recover deleted (and purged) messages in Cyrus IMAP. This is not as silly as it may at first sound. On Exchange there is a setting whereby deleted items are not deleted from the server until they are backed up. Before this time, they can be ``recovered'' from the deleted items folder. (Why not, they're still available to the system.) Likewise, in our AFS file space (those were the days) we mount the read-only backup volume in a directory called ./yesterday so that oops, I didn't mean to delete that! did not result in a phone call to operations to restore a file. In AFS the backup volume is a copy on write snapshot of the volume created for backup. Mounting these gave access to the contents prior to the start of that day's backup. (And, AFS and Cyrus IMAP share a mother, if only by adoption :-) With that as background, our Cyrus IMAP servers use LVM snapshots for backups. We make the snapshot, mount it, back it up, and unmount it. Fairly standard. After a couple calls to restore oops inboxes, I thought maybe the snapshot could be mounted as a read-only IMAP partition. Users could access this to recover accidentally deleted messages. Alas, it is not quite that simple. The snapshot would just appear to Cyrus IMAP. The mailboxes database would then have to be updated with this new information, the access information would need to be set, etc. However, I am wondering if there isn't some trick, latent, in Cyrus IMAP to allow this feature. (If not, it goes on my way to long projects list---and I wouldn't mind if somebody beat me to it.) Have you tried taking a snapshot of your configuration directory and contants of /var/imap (e.g.). If you start cyrus on the snapshots what happens? I haven't tried that, because we had some problems with LVS snapshots. I haven't got a clue how cyrus reacts on read only filesystems. Maybe Ken can answer that. Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert [EMAIL PROTECTED] tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur Direction ICT, Infrastructure dept. Groep Systemen Systems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
