Some part of bug from 2.3.4 still in 2.3.5
Hi Ken There's still a serious bug in 2.3.5 that causes copied messages to be pseudo invisible in the moved to folder. I can reproduce it as follows: . select inbox.Archive * FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected medeleted hasatt) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected medeleted hasatt \*)] * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY 1148525506] * OK [UIDNEXT 185] * OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox . OK [READ-WRITE] Completed . uid search 1:* * SEARCH 84 . OK Completed (1 msgs in 0.000 secs) So Archive folder has 1 message. . select inbox * FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected hasatt NonJunk Junk medeleted) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected hasatt NonJunk Junk medeleted \*)] * 13 EXISTS * 0 RECENT * OK [UNSEEN 12] * OK [UIDVALIDITY 1148525506] * OK [UIDNEXT 246932] * OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox . OK [READ-WRITE] Completed . uid search 1:* * SEARCH 180517 219479 227085 227785 228034 233668 238794 245170 245693 246382 246402 246930 246931 . OK Completed (13 msgs in 0.000 secs) . uid copy 180517 inbox.Archive . OK [COPYUID 1148525506 180517 185] Completed So copy 1 message from inbox to Archive folder: . select inbox.Archive * FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected medeleted hasatt) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected medeleted hasatt \*)] * 2 EXISTS * 1 RECENT * OK [UIDVALIDITY 1148525506] * OK [UIDNEXT 186] * OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox . OK [READ-WRITE] Completed . uid search 1:* * SEARCH 84 185 . OK Completed (2 msgs in 0.000 secs) Message appears in search index. . uid fetch 185 rfc822 . OK Completed (0.000 sec) Any attempt to retrieve message body data returns nothing :( Rob Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Some part of bug from 2.3.4 still in 2.3.5
I suspect what you're looking for is in the patch attached
to this message.
You forgot (entirely) to put modseq code in the append_copy
path. Since modseq appears to be a property of the target
folder rather than the message itself, I'm using the same
code as the 2.3.4-2.3.5 diff put into the other two append
pathways.
I've rolled this out on to the server we're testing on and
I've found that copy/move works. Now to go reconstruct all
the folders on that machine!
Bron.
On Thu, 25 May 2006 16:50:05 +1000, Robert Mueller [EMAIL PROTECTED] said:
Hi Ken
There's still a serious bug in 2.3.5 that causes copied messages to be
pseudo invisible in the moved to folder. I can reproduce it as follows:
. select inbox.Archive
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
medeleted hasatt)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt
selected medeleted hasatt \*)]
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1148525506]
* OK [UIDNEXT 185]
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
. OK [READ-WRITE] Completed
. uid search 1:*
* SEARCH 84
. OK Completed (1 msgs in 0.000 secs)
So Archive folder has 1 message.
. select inbox
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
hasatt
NonJunk Junk medeleted)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt
selected hasatt NonJunk Junk medeleted \*)]
* 13 EXISTS
* 0 RECENT
* OK [UNSEEN 12]
* OK [UIDVALIDITY 1148525506]
* OK [UIDNEXT 246932]
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
. OK [READ-WRITE] Completed
. uid search 1:*
* SEARCH 180517 219479 227085 227785 228034 233668 238794 245170 245693
246382 246402 246930 246931
. OK Completed (13 msgs in 0.000 secs)
. uid copy 180517 inbox.Archive
. OK [COPYUID 1148525506 180517 185] Completed
So copy 1 message from inbox to Archive folder:
. select inbox.Archive
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
medeleted hasatt)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt
selected medeleted hasatt \*)]
* 2 EXISTS
* 1 RECENT
* OK [UIDVALIDITY 1148525506]
* OK [UIDNEXT 186]
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
. OK [READ-WRITE] Completed
. uid search 1:*
* SEARCH 84 185
. OK Completed (2 msgs in 0.000 secs)
Message appears in search index.
. uid fetch 185 rfc822
. OK Completed (0.000 sec)
Any attempt to retrieve message body data returns nothing :(
Rob
--
Bron Gondwana
[EMAIL PROTECTED]
diff -ur cyrus-imapd-2.3.5/imap/append.c cyrus-imapd-2.3.5-modseq/imap/append.c
--- cyrus-imapd-2.3.5/imap/append.c 2006-05-24 14:37:42.0 -0400
+++ cyrus-imapd-2.3.5-modseq/imap/append.c 2006-05-25 03:00:36.0
-0400
@@ -857,6 +857,11 @@
for (msg = 0; msg nummsg; msg++) {
zero_index(message_index[msg]);
message_index[msg].uid = append_mailbox-last_uid + 1 + as-nummsg;
+ if (append_mailbox-options OPT_IMAP_CONDSTORE) {
+ message_index[msg].modseq = append_mailbox-highestmodseq + 1;
+ } else {
+ message_index[msg].modseq = 1;
+ }
message_index[msg].last_updated = time(0);
message_index[msg].internaldate = copymsg[msg].internaldate;
as-nummsg++;
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Can't locate object method reinit via package Pod::Man
On Wed, 2006-05-24 at 19:57 -0800, Noah wrote: freeBSD-4.11 I am finding that cyrus-imapd23 fails to build from FreeBSD /usr/ports . Clues please? Obviously your /usr/local/bin/perl doesn't know the reinit method. Looks like screwed perl 5.8.8 install. Do you prepare new machine. Consider going with newer FreeBSD if you can. cyrus-imapd23 port in 5.4 works ( compiles at least). And yes, the error has nothing to do with cyrus, so you shouldn't post it here in theory. M. -- Miroslaw Psyborg Jaworski GCS/IT d- s+:+ a C++$ UBI$ P+++$ L- E--- W++(+++)$ N++ o+ K- w-- O- M- V- PS+ PE++ Y+ PGP t 5? X+ R++ !tv b++(+++) DI++ D+ G e* h++ r+++ y? Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Some part of bug from 2.3.4 still in 2.3.5
And again...
[this message is brought to you by the my god, I'm coding in C
again, I'm too old for this shit department]
... it appears reconstruct just blindly re-applied the value of
modseq from the already existing entry without checking if it
was valid.
The attached patch contains my previous patch to fix the
append_copy code path along with an additional check in
reconstruct which will set modseq to 1 if it doesn't
already have a value. Since modseq always starts at 1 and
is increased each operation, I think that's OK for people
running in the CONDSTORE universe as well. It's certainly
fine for those of us where modseq will always be 1.
Bron.
On Thu, 25 May 2006 17:16:17 +1000, Bron Gondwana [EMAIL PROTECTED] said:
I suspect what you're looking for is in the patch attached
to this message.
You forgot (entirely) to put modseq code in the append_copy
path. Since modseq appears to be a property of the target
folder rather than the message itself, I'm using the same
code as the 2.3.4-2.3.5 diff put into the other two append
pathways.
I've rolled this out on to the server we're testing on and
I've found that copy/move works. Now to go reconstruct all
the folders on that machine!
Bron.
On Thu, 25 May 2006 16:50:05 +1000, Robert Mueller [EMAIL PROTECTED]
said:
Hi Ken
There's still a serious bug in 2.3.5 that causes copied messages to be
pseudo invisible in the moved to folder. I can reproduce it as follows:
. select inbox.Archive
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
medeleted hasatt)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt
selected medeleted hasatt \*)]
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1148525506]
* OK [UIDNEXT 185]
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
. OK [READ-WRITE] Completed
. uid search 1:*
* SEARCH 84
. OK Completed (1 msgs in 0.000 secs)
So Archive folder has 1 message.
. select inbox
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
hasatt
NonJunk Junk medeleted)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt
selected hasatt NonJunk Junk medeleted \*)]
* 13 EXISTS
* 0 RECENT
* OK [UNSEEN 12]
* OK [UIDVALIDITY 1148525506]
* OK [UIDNEXT 246932]
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
. OK [READ-WRITE] Completed
. uid search 1:*
* SEARCH 180517 219479 227085 227785 228034 233668 238794 245170 245693
246382 246402 246930 246931
. OK Completed (13 msgs in 0.000 secs)
. uid copy 180517 inbox.Archive
. OK [COPYUID 1148525506 180517 185] Completed
So copy 1 message from inbox to Archive folder:
. select inbox.Archive
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
medeleted hasatt)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt
selected medeleted hasatt \*)]
* 2 EXISTS
* 1 RECENT
* OK [UIDVALIDITY 1148525506]
* OK [UIDNEXT 186]
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
. OK [READ-WRITE] Completed
. uid search 1:*
* SEARCH 84 185
. OK Completed (2 msgs in 0.000 secs)
Message appears in search index.
. uid fetch 185 rfc822
. OK Completed (0.000 sec)
Any attempt to retrieve message body data returns nothing :(
Rob
--
Bron Gondwana
[EMAIL PROTECTED]
--
Bron Gondwana
[EMAIL PROTECTED]
diff -ur cyrus-imapd-2.3.5/imap/append.c cyrus-imapd-2.3.5-modseq2/imap/append.c
--- cyrus-imapd-2.3.5/imap/append.c 2006-05-24 14:37:42.0 -0400
+++ cyrus-imapd-2.3.5-modseq2/imap/append.c 2006-05-25 03:39:14.0
-0400
@@ -857,6 +857,11 @@
for (msg = 0; msg nummsg; msg++) {
zero_index(message_index[msg]);
message_index[msg].uid = append_mailbox-last_uid + 1 + as-nummsg;
+ if (append_mailbox-options OPT_IMAP_CONDSTORE) {
+ message_index[msg].modseq = append_mailbox-highestmodseq + 1;
+ } else {
+ message_index[msg].modseq = 1;
+ }
message_index[msg].last_updated = time(0);
message_index[msg].internaldate = copymsg[msg].internaldate;
as-nummsg++;
diff -ur cyrus-imapd-2.3.5/imap/reconstruct.c
cyrus-imapd-2.3.5-modseq2/imap/reconstruct.c
--- cyrus-imapd-2.3.5/imap/reconstruct.c2006-03-31 14:22:27.0
-0500
+++ cyrus-imapd-2.3.5-modseq2/imap/reconstruct.c2006-05-25
03:39:14.0 -0400
@@ -714,6 +714,10 @@
/* Use data in old index file, subject to validity checks */
message_index.internaldate = old_index.internaldate;
message_index.modseq = old_index.modseq;
+ /* This should never happen, but bugs in 2.3.4 and 2.3.5
+* could have left modseq blank. If so, update it */
+ if (!message_index.modseq)
+ message_index.modseq = 1;
Re: Cyrus IMAPd 2.3.4 Released
On Wed, 24 May 2006 12:44:42 -0400, Ken Murchison [EMAIL PROTECTED] said: It looks like in my haste to make a release with the pop3d bugfix, it didn't do enough testing. I'm seeing the same thing here,and I'm working on it. Is this a good spot to suggest branching from the last stable release (2.3.3 in this case) and just applying the bugfixes if HEAD is only half finished? I know branching sucks in CVS (which is why we've been a subversion shop for about 6 months now), but it's better than the mad scramble we've had the past couple of days due to incomplete features meaning corrupt indexes in a release. Bron ( who shouldn't have blindly gone ahead and rolled out the latest code either, even just to one server... ) -- Bron Gondwana [EMAIL PROTECTED] Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Some part of bug from 2.3.4 still in 2.3.5
And again...
[Rob suggested this bit might hit some filters, so I'm posting
again with it out]
... it appears reconstruct just blindly re-applied the value of
modseq from the already existing entry without checking if it
was valid.
The attached patch contains my previous patch to fix the
append_copy code path along with an additional check in
reconstruct which will set modseq to 1 if it doesn't
already have a value. Since modseq always starts at 1 and
is increased each operation, I think that's OK for people
running in the CONDSTORE universe as well. It's certainly
fine for those of us where modseq will always be 1.
Bron.
On Thu, 25 May 2006 17:16:17 +1000, Bron Gondwana [EMAIL PROTECTED]
said:
I suspect what you're looking for is in the patch attached
to this message.
You forgot (entirely) to put modseq code in the append_copy
path. Since modseq appears to be a property of the target
folder rather than the message itself, I'm using the same
code as the 2.3.4-2.3.5 diff put into the other two append
pathways.
I've rolled this out on to the server we're testing on and
I've found that copy/move works. Now to go reconstruct all
the folders on that machine!
Bron.
On Thu, 25 May 2006 16:50:05 +1000, Robert Mueller [EMAIL PROTECTED]
said:
Hi Ken
There's still a serious bug in 2.3.5 that causes copied messages to be
pseudo invisible in the moved to folder. I can reproduce it as follows:
. select inbox.Archive
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
medeleted hasatt)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt
selected medeleted hasatt \*)]
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1148525506]
* OK [UIDNEXT 185]
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
. OK [READ-WRITE] Completed
. uid search 1:*
* SEARCH 84
. OK Completed (1 msgs in 0.000 secs)
So Archive folder has 1 message.
. select inbox
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
hasatt
NonJunk Junk medeleted)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt
selected hasatt NonJunk Junk medeleted \*)]
* 13 EXISTS
* 0 RECENT
* OK [UNSEEN 12]
* OK [UIDVALIDITY 1148525506]
* OK [UIDNEXT 246932]
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
. OK [READ-WRITE] Completed
. uid search 1:*
* SEARCH 180517 219479 227085 227785 228034 233668 238794 245170 245693
246382 246402 246930 246931
. OK Completed (13 msgs in 0.000 secs)
. uid copy 180517 inbox.Archive
. OK [COPYUID 1148525506 180517 185] Completed
So copy 1 message from inbox to Archive folder:
. select inbox.Archive
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
medeleted hasatt)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt
selected medeleted hasatt \*)]
* 2 EXISTS
* 1 RECENT
* OK [UIDVALIDITY 1148525506]
* OK [UIDNEXT 186]
* OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
. OK [READ-WRITE] Completed
. uid search 1:*
* SEARCH 84 185
. OK Completed (2 msgs in 0.000 secs)
Message appears in search index.
. uid fetch 185 rfc822
. OK Completed (0.000 sec)
Any attempt to retrieve message body data returns nothing :(
Rob
--
Bron Gondwana
[EMAIL PROTECTED]
--
Bron Gondwana
[EMAIL PROTECTED]
--
Bron Gondwana
[EMAIL PROTECTED]
diff -ur cyrus-imapd-2.3.5/imap/append.c cyrus-imapd-2.3.5-modseq2/imap/append.c
--- cyrus-imapd-2.3.5/imap/append.c 2006-05-24 14:37:42.0 -0400
+++ cyrus-imapd-2.3.5-modseq2/imap/append.c 2006-05-25 03:39:14.0
-0400
@@ -857,6 +857,11 @@
for (msg = 0; msg nummsg; msg++) {
zero_index(message_index[msg]);
message_index[msg].uid = append_mailbox-last_uid + 1 + as-nummsg;
+ if (append_mailbox-options OPT_IMAP_CONDSTORE) {
+ message_index[msg].modseq = append_mailbox-highestmodseq + 1;
+ } else {
+ message_index[msg].modseq = 1;
+ }
message_index[msg].last_updated = time(0);
message_index[msg].internaldate = copymsg[msg].internaldate;
as-nummsg++;
diff -ur cyrus-imapd-2.3.5/imap/reconstruct.c
cyrus-imapd-2.3.5-modseq2/imap/reconstruct.c
--- cyrus-imapd-2.3.5/imap/reconstruct.c2006-03-31 14:22:27.0
-0500
+++ cyrus-imapd-2.3.5-modseq2/imap/reconstruct.c2006-05-25
03:39:14.0 -0400
@@ -714,6 +714,10 @@
/* Use data in old index file, subject to validity checks */
message_index.internaldate = old_index.internaldate;
message_index.modseq = old_index.modseq;
+ /* This should never happen, but bugs in 2.3.4 and 2.3.5
+* could have left modseq blank. If so, update it */
+ if (!message_index.modseq)
+ message_index.modseq = 1;
Re: Cyrus IMAPd 2.3.4 Released
On Thu, 2006-05-25 at 17:56 +1000, Bron Gondwana wrote: On Wed, 24 May 2006 12:44:42 -0400, Ken Murchison [EMAIL PROTECTED] said: It looks like in my haste to make a release with the pop3d bugfix, it didn't do enough testing. I'm seeing the same thing here,and I'm working on it. Is this a good spot to suggest branching from the last stable release (2.3.3 in this case) and just applying the bugfixes if HEAD is only half finished? 2.3 is beta branch. You want stable you go for 2.2. M. -- Miroslaw Psyborg Jaworski GCS/IT d- s+:+ a C++$ UBI$ P+++$ L- E--- W++(+++)$ N++ o+ K- w-- O- M- V- PS+ PE++ Y+ PGP t 5? X+ R++ !tv b++(+++) DI++ D+ G e* h++ r+++ y? Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.4 Released
Is this a good spot to suggest branching from the last stable release (2.3.3 in this case) and just applying the bugfixes if HEAD is only half finished? 2.3 is beta branch. You want stable you go for 2.2. Beta is may be, but still 1. There's no regression testing with cyrus at all. I did try and start a cyrus regression test a while back (just a perl script to test basic IMAP functionality) but there wasn't really interest in taking it up. I still strongly believe that some form of basic regression test that is built up more and more over time is is important. 2. This bug is something that would have turned up with 5 minutes of basic functionality testing (copying a message from one folder to another is broken) 3. 2.3.3 was working really well, and has been working well for months. A serious security issue was found so 2.3.4 was released. 2.3.4 was totally broken. 4. The reason 2.3.4 was totally brokwn was because significant functionality was added from 2.3.3 - 2.3.4 (CONDSTORE stuff). It's a hard compromise between releasing new features regularly and keeping a stable branch. Maybe we need a 2.3.x-rc1 type scheme before an official release so that the obvious things get caught before an actual point release? Or maybe a 2.3.3.1 release for urgent security/minor bug fixes? Just throwing things in the wind, seeing if anything sticks. Maybe this is just a more one off type situation, and a rethinking of release strategy is not needed. Rob Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
how to recover imap mail/ dir from old server, import into new server
Hi there, First time posting here so apologies if this question has (most likely) been asked before. have looked in the archives of this list but can't find it. My Cyrus Imapd mail server has crashed (hardware error). I have access to the hard drives but it has taken me some time to buy a new server and install a new cyrus imapd. Basically, I've got my previous server's /var/spool/cyrus/mail/ directory with all it's subfolders (s/user/stephen etc) and I'd like to import the mail files in there (1. 2. 3. etc) into my new imapd's mailstore. I've read and experimented that it's not a case of just dump the mails in the appropriate folder, since a mail has to go through the proper channels, be registered in cyrus.index etc so that my mail client knows which headers are available, which files they relate to etc. Is there a tool available for this purpose please? thanks for any and all help Steve Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.4 Released
Robert Mueller wrote: Is this a good spot to suggest branching from the last stable release (2.3.3 in this case) and just applying the bugfixes if HEAD is only half finished? 2.3 is beta branch. You want stable you go for 2.2. Beta is may be, but still 1. There's no regression testing with cyrus at all. I did try and start a cyrus regression test a while back (just a perl script to test basic IMAP functionality) but there wasn't really interest in taking it up. I still strongly believe that some form of basic regression test that is built up more and more over time is is important. 2. This bug is something that would have turned up with 5 minutes of basic functionality testing (copying a message from one folder to another is broken) 3. 2.3.3 was working really well, and has been working well for months. A serious security issue was found so 2.3.4 was released. 2.3.4 was totally broken. 4. The reason 2.3.4 was totally brokwn was because significant functionality was added from 2.3.3 - 2.3.4 (CONDSTORE stuff). It's a hard compromise between releasing new features regularly and keeping a stable branch. Maybe we need a 2.3.x-rc1 type scheme before an official release so that the obvious things get caught before an actual point release? Or maybe a 2.3.3.1 release for urgent security/minor bug fixes? Just throwing things in the wind, seeing if anything sticks. Maybe this is just a more one off type situation, and a rethinking of release strategy is not needed. This hole mess is a result of me having my head up my ass. I usually do pretty thorough testing of any code that I touch, and I *thought* that I had done that with the CONDSTORE stuff. Its now obvious that I didn't do enough testing of the non-CONDSTORE case. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Some part of bug from 2.3.4 still in 2.3.5
Robert Mueller wrote: Hi Ken There's still a serious bug in 2.3.5 that causes copied messages to be pseudo invisible in the moved to folder. I can reproduce it as follows: Have you uncovered any other bugs that need to be fixed before I embarrass myself again by making a premature 2.3.6 release? -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.4 Released
On Thu, 25 May 2006 08:43:46 -0400, Ken Murchison [EMAIL PROTECTED] said: This hole mess is a result of me having my head up my ass. I usually do pretty thorough testing of any code that I touch, and I *thought* that I had done that with the CONDSTORE stuff. Its now obvious that I didn't do enough testing of the non-CONDSTORE case. Happens to us all :( Of course, as Rob suggested, a regression test suite means you don't _have_ to think so much about things that worked in the past. Not entirely so, because your test suite is unlikely to cover every use case, but it should help a lot. We're a bit in the same boat - far too much of our system doesn't have test suites, and even when we do test stuff there's inevitably something different about production. At least we've moved to a relatively stable Debian based infrastructure now where you can get pretty much an entire server running with 'apt-get install fastmail-base; svn co $URL; make -C conf install'. Not quite there yet, but that's the goal. Means we're testing in an environment that's a much like production as we can make it and things don't get littered with as many ad-hoc changes. I hope my emergency patch is useful to you. I seems to have solved the obvious symptoms for us. Of course you probably know better than me if there's anywhere else that's generating index files that needs updating to set that struct member. Possibly a safer approach (bit late now, I know) is to have an init_index function which is _always_ called on one of those structs _before_ you do anything else to it, and then when you add a member to the struct you only need to do the default case in one place (new_index.modseq = 1;). That would have avoided the need to litter the code with bits of initialiser everywhere. There's my object-oriented training talking. We've moved most of our staff accounts (including me!) onto the server that is running 2.3.5+patches so we can eat our own dogfood. Mmm dogfood. Should help us see if anything else is strange. (now I just have to figure things out so that when we move users we retain the UIDVALIDITY so my entire offlineimap store doesn't get stale next time) Bron. -- Bron Gondwana [EMAIL PROTECTED] Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Some part of bug from 2.3.4 still in 2.3.5
On Thu, 25 May 2006 09:00:36 -0400, Ken Murchison [EMAIL PROTECTED] said: Robert Mueller wrote: Hi Ken There's still a serious bug in 2.3.5 that causes copied messages to be pseudo invisible in the moved to folder. I can reproduce it as follows: Have you uncovered any other bugs that need to be fixed before I embarrass myself again by making a premature 2.3.6 release? Not yet! The second patch I made to reconstruct seems to have made it clean up the mess correctly in the non CONDSTORE case. I can't speak for CONDSTORE users, but it looks to me like it does the right thing in that case too - because modseq should always have a non-zero value, and if it doesn't then the value of highestmodseq is probably 1 anyway, so 1 is still a better answer than 0. Release early, release often - isn't that what they say. Numbers are cheap. Bron. -- Bron Gondwana [EMAIL PROTECTED] Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: 2.2 murder backend authentication problems
Andrew Morgan wrote: On Thu, 18 May 2006, Brenden Conte wrote: I've been wrestling with a new cyrus murder setup. At this point, edits made to a backend server are properly propigated through the master and frontends. Now, i'm trying to test administration through the frontend servers. For example, when i connect to a frontend server and issue a `CREATE user.testuser imap-backend` the frontend logs show the following: In my testing, I had to create mailboxes on the backend. Maybe someone else can confirm this, but I don't think you can create mailboxes from a frontend. Andy Can anyone definitively confirm this? I get permission denied when i try, so i want to make sure this is by design and not an actual permissions problem. P.S. Thank you to everyone that responded to this thread! -- Brenden Conte System Programmer, CMT.CIO Rensselaer Polytechnic Institute (518)276-4264 Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.4 Released
Patrick Radtke wrote: On May 25, 2006, at 6:00 AM, Robert Mueller wrote: 1. There's no regression testing with cyrus at all. I did try and start a cyrus regression test a while back (just a perl script to test basic IMAP functionality) but there wasn't really interest in taking it up. I still strongly believe that some form of basic regression test that is built up more and more over time is is important. I'm interested in a nice regression suite. I've been doing some OpenLDAP stuff recently and they have (what seems like) 2 hours worth of tests that occur once you build the software. For a long time, I've been meaning to look at how they do it and see if a similar approach could be used with Cyrus IMAPd, but other projects always take a priority. The nice thing about regression testing is that we can start small. If there was some agreed upon methodology then when anyone (e.g. Ken, someone submitting a patch, someone reporting a bug, etc) can create a tests to show how a bug gets triggered or that new code/patch doesn't introduce bugs. I agree that a test suite is something that we need. I'll add this to our TODO list. As always, code contributions are always welcome. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.4 Released
On May 25, 2006, at 6:00 AM, Robert Mueller wrote:1. There's no regression testing with cyrus at all. I did try and start a cyrus regression test a while back (just a perl script to test basic IMAP functionality) but there wasn't really interest in taking it up. I still strongly believe that some form of basic regression test that is built up more and more over time is is important. I'm interested in a nice regression suite.I've been doing some OpenLDAP stuff recently and they have (what seems like) 2 hours worth of tests that occur once you build the software.For a long time, I've been meaning to look at how they do it and see if a similar approach could be used with Cyrus IMAPd, but other projects always take a priority.The nice thing about regression testing is that we can start small. If there was some agreed upon methodology then when anyone (e.g. Ken, someone submitting a patch, someone reporting a bug, etc) can create a tests to show how a bug gets triggered or that new code/patch doesn't introduce bugs.-Patrick Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
move /var/imap
Hello I want to move /var/imap to a new drive but while I'm doing this FreeBSD 5.4 could not move sockets (...yes I stopped before all daemons). How can I deal with that? Any hints are welcom. Thank you in advance. -- Martin Schweizer [EMAIL PROTECTED] Fax: +1 619 3300587 Tel.: +1 619 3300597 (VoIP) Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Some part of bug from 2.3.4 still in 2.3.5
Hi, On Thu, 25 May 2006 18:00:00 +1000 Bron Gondwana [EMAIL PROTECTED] said: brong ... it appears reconstruct just blindly re-applied the value of brong modseq from the already existing entry without checking if it brong was valid. brong The attached patch contains my previous patch to fix the brong append_copy code path along with an additional check in brong reconstruct which will set modseq to 1 if it doesn't brong already have a value. Since modseq always starts at 1 and brong is increased each operation, I think that's OK for people brong running in the CONDSTORE universe as well. It's certainly brong fine for those of us where modseq will always be 1. I met this problem, too, and your patch solved the problem. Thank you! Sincerely, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan [EMAIL PROTECTED] [EMAIL PROTECTED],jp.}FreeBSD.org http://www.imasy.org/~ume/ Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.4 Released
I agree that a test suite is something that we need. I'll add this to
our TODO list. As always, code contributions are always welcome.
See below and attached, a copy of what I sent april '04. It's not great, but
it's a start...
Rob
Hi Ken Rob
Given that there's currently no regression test for cyrus, I thought it
might be worth creating one. The basic idea being that the tests are kept up
to date, and any new features get a basic set of tests. Also anytime a bug
is discovered, a test is added that shows the bug, and left to make sure it
doesn't come back in the future. We're willing to fund the starting of this
project (geared towards the most recent patches for our needs), but hope CMU
will keep it up to date in the future as well. I'm not quite sure how any of
this would work (both funding and maintenance wise), but maybe you can make
some suggestions?
Anyway, I think there should be a generic imap component, and cyrus specific
component, in that it does some tests that rely on cyrus and/or it's
utilities (eg check location of spool directory and meta files, corrupt a
mailbox and run reconstruct, check quota and quota fixing, etc)
I think it's probably best to write the script in perl, and here's a couple
of modules that should help:
http://search.cpan.org/~robm/Mail-IMAPTalk-1.00/
http://search.cpan.org/~lhoward/Net-LMTP-0.02/
I tend to like creating two main types of tests:
1. A very structured set that test very specific set of actions. I tend to
make it so each 'batch' of tests is a function and is independent, and that
it leaves everything exactly how it was beforehand so you can easily run the
same test suite over and over, or a particular test function. eg
sub TestBasicFolder {
$Imap-create('user.test1')
|| die Could not create user.test1;
my $ACL = $Imap-getacl('user.test1')
|| die Could not get user.test1 ACL 1;
$Imap-setacl('user.test1', 'admin', 'lrswipcda')
|| die Could not set user.test1 ACL;
$ACL = $Imap-getacl('user.test1')
|| die Could not get user.test1 ACL 2;
... some more tests ...
$Imap-delete('user.test1')
|| die Could not delete user.test1;
}
2. Ones that pick a bunch of random commands from a set and executes them,
somehow keeping internally what it expects the state to be and making sure
that it is. eg something like:
sub TestRandomFolderCreateDelete {
my @Folders;
for (1 .. 10) {
my $Rand = rand();
# Create a random folder
if ($Rand 0.7) {
my $Folder = user. . RandomFolderName(int(rand(20)));
$Imap-create()
|| die Could not create folder $Folder: $@;
push @Folders, $Folder;
# Delete a random folder
} elsif ($Rand 0.9) {
my $Folder = splice @Folders, int(rand(@Folders));, 1;
$Imap-delete($Folder)
|| die Could not delete folder $Folder: $@;
# Check mailbox list
} else {
my %List = map { $_-[2] = 1 } $Imap-list('*', '*');
for (@Folders) {
delete $List{$_} || die Folder $_ not found;
}
!%List || die Not all folders found;
}
}
for (@Folders) {
$Imap-delete($_)
|| die Could not delete folder $Folder: $@;
}
my @List = $Imap-list('*', '*');
[EMAIL PROTECTED] || die Unexpectedly found folders;
}
I find tests like this quite important to find the more obscure bugs. Once
it does find a bug, I then find that I try differnet values of srand() at
the start (which then makes rand generate the same sequence each time the
program is run) until the bug is reproducible so I can track it down.
So basic things I can think of:
1. Login as 'administrator' user and create a bunch of user.* mailboxes and
sub-folders and listing
2. Test deleting some folders (eg test problem of deleting user.adminxxx
folder name) and listing
3. Set/get/myrights tests ACL's on the mailboxes (eg try read only, post
only, sharing other users folder, etc)
4. Set quotas on the mailboxes (different quota roots, etc)
5. Login as a user for one of the mailboxes. Test different ACL's set above
(eg try read only, post only, sharing other users folder, etc)
6. Appending messages to a mailbox
7. Reading messages (and seen flag)
8. Setting/getting other flags
9. Delivering email via LMTP (and of course + addressing)
10. LMTP new wildcard addressing
11. POP access to a mailbox
12. POP access to sub-folder
13. Group ACLs
14. Sieve actions
A couple of tricky things:
1. User logins. Since these are separate to the actual mailbox names, I'm
not sure how to easily handle this... maybe force usage of a saslauthd
daemon that always authorises all username/password combinations...
2. imapd.conf will need to be changed and then the server restarted from the
code
To get the ball rolling, I looked at some existing code I had, and started
the outlines of how I think things might work. I've attached the script,
which currently has 3 tests which I've just tested at home and verified that
it works.
Rob
cyrus_regression_test.pl
Description: Binary data
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
Cyrus IMAPd 2.3.6 Released
I am pleased (embarrassed) to announce the release of Cyrus IMAPd 2.3.6. This is a BETA-quality release, reflecting that it has significant numbers of new features that have not been tested on a wide-scale basis, although earlier versions of this code have been running at several sites for quite some time. This release fixes a bug in 2.3.[45] which caused copied messages to not be displayed by clients. My apologies to all of the early adopters that have been inconvenienced by this bug. Mailboxes that had messages copied into them by 2.3.[45] imap will need to be reconstructed. For full details, please see doc/changes.html and doc/install-upgrade.html which are included in the distribution. URLs for this release: ftp://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.6.tar.gz or http://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.6.tar.gz Questions and comments can be directed to info-cyrus@lists.andrew.cmu.edu (public list), or [EMAIL PROTECTED] -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: lmtpproxyd reappears in 2.3.4
Henrique de Moraes Holschuh wrote: On Wed, 24 May 2006, Simon Matter wrote: nobody need them in 2.3.3? So my question is, do those binaries act different depending on how they were called? If that's the case then the rpms should also ship with the hardlinked files. I sure hope those hardlinks are just there to keep people happy. If they are needed, something is wrong. If the binaries are acting differently depending on how they are called, this is an extremely bad idea IMHO (give us a --proxy switch or somesuch, instead!). No difference in functionality. They are just there for old cyrus.conf files. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.6 Released
Will this version seemlessly upgrade from 2.3.3 or will a reconstruct have to be run on each mailbox? Lenny -- Wisdom is to a man an infinite Treasure - Anonymous Quoting Ken Murchison [EMAIL PROTECTED]: I am pleased (embarrassed) to announce the release of Cyrus IMAPd 2.3.6. This is a BETA-quality release, reflecting that it has significant numbers of new features that have not been tested on a wide-scale basis, although earlier versions of this code have been running at several sites for quite some time. This release fixes a bug in 2.3.[45] which caused copied messages to not be displayed by clients. My apologies to all of the early adopters that have been inconvenienced by this bug. Mailboxes that had messages copied into them by 2.3.[45] imap will need to be reconstructed. For full details, please see doc/changes.html and doc/install-upgrade.html which are included in the distribution. URLs for this release: ftp://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.6.tar.gz or http://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.6.tar.gz Questions and comments can be directed to info-cyrus@lists.andrew.cmu.edu (public list), or [EMAIL PROTECTED] -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
