Anthony,I think this is a different error, just hitting the same assertion tripwire. I'm getting a problem in reconstruct(8), but your is showing up in imapd.
-nic
On 07/26/2018 04:16 AM, Anthony Prades via Cyrus-devel wrote:
Hi, Same problem here. Context: Upgrade cyrus from 2.4.20 to Cyrus 3.0.7.As our spool is very big, we plan to run reconstruct -V max in a second time as it seems to be possible from 2.5 (https://www.cyrusimap.org/2.5/imap/release-notes/2.5/x/2.5.0.html).After upgrade binaries, mailbox are available until we change a flag. After changing a flag, mailbox is unavailable with message:Jul 25 12:39:28 bluemind-debian cyrus/imap[9875]: Fatal error: Internal error: assertion failed: imap/message.c: 4286: !message_need(m, M_RECORD) Jul 25 12:39:28 bluemind-debian cyrus/master[8463]: process type:SERVICE name:imap path:/usr/lib/cyrus/imapd age:0.138s pid:9875 exited, status 75To reproduce: - create a new mailbox using Cyrus 2.4.20: user/ad...@apr-vmnet.loc - put at least 2 messages - using LMTP: # ls -l /var/spool/cyrus/data/mail/domain/a/apr-vmnet.loc/a/user/admin/ total 12 -rw------- 1 cyrus mail 1423 Jul 25 12:31 1. -rw------- 1 cyrus mail 1425 Jul 25 12:31 2. - upgrade to cyrus 3.0.7 - update flags on mail 1: # telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=PLAIN SASL-IR] server readyA1 LOGIN ad...@apr-vmnet.loc adminA1 OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SEARCH=FUZZY SORT SORT=MODSEQ SORT=DISPLAY SORT=UID THREAD=ORDEREDSUBJECT THREAD=REFERENCES THREAD=REFS ANNOTATEMORE ANNOTATE-EXPERIMENT-1 METADATA LIST-EXTENDED LIST-STATUS LIST-MYRIGHTS LIST-METADATA WITHIN QRESYNC SCAN XLIST XMOVE MOVE SPECIAL-USE CREATE-SPECIAL-USE DIGEST=SHA1 X-REPLICATION URLAUTH URLAUTH=BINARY LOGINDISABLED X-QUOTA=STORAGE X-QUOTA=MESSAGE X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE] User logged in SESSIONID=<cyrus-8408-1532514768-1-16031947181488935011>A2 SELECT INBOX * 2 EXISTS * 0 RECENT * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] Ok * OK [UNSEEN 1] Ok * OK [UIDVALIDITY 1532514648] Ok * OK [UIDNEXT 3] Ok * OK [HIGHESTMODSEQ 3] Ok * OK [URLMECH INTERNAL] Ok * OK [ANNOTATIONS 65536] Ok A2 OK [READ-WRITE] Completed A3 STORE 1 +FLAGS \Flagged * 1 FETCH (FLAGS (\Flagged)) A3 OK Completed A4 LOGOUT * BYE LOGOUT received A4 OK Completed Connection closed by foreign host. - select INBOX: # telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=PLAIN SASL-IR] server readyA1 LOGIN ad...@apr-vmnet.loc adminA1 OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SEARCH=FUZZY SORT SORT=MODSEQ SORT=DISPLAY SORT=UID THREAD=ORDEREDSUBJECT THREAD=REFERENCES THREAD=REFS ANNOTATEMORE ANNOTATE-EXPERIMENT-1 METADATA LIST-EXTENDED LIST-STATUS LIST-MYRIGHTS LIST-METADATA WITHIN QRESYNC SCAN XLIST XMOVE MOVE SPECIAL-USE CREATE-SPECIAL-USE DIGEST=SHA1 X-REPLICATION URLAUTH URLAUTH=BINARY LOGINDISABLED X-QUOTA=STORAGE X-QUOTA=MESSAGE X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE] User logged in SESSIONID=<cyrus-8409-1532514786-1-17619737167055332223>A2 SELECT INBOX* BYE Fatal error: Internal error: assertion failed: imap/message.c: 4286: !message_need(m, M_RECORD)Connection closed by foreign host.Running reconstruct -V max on this mailbox fix the problem but we loose mail 2 flags as it is rediscovered:# reconstruct -V max user/ad...@apr-vmnet.locapr-vmnet.loc!user.admin: update uniqueid from header (null) => 5d0161495b585158apr-vmnet.loc!user.admin uid 2 rediscovered - appending apr-vmnet.loc!user.admin updating quota_mailbox_used: 4273 => 2848 apr-vmnet.loc!user.admin: updating exists 3 => 2 apr-vmnet.loc!user.admin: updating sync_crc 3520663294 => 4247682740 user/ad...@apr-vmnet.loc Repacked user/ad...@apr-vmnet.loc to version 13 # ls -l /var/spool/cyrus/data/mail/domain/a/apr-vmnet.loc/a/user/admin/ total 16 -rwxr-x--- 1 cyrus mail 1423 Jul 25 12:31 1. -rwxr-x--- 1 cyrus mail 1425 Jul 25 12:31 3. Anthony On 07/24/2018 07:10 PM, Nic Bernstein wrote:Bron, et al.,Was this change ever cherry-picked to 3.0? I am seeing the same issue with recent 3.0 HEAD, but slightly different location:user.masked: updating sync_crc 521983118 => 503807715 fatal error: Internal error: assertion failed: imap/message.c: 4286: !message_need(m, M_RECORD)A git log of imap/message.c doesn't show a commit from 1/2/2017, and nothing affecting imap/message.c around that time seems to line up with this.Please advise, -nic On 01/02/2017 07:13 AM, Bron Gondwana via Cyrus-devel wrote:Thanks for the data. It was 8 bytes of zeros across a UID and INTERNALDATE in the cyrus.index file.I now have a fixed reconstruct which can detect and repair this rather than aborting, pushed to master. I also have a Cassandane testcase for this and a couple of other things that reconstruct does :)Bron. On Thu, 29 Dec 2016, at 09:45, Bron Gondwana via Cyrus-devel wrote:Wow, interesting. Are you willing to send me a tarball containing the spool folder including cyrus.index and cyrus.cache files as well as the email files themselves? I'll need your imapd.conf file as well :)Cheers, Bron. On Thu, 29 Dec 2016, at 00:28, Thomas Cataldo via Cyrus-devel wrote:Hi,Running a build of 3.0.0-beta6 I hit the following assertion on one of my test mailboxes after playing a bit with the replication stuff :root@bm1604:~# /usr/lib/cyrus/sbin/sync_client -n eclipse -o -u t...@ex2016.vmwFatal error: Internal error: assertion failed: imap/message.c: 4246: !message_need(m, M_RECORD)root@bm1604:~# cyradm -u admin0 localhost Password: localhost> version name : Cyrus IMAPD version : 3.0.0-beta6-3-gf721e5b vendor : Project Cyrus support-url: http://www.cyrusimap.org os : Linux os-version : 4.4.0-57-generic environment: Built w/Cyrus SASL 2.1.26 Running w/Cyrus SASL 2.1.26 Built w/OpenSSL 1.0.2g 1 Mar 2016 Running w/OpenSSL 1.0.2g 1 Mar 2016 Built w/zlib 1.2.8 Running w/zlib 1.2.8 CMU Sieve 2.4 mmap = shared lock = fcntl nonblock = ioctl idle = idled root@bm1604:~# telnet localhost 1143 Connected to localhost. Escape character is '^]'.* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE STARTTLS AUTH=PLAIN SASL-IR] server ready. login t...@ex2016.vmw xxxxxx. OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SEARCH=FUZZY SORT SORT=MODSEQ SORT=DISPLAY SORT=UID THREAD=ORDEREDSUBJECT THREAD=REFERENCES THREAD=REFS ANNOTATEMORE ANNOTATE-EXPERIMENT-1 METADATA LIST-EXTENDED LIST-STATUS LIST-MYRIGHTS LIST-METADATA WITHIN QRESYNC SCAN XLIST XMOVE MOVE SPECIAL-USE CREATE-SPECIAL-USE DIGEST=SHA1 X-REPLICATION URLAUTH URLAUTH=BINARY LOGINDISABLED COMPRESS=DEFLATE X-QUOTA=STORAGE X-QUOTA=MESSAGE X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE] User logged in SESSIONID=<cyrus-43098-1482931179-1-18150325414286070410>. select inbox* BYE Fatal error: Internal error: assertion failed: imap/message.c: 4246: !message_need(m, M_RECORD)Connection closed by foreign host. Trying to reconstruct the mailbox does not help :root@bm1604:~# /usr/lib/cyrus/sbin/reconstruct -rfxGROU t...@ex2016.vmwt...@ex2016.vmw The error is still here after that. Any idea ? Regards, Thomas.-- Bron Gondwana br...@fastmail.fm-- Bron Gondwana br...@fastmail.fm-- Nic bernstein...@onlight.com Onlight, Inc.www.onlight.com 6525 W Bluemound Road, Suite 24 v. 414.272.4477 Milwaukee, Wisconsin 53213-4073
-- Nic Bernstein n...@onlight.com Onlight, Inc. www.onlight.com 6525 W Bluemound Road, Suite 24 v. 414.272.4477 Milwaukee, Wisconsin 53213-4073
<<attachment: nic.vcf>>
