[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
This bug was fixed in the package dovecot - 1:2.3.7.2-1ubuntu3.5 --- dovecot (1:2.3.7.2-1ubuntu3.5) focal; urgency=medium * d/p/handle-unbounded-mime.patch: Fix crash during deinit when searching mails with non-ending MIME boundaries. (LP: #1912118) -- Bryce Harrington Sun, 11 Apr 2021 13:25:41 -0700 ** Changed in: dovecot (Ubuntu Focal) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Sure, thanks & done! ** Tags removed: verification-needed verification-needed-focal ** Tags added: verification-done verification-done-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Hello Bryce, sorry for the long wait. I've installed dovecot-core with your new build and can confirm: the bug is fixed :). Thank you very much for your help. Unfortunately I can not find the option do add "verification-done- focal". Can you perform this operation? Regards Timo -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Dave and Timo - can you please install dovecot from focal-proposed (as described above) and verify that the package update solves the search issue you encountered? Thanks ahead of time! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Hello Dave, or anyone else affected, Accepted dovecot into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dovecot/1:2.3.7.2-1ubuntu3.5 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-focal. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: dovecot (Ubuntu Focal) Status: In Progress => Fix Committed ** Tags added: verification-needed verification-needed-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
** Merge proposal linked: https://code.launchpad.net/~bryce/ubuntu/+source/dovecot/+git/dovecot/+merge/408909 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
** Changed in: dovecot (Ubuntu Focal) Status: Incomplete => In Progress ** Description changed: [Impact] Searching emails triggers a assertion failure if malformed emails are present in the mail box. [Test Case] - * TBD + With an INBOX containing a malformed email, perform a server-side search, such as: + + # doveadm index + # doveadm search -u ubuntu mailbox inbox body blah + + or + + # doveadm search -u ubuntu mailbox inbox body JPMorgan + + In an affected situation, this triggers an assertion failure in message-parser.c. + In an unaffected case, these searches will return one or more hash ids of email messages. [Where Problems Could Occur] This patch alters code involving iteration over a C linked list in relation to an assertion, so problems to look for would involve assertions, crashes or memory handling issues particularly associated with message handling functionality. Further, as this section of code involves message processing, issues related to searching, organizizing, or otherwise iterating through email messages may be of relevance. Mitigating the risk is that the patch's actual change is simply to add a pre-condition to a section of logic, meaning that any issues would probably be limited to situations where preparsed messages are being handled. [Original Report] I'm experiencing identical behaviour to Debian bug #970386 (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386) ... Can we please have this fix applied to dovecot-core 1:2.3.7.2-1ubuntu3? Searching is fairly broken atm. Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Hi Dave, Thanks for confirming the fix. We've had a second confirmation of the issue and fix by Timo, so although I've not been able to reproduce this myself, I'll proceed with rolling the fix out. I'll need to ask help from you and Timo to perform the validation when the SRU team requests it. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Hi Bryce, I've installed the PPA on a cloned image of the server in question, and can confirm that it fixes the problem. There is nothing non-stock in my environment. Thank you so much for working on this! It's greatly appreciated. Best regards Dave -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Hi Dave, thanks for getting back quickly. Yes there's a couple things that might help this move forward. Before that, this is what I see on my end: root@dovecot-sru-lp1912118-focal:/home/ubuntu# apt-cache policy dovecot-core dovecot-core: Installed: 1:2.3.7.2-1ubuntu3 Candidate: 1:2.3.7.2-1ubuntu3.3 Version table: 1:2.3.7.2-1ubuntu3.3 500 500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages *** 1:2.3.7.2-1ubuntu3 500 500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages 100 /var/lib/dpkg/status root@dovecot-sru-lp1912118-focal:/home/ubuntu# doveadm search -u ubuntu mailbox inbox body blah root@dovecot-sru-lp1912118-focal:/home/ubuntu# doveadm search -u ubuntu mailbox inbox body JPMorgan 193f6434a25b7f60548c01009e423642 1 root@dovecot-sru-lp1912118-focal:/home/ubuntu# systemctl status dovecot ● dovecot.service - Dovecot IMAP/POP3 email server Loaded: loaded (/lib/systemd/system/dovecot.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2021-05-04 23:01:36 UTC; 18min ago Docs: man:dovecot(1) http://wiki2.dovecot.org/ Main PID: 306 (dovecot) Tasks: 7 (limit: 77019) Memory: 7.2M CGroup: /system.slice/dovecot.service ├─306 /usr/sbin/dovecot -F ├─335 dovecot/anvil ├─336 dovecot/log ├─338 dovecot/config ├─503 dovecot/stats ├─526 dovecot/auth └─543 dovecot/auth -w May 04 23:01:36 dovecot-sru-lp1912118-focal systemd[1]: Started Dovecot IMAP/POP3 email server. May 04 23:01:36 dovecot-sru-lp1912118-focal dovecot[306]: master: Dovecot v2.3.7.2 (3c910f64b) starting up without an> root@dovecot-sru-lp1912118-focal:/home/ubuntu# The first option you could do to help is since you *can* reproduce it, would be to install the PPA I mentioned above and verify whether that package solves the problem on your end. Although I much prefer being able to reproduce issues myself, that would at least allow us to move ahead with the SRU to get the fix released publicly. The second option is if you could send me an inbox that reproduces it for you - not necessarily the one you have now, but maybe download the emails from the debian bug report and add them to your inbox. (I'm wondering if I constructed the inbox incorrectly.) The third thing is if you have anything non-stock in your dovecot config files, to attach them. Also if you have any 3rd party FTS software, or other dovecot-* components installed, let me know. That will help me ensure I'm more closely reproducing your environment. Thanks ahead of time. Bryce (br...@canonical.com) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Here's what I'm getting: doveadm search mailbox inbox body blah doveadm(dave): Panic: file message-parser.c: line 174 (message_part_finish): assertion failed: (ctx->nested_parts_count > 0) doveadm(dave): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xf0b2b) [0x7f2f562cfb2b] -> /usr/lib/dovecot/libdovecot.so.0(+0xf0b67) [0x7f2f562cfb67] -> /usr/lib/dovecot/libdovecot.so.0(+0x54bcf) [0x7f2f56233bcf] -> /usr/lib/dovecot/libdovecot.so.0(+0x51e31) [0x7f2f56230e31] -> /usr/lib/dovecot/libdovecot.so.0(message_parser_parse_next_block+0x104) [0x7f2f562b5304] -> /usr/lib/dovecot/libdovecot.so.0(message_search_msg+0xa0) [0x7f2f562b7700] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xd614e) [0x7f2f5646214e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_foreach+0x4d) [0x7f2f563e584d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xd6e98) [0x7f2f56462e98] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xd822c) [0x7f2f5646422c] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x11d) [0x7f2f5646498d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x24) [0x7f2f563ef114] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next+0x47) [0x7f2f563ef187] -> /usr/bin/doveadm(+0x3cf42) [0x56367fc8ff42] -> /usr/bin/doveadm(+0x33595) [0x56367fc86595] -> /usr/bin/doveadm(+0x34166) [0x56367fc87166] -> /usr/bin/doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x225) [0x56367fc87f05] -> /usr/bin/doveadm(doveadm_cmd_run_ver2+0x500) [0x56367fc983d0] -> /usr/bin/doveadm(doveadm_cmd_try_run_ver2+0x3e) [0x56367fc9842e] -> /usr/bin/doveadm(main+0x1d9) [0x56367fc77a69] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f2f55e6b0b3] -> /usr/bin/doveadm(_start+0x2e) [0x56367fc77efe] Aborted (core dumped) Short of sending you the inbox in question, is there any other info I can provide that would be helpful? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Hi Dave, Ubuntu focal is up to version 1:2.3.7.2-1ubuntu3.3 now, which includes fixes for malformed MIME patches, however does not yet have the fix from deb #970386. However, I'm also not able to reproduce the reported issue when downgrading to 1:2.3.7.2-1ubuntu3: dovecot-core: Installed: 1:2.3.7.2-1ubuntu3 Candidate: 1:2.3.7.2-1ubuntu3.3 Version table: 1:2.3.7.2-1ubuntu3.3 500 500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages *** 1:2.3.7.2-1ubuntu3 500 500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages 100 /var/lib/dpkg/status # doveadm search -u ubuntu BODY JPMorgan 193f6434a25b7f60548c01009e423642 1 I've tested SUBJECT and KEYWORD searches, as well as `doveadm index` but do not trigger the crash so far. Could you assist us in identifying a simple test case we can use to reproduce the issue and validate the fix? ** Changed in: dovecot (Ubuntu Focal) Status: Triaged => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
I've packaged this fix in PPA here: https://launchpad.net/~bryce/+archive/ubuntu/dovecot-sru-lp1912118 However, I'm not clear exactly how to reproduce the crash using the emails. I've downloaded them and stuck them in a user INBOX, but the debian bug report doesn't list the exact commands being used to execute the server-side search, but I gather a simple `doveadm index` isn't sufficient to trigger it? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
** Description changed: - I'm experiencing identical behaviour to Debian bug #970386 - (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386) ... Can we - please have this fix applied to dovecot-core 1:2.3.7.2-1ubuntu3? - Searching is fairly broken atm. + [Impact] + + Searching emails triggers a assertion failure if malformed emails are + present in the mail box. + + [Test Case] + * TBD + + [Where Problems Could Occur] + This patch alters code involving iteration over a C linked list in + relation to an assertion, so problems to look for would involve + assertions, crashes or memory handling issues particularly associated + with message handling functionality. Further, as this section of code + involves message processing, issues related to searching, organizizing, + or otherwise iterating through email messages may be of relevance. + + Mitigating the risk is that the patch's actual change is simply to add a + pre-condition to a section of logic, meaning that any issues would + probably be limited to situations where preparsed messages are being + handled. + + [Original Report] + I'm experiencing identical behaviour to Debian bug #970386 (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386) ... Can we please have this fix applied to dovecot-core 1:2.3.7.2-1ubuntu3? Searching is fairly broken atm. Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
The code for dovecot 2.2.33.2 currently in bionic appears to be significantly different from what's in focal. The code that this patch fixes doesn't seem to exist in the bionic codebase, at least not in the same form. Still worth checking, but it'll have to be done using a test case. ** Changed in: dovecot (Ubuntu Focal) Assignee: (unassigned) => Bryce Harrington (bryce) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Verified code changes are already present in hirsute's dovecot-gu. Should probably also check if affects bionic. There are sample emails in the debian bug report https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386, at comment #34. ** Changed in: dovecot (Ubuntu) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
** Changed in: dovecot (Debian) Status: Unknown => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
** Also affects: dovecot (Ubuntu Focal) Importance: Undecided Status: New ** Changed in: dovecot (Ubuntu Focal) Status: New => Triaged ** Changed in: dovecot (Ubuntu Focal) Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912118] Re: assertion failure in message_part_finish when searching large folder
Thank you for taking the time to report this bug and helping to make Ubuntu better. This is the upstream patch: https://github.com/dovecot/core/commit/a668d767a710ca18ab6e7177d8e8be22a6b024fb ** Changed in: dovecot (Ubuntu) Importance: Undecided => High ** Tags added: server-next ** Bug watch added: Debian Bug tracker #970386 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386 ** Also affects: dovecot (Debian) via https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386 Importance: Unknown Status: Unknown ** Changed in: dovecot (Ubuntu) Status: New => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912118 Title: assertion failure in message_part_finish when searching large folder To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/dovecot/+bug/1912118/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
