Hello community, here is the log from the commit of package dovecot23 for openSUSE:Leap:15.2 checked in at 2020-05-23 16:06:53 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.2/dovecot23 (Old) and /work/SRC/openSUSE:Leap:15.2/.dovecot23.new.2738 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "dovecot23" Sat May 23 16:06:53 2020 rev:28 rq:808119 version:2.3.10 Changes: -------- --- /work/SRC/openSUSE:Leap:15.2/dovecot23/dovecot23.changes 2020-01-15 14:52:57.257507334 +0100 +++ /work/SRC/openSUSE:Leap:15.2/.dovecot23.new.2738/dovecot23.changes 2020-05-23 16:07:04.557008790 +0200 @@ -1,0 +2,474 @@ +Fri May 15 16:08:49 UTC 2020 - Peter Varkoly <[email protected]> + +- update pigeonhole to 0.5.10 + * imap_sieve_filter: Change result action logging to include IMAP UID + - vacation: Addresses were compared case-sensitively. + + Added events for Sieve and ManageSieve, see + https://doc.dovecot.org/admin_manual/list_of_events/#pigeonhole + + Pigeonhole: Implement the Sieve "special-use" extension described in + RFC 8579. + - duplicate: Test only compared the handles which would cause + different values to be cached as the same duplicate test. Fix to also + compare the actual hashes. + - imap_sieve_filter: IMAP FILTER Command had various bugs in error + handling. Errors may have been duplicated for each email, errors + may have been missing entirely, command tag and ERRORS/WARNINGS + parameters were swapped. + - Sieve may leak resources in rare cases when a redirect, vacation or + report action fails to send the message. This mainly applies when + Sieve is executed in IMAP context; i.e., for the IMAPSIEVE or + FILTER=SIEVE capabilities. + - dsync: Sieve script syncing failed if mailbox attributes weren't + enabled. + + vacation: Made the subject for the automatic response message produced + by the Sieve vacation action configurable. Both the default subject + (if the script defines none) and the subject template (e.g. used to + add a subject prefix) can be configured. + - dsync: dsync-replication does not synchronize Sieve scripts. + - imap_sieve_filter: Reduce FILTER=SIEVE verbosity over IMAP connection. + - testsuite: Pigeonhole testsuite segfaulted if it was compiled with + GCC 9 + + sieve: Redirect loop prevention is sometimes ineffective. Improve + existing loop detection by also recognizing the + X-Sieve-Redirected-From header in incoming messages and dropping + redirect actions when it points to the sending account. This header + is already added by the redirect action, so this improvement only + adds an additional use of this header. + - sieve: Prevent execution of implicit keep upon temporary failure + occurring at runtime. + + IMAPSieve: Add new plugin/imapsieve_expunge_discarded setting which + causes messages discarded by an IMAPSieve script to be expunged + immediately, rather than only being marked as "\Deleted" (which is + still the default behavior). + - IMAPSieve: Fix panic crash occurring when a COPY command copies + messages from a virtual mailbox where the source messages originate + from more than a single real mailbox. + - imap4flags extension: Fix deleting all keywords. When the action + resulted in all keywords being removed, no changes were actually + applied. + - variables extension: Fix truncation of UTF-8 variable content. The + maximum size of Sieve variables was enforced by truncating the + variable string content bluntly at the limit, but this does not + consider UTF-8 code point boundaries. This resulted in broken UTF-8 + strings. This problem also surfaced for variable modifiers, such as + the ":encodeurl" modifier provided by the Sieve "enotify" extension. + In that case, the resulting URI escaping could also be truncated + inappropriately. + - IMAPSieve, IMAP FILTER=SIEVE: Fix replacing a modified message. Sieve + scripts running in IMAPSIEVE or IMAP FILTER=SIEVE context that + modify the message, stored the message a second time, rather than + replacing the originally stored unmodified message. + - Fix segmentation fault occurring when both the sieve_extprograms + plugin (for the Sieve interpreter) and the imap_filter_sieve plugin + (for IMAP) are loaded at the same time. A symbol was defined by both + plugins, causing a clash when both were loaded. + * Adjustments to several changes in Dovecot v2.3.4 make this Pigeonhole + release dependent on that Dovecot release; it will not compile against + older Dovecot versions. And, conversely, you need to upgrade + Pigeonhole when upgrading Dovecot to v2.3.4. + * The changes regarding the default postmaster_address in Dovecot v2.3.4 + mainly apply to Pigeonhole. The new default should work for all + existing installations, thereby fixing several reported v2.3/v0.5 + migration problems. + - IMAP FILTER=SIEVE capability: Fix assert crash occurring when running + UID FILTER on a Sieve script with errors. +- update to 2.3.10 + * Disable retpoline migitations by default. These can cause severe + performance regressions, so they should be only enabled when + applicable. + * IMAP MOVE now commits transactions in batches of 1000 mails. This + helps especially with lazy_expunge when moving a lot of mails. It + mainly avoids situations where multiple IMAP sessions are running the + same MOVE command and duplicating the mails in the lazy_expunge folder. + With this change there can still be some duplication, but the MOVE + always progresses forward. Also if the MOVE fails at some point, the + changes up to the last 1000 mails are still committed instead of + rolled back. Note that the COPY command behavior hasn't changed, + because it is required by IMAP standard to be an atomic operation. + * IMAP EXPUNGE and CLOSE now expunges mails in batches of 1000 mails. + This helps especially with lazy_expunge when expunging a lot of mails + (e.g. millions) to make sure that the progress always moves forward + even if the process is killed. + * Autoexpunging now expunges mails in batches of 1000 mails. This helps + especially with lazy_expunge when expunging a lot of mails + (e.g. millions) to make sure that the progress always moves forward + even if the process is killed. + + Add tool for generating sysreport called dovecot-sysreport. + This generates a bundle of information usually needed for support + requests. + + Add support for the new IMAP \Important SPECIAL-USE flag (RFC 8457). + + Add metric { group_by } setting. This allows automatically creating + new metrics based on the fields you want to group statistics by. + NOTE: This feature is considered experimental and syntax is subject + to change in future release. + + auth: Support SCRAM-SHA-256 authentication mechanism. + + imap: Support the new IMAP STATUS=SIZE extension. + + Use TCP_QUICKACK to reduce latency for some TCP connections. + + quota-status: Made the service more robust against erroneous use with + Postfix ACL policies other than smtpd_recipient_restrictions. + + Add "revision" field support to imap_id_send setting. Using + "revision *" will send in IMAP ID command response the short commit + hash of the Dovecot git source tree HEAD (same as in dovecot --version). + + IMAP ENVELOPE includes now all addresses when there are multiple + headers (From, To, Cc, etc.) The standard way of having multiple + addresses is to just list them all in a single header. It's + non-standard to have multiple headers. However, since MTAs allow these + mails to pass through and different software may handle them in + different ways, it's better from security point of view to show all + the addresses. + + Event filters now support using "field_name=" to match a field that + doesn't exist or has an empty value. For example use "error=" to match + only events that didn't fail. + - acl: INBOX ACLs shouldn't apply for IMAP GETMETADATA/SETMETADATA + commands. + - cassandra: CASS_ERROR_SERVER_WRITE_FAILURE error should also be + treated as "uncertain write failure". + - dict-redis: Using quota_clone configured with dict-redis could have + crashed when Redis responded slowly. + - imap-hibernate: Communication trouble with imap-master leads to + segfault. + - imap-hibernate: Unhibernation retrying wasn't working. + - imap: Fixed auth lookup privilege problem when imap process was reused + and user was being un-hibernated. + - Fix potential crash when copying/moving mails within the same folder. + This happened only when there were a lot of fields in dovecot.index.cache. + - lib-index: Recreating dovecot.index.cache file could have crashed when + merging bitmask fields. + - lib-index: Using public/shared folders with INDEXPVT configured to use + private \Seen flags, trying to search seen/unseen in an empty folder + crashes with segfault. + - lib-mail: Large base64-encoded mails weren't decoded properly. + This could have affected searching/indexing mails and message snippet + generation. + - lib-mail: Message with only quoted text could have caused message + snippet to ignore its 200 character limit and return the entire + message. This was added also to dovecot.index.cache file, which + increased disk space and memory usage unnecessarily. + v2.3.9.2 regression (previous versions cached the quoted snippet as + empty). In a large mail quoted text could have become wrongly added + to the snippet, possibly mixed together with non-quoted text. + - lib-smtp: client could have assert-crashed if STARTTLS handshake + finished earlier than usually. + - lib-ssl-iostream: remove -static flag for lib-ssl-iostream linking to + prevent a compile issue. + - lib-storage: Mailbox synchronization may have assert-crashed in some + rare situations. + - lib-storage: mdbox didn't preserve date.saved with dsync. + - lib: Don't require EAI_{ADDRFAMILY,NODATA}, breaks FreeBSD + - master: Some services could respawn unthrottled if they crash during + startup. + - push-notification: Do not send push_notification_finished event if + nothing was done. This happens when mail transaction is started and + ended with no changes. + - quota-status: Addresses with special characters in the local part caused + problems in the interaction between Postfix and Dovecot. Postfix sent + its own internal representation in the recipient field, while Dovecot + expected a valid RFC5321 mailbox address. + - submission-login: SESSION was not correctly encoded field for the + XCLIENT command. Particularly, a '+' character introduced by the + session ID's Base64 encoding causes problems. + - submission: Fix submission_max_mail_size to work correctly on 32-bit + systems. + - submission: Trusted connections crashed in second connection's EHLO + if submission-login { service_count } is something else than 1 (which + is the default). + - submission: XCLIENT command was never used in the protocol exchange + with the relay MTA when submission_backend_capabilities is configured, + even when the relay MTA was properly configured to accept the XCLIENT + command. + * CVE-2020-7046: Truncated UTF-8 can be used to DoS + submission-login and lmtp processes. + * CVE-2020-7957: Specially crafted mail can crash snippet generation. + - Mails with empty From/To headers can also cause crash + in push notification drivers. + * CVE-2019-19722: Mails with group addresses in From or To fields + caused crash in push notification drivers. + * Changed several event field names for consistency and to avoid + conflicts in parent-child event relationships: + * SMTP server command events: Renamed "name" to "cmd_name" + * Events inheriting from a mailbox: Renamed "name" to "mailbox" + * Server connection events have only "remote_ip", "remote_port", + "local_ip" and "local_port". + * Removed duplicate "client_ip", "ip" and "port". + * Mail storage events: Removed "service" field. + Use "service:<name>" category instead. + * HTTP client connection events: Renamed "host" to "dest_host" and + "port" to "dest_port" + * auth: Drop Postfix socketmap support. It hasn't been working ++++ 277 more lines (skipped) ++++ between /work/SRC/openSUSE:Leap:15.2/dovecot23/dovecot23.changes ++++ and /work/SRC/openSUSE:Leap:15.2/.dovecot23.new.2738/dovecot23.changes Old: ---- 0001-lib-imap-Don-t-accept-strings-with-NULs.patch 0001-lib-managesieve-Don-t-accept-strings-with-NULs.patch 0002-lib-imap-Make-sure-str_unescape-won-t-be-writing-pas.patch 0002-lib-managesieve-Make-sure-str_unescape-won-t-be-writ.patch dovecot-2.3-pigeonhole-0.5.3.tar.gz dovecot-2.3-pigeonhole-0.5.3.tar.gz.sig dovecot-2.3.3.tar.gz dovecot-2.3.3.tar.gz.sig dovecot-CVE-2019-11494-fix-disconnects.patch dovecot-CVE-2019-11494-fix-error-handling.patch dovecot-CVE-2019-11499-fix-pending-starttls.patch New: ---- 0001-lib-smtp-smtp-server-cmd-vrfy-Restructure-parameter-.patch 0002-lib-smtp-smtp-syntax-Do-not-allow-NULL-return-parame.patch 0003-lib-smtp-smtp-syntax-Do-not-allow-NULL-return-parame.patch 0004-lib-smtp-smtp-syntax-Do-not-allow-NULL-return-parame.patch 0005-lib-smtp-smtp-syntax-Return-0-for-smtp_string_parse-.patch 0006-lib-smtp-Add-tests-for-smtp_string_parse-and-smtp_st.patch 0007-lib-smtp-test-smtp-server-errors-Add-tests-for-VRFY-.patch 0008-lib-smtp-smtp-server-command-Guarantee-that-non-dest.patch 0009-lib-smtp-smtp-server-command-Assign-cmd-reg-immediat.patch 0010-lib-smtp-smtp-server-command-Perform-initial-command.patch 0011-lib-smtp-smtp-server-connection-Hold-a-command-refer.patch 0012-lib-smtp-test-smtp-server-errors-Add-tests-for-large.patch 0013-lib-smtp-smtp-address-Don-t-return-NULL-from-smtp_ad.patch 0014-lib-smtp-smtp-address-Don-t-recognize-an-address-wit.patch 0015-lib-smtp-smtp-address-Only-produce-a-address-in-smtp.patch 0016-lmtp-lmtp-commands-Explicity-prohibit-empty-RCPT-pat.patch dovecot-2.3-pigeonhole-0.5.10.tar.gz dovecot-2.3-pigeonhole-0.5.10.tar.gz.sig dovecot-2.3.10.tar.gz dovecot-2.3.10.tar.gz.sig ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ dovecot23.spec ++++++ --- /var/tmp/diff_new_pack.1vWNa7/_old 2020-05-23 16:07:05.433010668 +0200 +++ /var/tmp/diff_new_pack.1vWNa7/_new 2020-05-23 16:07:05.437010677 +0200 @@ -17,11 +17,11 @@ Name: dovecot23 -Version: 2.3.3 +Version: 2.3.10 Release: 0 %define pkg_name dovecot -%define dovecot_version 2.3.3 -%define dovecot_pigeonhole_version 0.5.3 +%define dovecot_version 2.3.10 +%define dovecot_pigeonhole_version 0.5.10 %define dovecot_branch 2.3 %define dovecot_pigeonhole_source_dir %{pkg_name}-%{dovecot_branch}-pigeonhole-%{dovecot_pigeonhole_version} %define dovecot_pigeonhole_docdir %{_docdir}/%{pkg_name}/dovecot-pigeonhole @@ -136,13 +136,24 @@ Source12: dovecot23.keyring Patch: dovecot-2.3.0-dont_use_etc_ssl_certs.patch Patch1: dovecot-2.3.0-better_ssl_defaults.patch -Patch2: 0001-lib-managesieve-Don-t-accept-strings-with-NULs.patch -Patch3: 0002-lib-managesieve-Make-sure-str_unescape-won-t-be-writ.patch -Patch4: 0001-lib-imap-Don-t-accept-strings-with-NULs.patch -Patch5: 0002-lib-imap-Make-sure-str_unescape-won-t-be-writing-pas.patch -Patch6: dovecot-CVE-2019-11499-fix-pending-starttls.patch -Patch7: dovecot-CVE-2019-11494-fix-disconnects.patch -Patch8: dovecot-CVE-2019-11494-fix-error-handling.patch +Patch20: 0001-lib-smtp-smtp-server-cmd-vrfy-Restructure-parameter-.patch +Patch21: 0002-lib-smtp-smtp-syntax-Do-not-allow-NULL-return-parame.patch +Patch22: 0003-lib-smtp-smtp-syntax-Do-not-allow-NULL-return-parame.patch +Patch23: 0004-lib-smtp-smtp-syntax-Do-not-allow-NULL-return-parame.patch +Patch24: 0005-lib-smtp-smtp-syntax-Return-0-for-smtp_string_parse-.patch +Patch25: 0006-lib-smtp-Add-tests-for-smtp_string_parse-and-smtp_st.patch +Patch26: 0007-lib-smtp-test-smtp-server-errors-Add-tests-for-VRFY-.patch +Patch27: 0008-lib-smtp-smtp-server-command-Guarantee-that-non-dest.patch +Patch28: 0009-lib-smtp-smtp-server-command-Assign-cmd-reg-immediat.patch +Patch29: 0010-lib-smtp-smtp-server-command-Perform-initial-command.patch +Patch30: 0011-lib-smtp-smtp-server-connection-Hold-a-command-refer.patch +Patch31: 0012-lib-smtp-test-smtp-server-errors-Add-tests-for-large.patch +Patch32: 0013-lib-smtp-smtp-address-Don-t-return-NULL-from-smtp_ad.patch +Patch33: 0014-lib-smtp-smtp-address-Don-t-recognize-an-address-wit.patch +Patch34: 0015-lib-smtp-smtp-address-Only-produce-a-address-in-smtp.patch +Patch35: 0016-lmtp-lmtp-commands-Explicity-prohibit-empty-RCPT-pat.patch + + Summary: IMAP and POP3 Server Written Primarily with Security in Mind License: BSD-3-Clause AND LGPL-2.1-or-later AND MIT @@ -323,13 +334,23 @@ %setup -q -n %{pkg_name}-%{dovecot_version} -a 1 %patch -p1 %patch1 -p1 -%patch2 -p0 -%patch3 -p0 -%patch4 -p1 -%patch5 -p1 -%patch6 -p1 -%patch7 -p1 -%patch8 -p1 +%patch20 -p1 +%patch21 -p1 +%patch22 -p1 +%patch23 -p1 +%patch24 -p1 +%patch25 -p1 +%patch26 -p1 +%patch27 -p1 +%patch28 -p1 +%patch29 -p1 +%patch30 -p1 +%patch31 -p1 +%patch32 -p1 +%patch33 -p1 +%patch34 -p1 +%patch35 -p1 + gzip -9v ChangeLog # Fix plugins dir. sed -i 's|#mail_plugin_dir = /usr/lib/dovecot|mail_plugin_dir = %{_libdir}/dovecot/modules|' doc/example-config/conf.d/10-mail.conf @@ -538,6 +559,7 @@ %{_sbindir}/%{pkg_name} %{_bindir}/doveadm %{_bindir}/doveconf +%{_bindir}/dovecot-sysreport %{_bindir}/dsync %{_bindir}/sieve-test %{_bindir}/sievec @@ -558,6 +580,7 @@ %{_prefix}/lib/%{pkg_name}/doveadm-server %{_prefix}/lib/%{pkg_name}/dovecot-lda %{_prefix}/lib/%{pkg_name}/gdbhelper +%{_prefix}/lib/%{pkg_name}/health-check.sh %{_prefix}/lib/%{pkg_name}/imap %{_prefix}/lib/%{pkg_name}/imap-hibernate %{_prefix}/lib/%{pkg_name}/imap-login @@ -603,6 +626,7 @@ %dir %{_libdir}/%{pkg_name}/modules/ %{_libdir}/%{pkg_name}/modules/lib01_acl_plugin.so %{_libdir}/%{pkg_name}/modules/lib01_apparmor_plugin.so +%{_libdir}/%{pkg_name}/modules/lib01_mail_lua_plugin.so %{_libdir}/%{pkg_name}/modules/lib02_lazy_expunge_plugin.so %{_libdir}/%{pkg_name}/modules/lib05_mail_crypt_acl_plugin.so %{_libdir}/%{pkg_name}/modules/lib05_pop3_migration_plugin.so @@ -626,6 +650,7 @@ %{_libdir}/%{pkg_name}/modules/lib20_quota_clone_plugin.so %{_libdir}/%{pkg_name}/modules/lib20_replication_plugin.so %{_libdir}/%{pkg_name}/modules/lib20_virtual_plugin.so +%{_libdir}/%{pkg_name}/modules/lib22_push_notification_lua_plugin.so %{_libdir}/%{pkg_name}/modules/lib30_imap_zlib_plugin.so %{_libdir}/%{pkg_name}/modules/lib02_imap_acl_plugin.so %{_libdir}/%{pkg_name}/modules/lib11_imap_quota_plugin.so ++++++ 0001-lib-smtp-smtp-server-cmd-vrfy-Restructure-parameter-.patch ++++++ >From d143ca6b7ee1196ae3eafffbf6dee71a95a5e0b8 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 21:05:17 +0100 Subject: [PATCH 01/16] lib-smtp: smtp-server-cmd-vrfy - Restructure parameter parsing. --- src/lib-smtp/smtp-server-cmd-vrfy.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/src/lib-smtp/smtp-server-cmd-vrfy.c b/src/lib-smtp/smtp-server-cmd-vrfy.c index 56019d7e25..6d6537b4f0 100644 --- a/src/lib-smtp/smtp-server-cmd-vrfy.c +++ b/src/lib-smtp/smtp-server-cmd-vrfy.c @@ -17,15 +17,13 @@ void smtp_server_cmd_vrfy(struct smtp_server_cmd_ctx *cmd, int ret; /* vrfy = "VRFY" SP String CRLF */ - if ((ret=smtp_string_parse(params, ¶m, &error)) <= 0) { - if (ret < 0) { - smtp_server_reply(cmd, - 501, "5.5.4", - "Invalid string parameter: %s", error); - } else { - smtp_server_reply(cmd, - 501, "5.5.4", "Invalid parameters"); - } + ret = smtp_string_parse(params, ¶m, &error); + if (ret < 0) { + smtp_server_reply(cmd, 501, "5.5.4", + "Invalid string parameter: %s", error); + return; + } else if (ret == 0) { + smtp_server_reply(cmd, 501, "5.5.4", "Invalid parameters"); return; } -- 2.11.0 ++++++ 0002-lib-smtp-smtp-syntax-Do-not-allow-NULL-return-parame.patch ++++++ >From 606724bd528b92347dce580d3ab48fc1e3c2f4d7 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 20:57:03 +0100 Subject: [PATCH 02/16] lib-smtp: smtp-syntax - Do not allow NULL return parameters for smtp_string_parse(). --- src/lib-smtp/smtp-server-cmd-noop.c | 8 ++++++-- src/lib-smtp/smtp-syntax.c | 9 +++++---- 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/src/lib-smtp/smtp-server-cmd-noop.c b/src/lib-smtp/smtp-server-cmd-noop.c index 4986f800c8..550d709eab 100644 --- a/src/lib-smtp/smtp-server-cmd-noop.c +++ b/src/lib-smtp/smtp-server-cmd-noop.c @@ -13,11 +13,15 @@ void smtp_server_cmd_noop(struct smtp_server_cmd_ctx *cmd, struct smtp_server_connection *conn = cmd->conn; struct smtp_server_command *command = cmd->cmd; const struct smtp_server_callbacks *callbacks = conn->callbacks; + const char *param, *error; int ret; /* "NOOP" [ SP String ] CRLF */ - if (*params != '\0' && smtp_string_parse(params, NULL, NULL) < 0) { - smtp_server_reply(cmd, 501, "5.5.4", "Invalid parameters"); + ret = smtp_string_parse(params, ¶m, &error); + if (ret < 0) { + smtp_server_reply(cmd, 501, "5.5.4", + "Invalid string parameter: %s", + error); return; } diff --git a/src/lib-smtp/smtp-syntax.c b/src/lib-smtp/smtp-syntax.c index 5d22445f72..6826682af1 100644 --- a/src/lib-smtp/smtp-syntax.c +++ b/src/lib-smtp/smtp-syntax.c @@ -17,7 +17,9 @@ int smtp_string_parse(const char *string, const char **value_r, const char **error_r) { struct smtp_parser parser; - int ret; + + *value_r = NULL; + *error_r = NULL; if (string == NULL || *string == '\0') { *value_r = ""; @@ -26,9 +28,8 @@ int smtp_string_parse(const char *string, smtp_parser_init(&parser, pool_datastack_create(), string); - if ((ret=smtp_parser_parse_string(&parser, value_r)) < 0) { - if (error_r != NULL) - *error_r = parser.error; + if (smtp_parser_parse_string(&parser, value_r) < 0) { + *error_r = parser.error; return -1; } if (parser.cur < parser.end) { -- 2.11.0 ++++++ 0003-lib-smtp-smtp-syntax-Do-not-allow-NULL-return-parame.patch ++++++ >From aedb205c79395de77127fb7166b29b09319df23c Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 21:11:01 +0100 Subject: [PATCH 03/16] lib-smtp: smtp-syntax - Do not allow NULL return parameters for smtp_xtext_parse(). --- src/lib-smtp/smtp-syntax.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/src/lib-smtp/smtp-syntax.c b/src/lib-smtp/smtp-syntax.c index 6826682af1..0b0a91ce07 100644 --- a/src/lib-smtp/smtp-syntax.c +++ b/src/lib-smtp/smtp-syntax.c @@ -86,20 +86,20 @@ int smtp_xtext_parse(const char *xtext, { struct smtp_parser parser; string_t *value = NULL; - int ret; + + *value_r = NULL; + *error_r = NULL; if (xtext == NULL || *xtext == '\0') { *value_r = ""; return 1; } - if (value_r != NULL) - value = t_str_new(256); + value = t_str_new(256); smtp_parser_init(&parser, pool_datastack_create(), xtext); - if ((ret=smtp_parser_parse_xtext(&parser, value)) < 0) { - if (error_r != NULL) - *error_r = parser.error; + if (smtp_parser_parse_xtext(&parser, value) < 0) { + *error_r = parser.error; return -1; } if (parser.cur < parser.end) { @@ -110,8 +110,7 @@ int smtp_xtext_parse(const char *xtext, if (value_r != NULL) { *value_r = str_c(value); if (strlen(*value_r) != str_len(value)) { - if (*error_r != NULL) - *error_r = "Encountered NUL character in xtext"; + *error_r = "Encountered NUL character in xtext"; return -1; } } -- 2.11.0 ++++++ 0004-lib-smtp-smtp-syntax-Do-not-allow-NULL-return-parame.patch ++++++ >From 874817b169d19a4ae51d80ad5798a396bfe90136 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 21:14:34 +0100 Subject: [PATCH 04/16] lib-smtp: smtp-syntax - Do not allow NULL return parameters for smtp_ehlo_line_parse(). --- src/lib-smtp/smtp-syntax.c | 39 +++++++++++++++++---------------------- 1 file changed, 17 insertions(+), 22 deletions(-) diff --git a/src/lib-smtp/smtp-syntax.c b/src/lib-smtp/smtp-syntax.c index 0b0a91ce07..5cca6c40fd 100644 --- a/src/lib-smtp/smtp-syntax.c +++ b/src/lib-smtp/smtp-syntax.c @@ -249,12 +249,10 @@ static int smtp_parse_ehlo_line(struct smtp_parser *parser, (i_isalnum(*parser->cur) || *parser->cur == '-')) parser->cur++; - if (key_r != NULL) - *key_r = p_strdup_until(parser->pool, pbegin, parser->cur); + *key_r = p_strdup_until(parser->pool, pbegin, parser->cur); if (parser->cur >= parser->end) { - if (params_r != NULL) - *params_r = p_new(parser->pool, const char *, 1); + *params_r = p_new(parser->pool, const char *, 1); return 1; } if (*parser->cur != ' ') { @@ -264,18 +262,16 @@ static int smtp_parse_ehlo_line(struct smtp_parser *parser, parser->cur++; pbegin = parser->cur; - if (params_r != NULL) - p_array_init(¶ms, parser->pool, 32); + p_array_init(¶ms, parser->pool, 32); while (parser->cur < parser->end) { if (*parser->cur == ' ') { if (parser->cur+1 >= parser->end || *(parser->cur+1) == ' ') { parser->error = "Missing EHLO parameter after ' '"; return -1; } - if (params_r != NULL) { - param = p_strdup_until(parser->pool, pbegin, parser->cur); - array_push_back(¶ms, ¶m); - } + param = p_strdup_until(parser->pool, pbegin, + parser->cur); + array_push_back(¶ms, ¶m); pbegin = parser->cur + 1; } else if (!smtp_char_is_ehlo_param(*parser->cur)) { parser->error = "Unexpected character in EHLO parameter"; @@ -284,12 +280,10 @@ static int smtp_parse_ehlo_line(struct smtp_parser *parser, parser->cur++; } - if (params_r != NULL) { - param = p_strdup_until(parser->pool, pbegin, parser->cur); - array_push_back(¶ms, ¶m); - array_append_zero(¶ms); - *params_r = array_front(¶ms); - } + param = p_strdup_until(parser->pool, pbegin, parser->cur); + array_push_back(¶ms, ¶m); + array_append_zero(¶ms); + *params_r = array_front(¶ms); return 1; } @@ -297,19 +291,20 @@ int smtp_ehlo_line_parse(const char *ehlo_line, const char **key_r, const char *const **params_r, const char **error_r) { struct smtp_parser parser; - int ret; + + *key_r = NULL; + *params_r = NULL; + *error_r = NULL; if (ehlo_line == NULL || *ehlo_line == '\0') { - if (error_r != NULL) - *error_r = "Parameter is empty"; + *error_r = "Parameter is empty"; return -1; } smtp_parser_init(&parser, pool_datastack_create(), ehlo_line); - if ((ret=smtp_parse_ehlo_line(&parser, key_r, params_r)) <= 0) { - if (error_r != NULL) - *error_r = parser.error; + if (smtp_parse_ehlo_line(&parser, key_r, params_r) <= 0) { + *error_r = parser.error; return -1; } return 1; -- 2.11.0 ++++++ 0005-lib-smtp-smtp-syntax-Return-0-for-smtp_string_parse-.patch ++++++ >From 5efeccc10beccbf8d7700adec1278f97d416cbc6 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 22:42:15 +0100 Subject: [PATCH 05/16] lib-smtp: smtp-syntax - Return 0 for smtp_string_parse() with empty input. This is what the current users of this function actually expect. --- src/lib-smtp/smtp-syntax.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib-smtp/smtp-syntax.c b/src/lib-smtp/smtp-syntax.c index 5cca6c40fd..cf86f87336 100644 --- a/src/lib-smtp/smtp-syntax.c +++ b/src/lib-smtp/smtp-syntax.c @@ -23,7 +23,7 @@ int smtp_string_parse(const char *string, if (string == NULL || *string == '\0') { *value_r = ""; - return 1; + return 0; } smtp_parser_init(&parser, pool_datastack_create(), string); -- 2.11.0 ++++++ 0006-lib-smtp-Add-tests-for-smtp_string_parse-and-smtp_st.patch ++++++ >From 363af76535f8137ba76d9de7935023bab9a045ef Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 22:24:20 +0100 Subject: [PATCH 06/16] lib-smtp: Add tests for smtp_string_parse() and smtp_string_write(). --- src/lib-smtp/Makefile.am | 5 ++ src/lib-smtp/test-smtp-syntax.c | 150 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 155 insertions(+) create mode 100644 src/lib-smtp/test-smtp-syntax.c diff --git a/src/lib-smtp/Makefile.am b/src/lib-smtp/Makefile.am index b03761df8b..d87cd4e6d3 100644 --- a/src/lib-smtp/Makefile.am +++ b/src/lib-smtp/Makefile.am @@ -72,6 +72,7 @@ pkginc_libdir=$(pkgincludedir) pkginc_lib_HEADERS = $(headers) test_programs = \ + test-smtp-syntax \ test-smtp-address \ test-smtp-params \ test-smtp-reply \ @@ -121,6 +122,10 @@ if BUILD_OPENSSL test_libs_ssl += ../lib-ssl-iostream/libssl_iostream_openssl.la endif +test_smtp_syntax_SOURCES = test-smtp-syntax.c +test_smtp_syntax_LDADD = $(test_libs) +test_smtp_syntax_DEPENDENCIES = $(test_deps) + test_smtp_address_SOURCES = test-smtp-address.c test_smtp_address_LDFLAGS = -export-dynamic test_smtp_address_LDADD = $(test_libs) diff --git a/src/lib-smtp/test-smtp-syntax.c b/src/lib-smtp/test-smtp-syntax.c new file mode 100644 index 0000000000..735cd01220 --- /dev/null +++ b/src/lib-smtp/test-smtp-syntax.c @@ -0,0 +1,150 @@ +/* Copyright (c) 2020 Dovecot authors, see the included COPYING file */ + +#include "lib.h" +#include "str.h" +#include "str-sanitize.h" +#include "test-common.h" +#include "smtp-syntax.h" + +/* + * Valid string parse tests + */ + +struct valid_string_parse_test { + const char *input, *parsed, *output; +}; + +static const struct valid_string_parse_test +valid_string_parse_tests[] = { + { + .input = "", + .parsed = "", + }, + { + .input = "atom", + .parsed = "atom", + }, + { + .input = "abcdefghijklmnopqrstuvwxyz" + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "0123456789!#$%&'*+-/=?^_`{|}~", + .parsed = "abcdefghijklmnopqrstuvwxyz" + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "0123456789!#$%&'*+-/=?^_`{|}~", + }, + { + .input = "\"quoted-string\"", + .parsed = "quoted-string", + .output = "quoted-string", + }, + { + .input = "\"quoted \\\"string\\\"\"", + .parsed = "quoted \"string\"", + }, + { + .input = "\"quoted \\\\string\\\\\"", + .parsed = "quoted \\string\\", + }, +}; + +static const unsigned int valid_string_parse_test_count = + N_ELEMENTS(valid_string_parse_tests); + +static void test_smtp_string_parse_valid(void) +{ + unsigned int i; + + for (i = 0; i < valid_string_parse_test_count; i++) T_BEGIN { + const struct valid_string_parse_test *test = + &valid_string_parse_tests[i]; + const char *parsed, *error = NULL; + int ret; + + ret = smtp_string_parse(test->input, &parsed, &error); + + test_begin(t_strdup_printf("smtp string valid [%d]", i)); + test_out_reason(t_strdup_printf("parse(\"%s\")", test->input), + ret >= 0, error); + test_assert(ret != 0 || *test->input == '\0'); + + if (!test_has_failed()) { + string_t *encoded; + const char *output; + + test_out(t_strdup_printf("parsed = \"%s\"", parsed), + null_strcmp(parsed, test->parsed) == 0); + + encoded = t_str_new(255); + smtp_string_write(encoded, parsed); + output = (test->output == NULL ? + test->input : test->output); + test_out(t_strdup_printf("write() = \"%s\"", + str_c(encoded)), + strcmp(str_c(encoded), output) == 0); + } + test_end(); + } T_END; +} + +/* + * Invalid string parse tests + */ + +struct invalid_string_parse_test { + const char *input; +}; + +static const struct invalid_string_parse_test +invalid_string_parse_tests[] = { + { + .input = " ", + }, + { + .input = "\\", + }, + { + .input = "\"", + }, + { + .input = "\"aa", + }, + { + .input = "aa\"", + }, +}; + +static const unsigned int invalid_string_parse_test_count = + N_ELEMENTS(invalid_string_parse_tests); + +static void test_smtp_string_parse_invalid(void) +{ + unsigned int i; + + for (i = 0; i < invalid_string_parse_test_count; i++) T_BEGIN { + const struct invalid_string_parse_test *test = + &invalid_string_parse_tests[i]; + const char *parsed, *error; + int ret; + + ret = smtp_string_parse(test->input, &parsed, &error); + + test_begin(t_strdup_printf("smtp string invalid [%d]", i)); + test_out_reason(t_strdup_printf("parse(\"%s\")", test->input), + ret < 0, error); + test_end(); + } T_END; +} + +/* + * Tests + */ + +int main(void) +{ + static void (*test_functions[])(void) = { + test_smtp_string_parse_valid, + test_smtp_string_parse_invalid, + NULL + }; + return test_run(test_functions); +} -- 2.11.0 ++++++ 0007-lib-smtp-test-smtp-server-errors-Add-tests-for-VRFY-.patch ++++++ >From f206dc478d1b22183f645ea8c98ffe3712e106b2 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 22:33:45 +0100 Subject: [PATCH 07/16] lib-smtp: test-smtp-server-errors - Add tests for VRFY and NOOP commands with invalid parameters. --- src/lib-smtp/test-smtp-server-errors.c | 312 +++++++++++++++++++++++++++++++++ 1 file changed, 312 insertions(+) diff --git a/src/lib-smtp/test-smtp-server-errors.c b/src/lib-smtp/test-smtp-server-errors.c index 759c332d88..13d3b97def 100644 --- a/src/lib-smtp/test-smtp-server-errors.c +++ b/src/lib-smtp/test-smtp-server-errors.c @@ -1274,6 +1274,316 @@ static void test_bad_rcpt(void) } /* + * Bad VRFY + */ + +/* client */ + +struct _bad_vrfy_client { + struct smtp_reply_parser *parser; + unsigned int reply; + + bool replied:1; +}; + +static void +test_bad_vrfy_client_input(struct client_connection *conn) +{ + struct _bad_vrfy_client *ctx = conn->context; + struct smtp_reply *reply; + const char *error; + int ret; + + while ((ret = smtp_reply_parse_next(ctx->parser, FALSE, + &reply, &error)) > 0) { + if (debug) + i_debug("REPLY: %s", smtp_reply_log(reply)); + + switch (ctx->reply++) { + case 0: /* greeting */ + i_assert(reply->status == 220); + break; + case 1: /* bad command reply */ + switch (client_index) { + case 0: case 1: case 2: + i_assert(reply->status == 501); + break; + case 3: + i_assert(smtp_reply_is_success(reply)); + break; + default: + i_unreached(); + } + ctx->replied = TRUE; + io_loop_stop(ioloop); + connection_disconnect(&conn->conn); + return; + default: + i_unreached(); + } + } + + i_assert(ret == 0); +} + +static void +test_bad_vrfy_client_connected(struct client_connection *conn) +{ + struct _bad_vrfy_client *ctx; + + ctx = p_new(conn->pool, struct _bad_vrfy_client, 1); + ctx->parser = smtp_reply_parser_init(conn->conn.input, (size_t)-1); + conn->context = ctx; + + switch (client_index) { + case 0: + o_stream_nsend_str(conn->conn.output, + "VRFY\r\n"); + break; + case 1: + o_stream_nsend_str(conn->conn.output, + "VRFY \"hendrik\r\n"); + break; + case 2: + o_stream_nsend_str(conn->conn.output, + "VRFY hen\"drik\r\n"); + break; + case 3: + o_stream_nsend_str(conn->conn.output, + "VRFY \"hendrik\"\r\n"); + break; + default: + i_unreached(); + } +} + +static void +test_bad_vrfy_client_deinit(struct client_connection *conn) +{ + struct _bad_vrfy_client *ctx = conn->context; + + i_assert(ctx->replied); + smtp_reply_parser_deinit(&ctx->parser); +} + +static void test_client_bad_vrfy(unsigned int index) +{ + test_client_input = test_bad_vrfy_client_input; + test_client_connected = test_bad_vrfy_client_connected; + test_client_deinit = test_bad_vrfy_client_deinit; + test_client_run(index); +} + +/* server */ + +static void +test_server_bad_vrfy_disconnect(void *context ATTR_UNUSED, const char *reason) +{ + if (debug) + i_debug("Disconnect: %s", reason); +} + +static int +test_server_bad_vrfy_rcpt(void *conn_ctx ATTR_UNUSED, + struct smtp_server_cmd_ctx *cmd ATTR_UNUSED, + struct smtp_server_recipient *rcpt ATTR_UNUSED) +{ + test_assert(FALSE); + return 1; +} + +static int +test_server_bad_vrfy_data_begin( + void *conn_ctx ATTR_UNUSED, struct smtp_server_cmd_ctx *cmd ATTR_UNUSED, + struct smtp_server_transaction *trans ATTR_UNUSED, + struct istream *data_input ATTR_UNUSED) +{ + test_assert(FALSE); + return 1; +} + +static void +test_server_bad_vrfy(const struct smtp_server_settings *server_set) +{ + server_callbacks.conn_disconnect = test_server_bad_vrfy_disconnect; + + server_callbacks.conn_cmd_rcpt = test_server_bad_vrfy_rcpt; + server_callbacks.conn_cmd_data_begin = test_server_bad_vrfy_data_begin; + test_server_run(server_set); +} + +/* test */ + +static void test_bad_vrfy(void) +{ + struct smtp_server_settings smtp_server_set; + + test_server_defaults(&smtp_server_set); + smtp_server_set.max_client_idle_time_msecs = 1000; + + test_begin("bad VRFY"); + test_run_client_server(&smtp_server_set, + test_server_bad_vrfy, + test_client_bad_vrfy, 4); + test_end(); +} + +/* + * Bad NOOP + */ + +/* client */ + +struct _bad_noop_client { + struct smtp_reply_parser *parser; + unsigned int reply; + + bool replied:1; +}; + +static void +test_bad_noop_client_input(struct client_connection *conn) +{ + struct _bad_noop_client *ctx = conn->context; + struct smtp_reply *reply; + const char *error; + int ret; + + while ((ret = smtp_reply_parse_next(ctx->parser, FALSE, + &reply, &error)) > 0) { + if (debug) + i_debug("REPLY: %s", smtp_reply_log(reply)); + + switch (ctx->reply++) { + case 0: /* greeting */ + i_assert(reply->status == 220); + break; + case 1: /* bad command reply */ + switch (client_index) { + case 1: case 2: + i_assert(reply->status == 501); + break; + case 0: case 3: + i_assert(smtp_reply_is_success(reply)); + break; + default: + i_unreached(); + } + ctx->replied = TRUE; + io_loop_stop(ioloop); + connection_disconnect(&conn->conn); + return; + default: + i_unreached(); + } + } + + i_assert(ret == 0); +} + +static void +test_bad_noop_client_connected(struct client_connection *conn) +{ + struct _bad_noop_client *ctx; + + ctx = p_new(conn->pool, struct _bad_noop_client, 1); + ctx->parser = smtp_reply_parser_init(conn->conn.input, (size_t)-1); + conn->context = ctx; + + switch (client_index) { + case 0: + o_stream_nsend_str(conn->conn.output, + "NOOP\r\n"); + break; + case 1: + o_stream_nsend_str(conn->conn.output, + "NOOP \"frop\r\n"); + break; + case 2: + o_stream_nsend_str(conn->conn.output, + "NOOP fr\"op\r\n"); + break; + case 3: + o_stream_nsend_str(conn->conn.output, + "NOOP \"frop\"\r\n"); + break; + default: + i_unreached(); + } +} + +static void +test_bad_noop_client_deinit(struct client_connection *conn) +{ + struct _bad_noop_client *ctx = conn->context; + + i_assert(ctx->replied); + smtp_reply_parser_deinit(&ctx->parser); +} + +static void test_client_bad_noop(unsigned int index) +{ + test_client_input = test_bad_noop_client_input; + test_client_connected = test_bad_noop_client_connected; + test_client_deinit = test_bad_noop_client_deinit; + test_client_run(index); +} + +/* server */ + +static void +test_server_bad_noop_disconnect(void *context ATTR_UNUSED, const char *reason) +{ + if (debug) + i_debug("Disconnect: %s", reason); +} + +static int +test_server_bad_noop_rcpt(void *conn_ctx ATTR_UNUSED, + struct smtp_server_cmd_ctx *cmd ATTR_UNUSED, + struct smtp_server_recipient *rcpt ATTR_UNUSED) +{ + test_assert(FALSE); + return 1; +} + +static int +test_server_bad_noop_data_begin( + void *conn_ctx ATTR_UNUSED, struct smtp_server_cmd_ctx *cmd ATTR_UNUSED, + struct smtp_server_transaction *trans ATTR_UNUSED, + struct istream *data_input ATTR_UNUSED) +{ + test_assert(FALSE); + return 1; +} + +static void +test_server_bad_noop(const struct smtp_server_settings *server_set) +{ + server_callbacks.conn_disconnect = test_server_bad_noop_disconnect; + + server_callbacks.conn_cmd_rcpt = test_server_bad_noop_rcpt; + server_callbacks.conn_cmd_data_begin = test_server_bad_noop_data_begin; + test_server_run(server_set); +} + +/* test */ + +static void test_bad_noop(void) +{ + struct smtp_server_settings smtp_server_set; + + test_server_defaults(&smtp_server_set); + smtp_server_set.max_client_idle_time_msecs = 1000; + + test_begin("bad NOOP"); + test_run_client_server(&smtp_server_set, + test_server_bad_noop, + test_client_bad_noop, 4); + test_end(); +} + +/* * MAIL workarounds */ @@ -2246,6 +2556,8 @@ static void (*const test_functions[])(void) = { test_bad_ehlo, test_bad_mail, test_bad_rcpt, + test_bad_vrfy, + test_bad_noop, test_mail_workarounds, test_rcpt_workarounds, test_too_many_recipients, -- 2.11.0 ++++++ 0008-lib-smtp-smtp-server-command-Guarantee-that-non-dest.patch ++++++ >From 2b4f1e47a4ca8a192bf3f7e944c0ad07b21b2ed1 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 12:13:43 +0100 Subject: [PATCH 08/16] lib-smtp: smtp-server-command - Guarantee that non-destroy hooks aren't called for an ended command. --- src/lib-smtp/smtp-server-command.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/lib-smtp/smtp-server-command.c b/src/lib-smtp/smtp-server-command.c index ac25933a64..a4d0bc70b6 100644 --- a/src/lib-smtp/smtp-server-command.c +++ b/src/lib-smtp/smtp-server-command.c @@ -389,8 +389,11 @@ bool smtp_server_command_call_hooks(struct smtp_server_command **_cmd, struct smtp_server_command *cmd = *_cmd; struct smtp_server_command_hook *hook; - if (type != SMTP_SERVER_COMMAND_HOOK_DESTROY) + if (type != SMTP_SERVER_COMMAND_HOOK_DESTROY) { + if (cmd->state >= SMTP_SERVER_COMMAND_STATE_FINISHED) + return FALSE; smtp_server_command_ref(cmd); + } hook = cmd->hooks_head; while (hook != NULL) { -- 2.11.0 ++++++ 0009-lib-smtp-smtp-server-command-Assign-cmd-reg-immediat.patch ++++++ >From df12ba82bda2469a21ff07a5cefda8096fd27ee3 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 17 Mar 2020 11:58:52 +0100 Subject: [PATCH 09/16] lib-smtp: smtp-server-command - Assign cmd->reg immediately. --- src/lib-smtp/smtp-server-command.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/lib-smtp/smtp-server-command.c b/src/lib-smtp/smtp-server-command.c index a4d0bc70b6..b5cdf992bf 100644 --- a/src/lib-smtp/smtp-server-command.c +++ b/src/lib-smtp/smtp-server-command.c @@ -182,11 +182,12 @@ smtp_server_command_new(struct smtp_server_connection *conn, const char *name, const char *params) { struct smtp_server *server = conn->server; - const struct smtp_server_command_reg *cmd_reg; struct smtp_server_command *cmd; cmd = smtp_server_command_alloc(conn); cmd->context.name = p_strdup(cmd->context.pool, name); + cmd->reg = smtp_server_command_find(server, name); + smtp_server_command_update_event(cmd); struct event_passthrough *e = @@ -194,7 +195,7 @@ smtp_server_command_new(struct smtp_server_connection *conn, set_name("smtp_server_command_started"); e_debug(e->event(), "New command"); - if ((cmd_reg=smtp_server_command_find(server, name)) == NULL) { + if (cmd->reg == NULL) { /* RFC 5321, Section 4.2.4: Reply Code 502 Questions have been raised as to when reply code 502 (Command @@ -207,7 +208,7 @@ smtp_server_command_new(struct smtp_server_connection *conn, 500, "5.5.1", "Unknown command"); } else if (!conn->ssl_secured && conn->set.tls_required && - (cmd_reg->flags & SMTP_SERVER_CMD_FLAG_PRETLS) == 0) { + (cmd->reg->flags & SMTP_SERVER_CMD_FLAG_PRETLS) == 0) { /* RFC 3207, Section 4: A SMTP server that is not publicly referenced may choose to @@ -226,7 +227,7 @@ smtp_server_command_new(struct smtp_server_connection *conn, 530, "5.7.0", "TLS required."); } else if (!conn->authenticated && !conn->set.auth_optional && - (cmd_reg->flags & SMTP_SERVER_CMD_FLAG_PREAUTH) == 0) { + (cmd->reg->flags & SMTP_SERVER_CMD_FLAG_PREAUTH) == 0) { /* RFC 4954, Section 6: Status Codes 530 5.7.0 Authentication required @@ -242,10 +243,9 @@ smtp_server_command_new(struct smtp_server_connection *conn, } else { struct smtp_server_command *tmp_cmd = cmd; - i_assert(cmd_reg->func != NULL); + i_assert(cmd->reg->func != NULL); smtp_server_command_ref(tmp_cmd); - tmp_cmd->reg = cmd_reg; - cmd_reg->func(&tmp_cmd->context, params); + cmd->reg->func(&tmp_cmd->context, params); if (tmp_cmd->state == SMTP_SERVER_COMMAND_STATE_NEW) tmp_cmd->state = SMTP_SERVER_COMMAND_STATE_PROCESSING; if (!smtp_server_command_unref(&tmp_cmd)) -- 2.11.0 ++++++ 0010-lib-smtp-smtp-server-command-Perform-initial-command.patch ++++++ >From 563bf21d8228a3c06c63b3f289a90ca3d0c579a4 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 12:23:32 +0100 Subject: [PATCH 10/16] lib-smtp: smtp-server-command - Perform initial command execution in separate function. --- src/lib-smtp/smtp-server-command.c | 11 +++++++++-- src/lib-smtp/smtp-server-connection.c | 3 ++- src/lib-smtp/smtp-server-private.h | 7 +++++-- 3 files changed, 16 insertions(+), 5 deletions(-) diff --git a/src/lib-smtp/smtp-server-command.c b/src/lib-smtp/smtp-server-command.c index b5cdf992bf..67424b60a7 100644 --- a/src/lib-smtp/smtp-server-command.c +++ b/src/lib-smtp/smtp-server-command.c @@ -179,7 +179,7 @@ smtp_server_command_new_invalid(struct smtp_server_connection *conn) struct smtp_server_command * smtp_server_command_new(struct smtp_server_connection *conn, - const char *name, const char *params) + const char *name) { struct smtp_server *server = conn->server; struct smtp_server_command *cmd; @@ -195,6 +195,14 @@ smtp_server_command_new(struct smtp_server_connection *conn, set_name("smtp_server_command_started"); e_debug(e->event(), "New command"); + return cmd; +} + +void smtp_server_command_execute(struct smtp_server_command *cmd, + const char *params) +{ + struct smtp_server_connection *conn = cmd->context.conn; + if (cmd->reg == NULL) { /* RFC 5321, Section 4.2.4: Reply Code 502 @@ -251,7 +259,6 @@ smtp_server_command_new(struct smtp_server_connection *conn, if (!smtp_server_command_unref(&tmp_cmd)) cmd = NULL; } - return cmd; } void smtp_server_command_ref(struct smtp_server_command *cmd) diff --git a/src/lib-smtp/smtp-server-connection.c b/src/lib-smtp/smtp-server-connection.c index 1bfb728591..24843088fc 100644 --- a/src/lib-smtp/smtp-server-connection.c +++ b/src/lib-smtp/smtp-server-connection.c @@ -295,7 +295,8 @@ smtp_server_connection_handle_command(struct smtp_server_connection *conn, struct smtp_server_command *cmd; smtp_server_connection_ref(tmp_conn); - cmd = smtp_server_command_new(tmp_conn, cmd_name, cmd_params); + cmd = smtp_server_command_new(tmp_conn, cmd_name); + smtp_server_command_execute(cmd, cmd_params); if (!smtp_server_connection_unref(&tmp_conn)) { /* the command start callback managed to get this connection destroyed */ diff --git a/src/lib-smtp/smtp-server-private.h b/src/lib-smtp/smtp-server-private.h index e0f2d8d274..aa0d30a5ea 100644 --- a/src/lib-smtp/smtp-server-private.h +++ b/src/lib-smtp/smtp-server-private.h @@ -241,8 +241,11 @@ void smtp_server_command_debug(struct smtp_server_cmd_ctx *cmd, struct smtp_server_command * smtp_server_command_new_invalid(struct smtp_server_connection *conn); struct smtp_server_command * -smtp_server_command_new(struct smtp_server_connection *conn, - const char *name, const char *params); +smtp_server_command_new(struct smtp_server_connection *conn, const char *name); + +void smtp_server_command_execute(struct smtp_server_command *cmd, + const char *params); + void smtp_server_command_ref(struct smtp_server_command *cmd); bool smtp_server_command_unref(struct smtp_server_command **_cmd); void smtp_server_command_abort(struct smtp_server_command **_cmd); -- 2.11.0 ++++++ 0011-lib-smtp-smtp-server-connection-Hold-a-command-refer.patch ++++++ >From 18d5837748d3eafe56e080653d5ed0b3e221be0b Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 12:25:03 +0100 Subject: [PATCH 11/16] lib-smtp: smtp-server-connection - Hold a command reference while executing a command. This fixes a use-after-free problem at the end of smtp_server_connection_handle_command(). --- src/lib-smtp/smtp-server-connection.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/src/lib-smtp/smtp-server-connection.c b/src/lib-smtp/smtp-server-connection.c index 24843088fc..a4b7047ed7 100644 --- a/src/lib-smtp/smtp-server-connection.c +++ b/src/lib-smtp/smtp-server-connection.c @@ -293,21 +293,28 @@ smtp_server_connection_handle_command(struct smtp_server_connection *conn, { struct smtp_server_connection *tmp_conn = conn; struct smtp_server_command *cmd; + bool finished; - smtp_server_connection_ref(tmp_conn); cmd = smtp_server_command_new(tmp_conn, cmd_name); + + smtp_server_command_ref(cmd); + + smtp_server_connection_ref(tmp_conn); smtp_server_command_execute(cmd, cmd_params); if (!smtp_server_connection_unref(&tmp_conn)) { /* the command start callback managed to get this connection destroyed */ + smtp_server_command_unref(&cmd); return FALSE; } - if (cmd != NULL && conn->command_queue_head == cmd) + if (conn->command_queue_head == cmd) (void)smtp_server_command_next_to_reply(&cmd); smtp_server_connection_timeout_update(conn); - return (cmd == NULL || !cmd->input_locked); + + finished = !cmd->input_locked; + return (!smtp_server_command_unref(&cmd) || finished); } static int -- 2.11.0 ++++++ 0012-lib-smtp-test-smtp-server-errors-Add-tests-for-large.patch ++++++ >From a66d73c5bf89ba1d063942f482e24c41f4e10795 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Tue, 24 Mar 2020 12:59:15 +0100 Subject: [PATCH 12/16] lib-smtp: test-smtp-server-errors - Add tests for large series of empty and bad commands. --- src/lib-smtp/test-smtp-server-errors.c | 169 +++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+) diff --git a/src/lib-smtp/test-smtp-server-errors.c b/src/lib-smtp/test-smtp-server-errors.c index 13d3b97def..f76bd5eb17 100644 --- a/src/lib-smtp/test-smtp-server-errors.c +++ b/src/lib-smtp/test-smtp-server-errors.c @@ -574,6 +574,174 @@ static void test_bad_command(void) } /* + * Many bad commands + */ + +/* client */ + +struct _many_bad_commands_client { + struct smtp_reply_parser *parser; + unsigned int reply; + bool replied:1; +}; + +static void +test_many_bad_commands_client_input(struct client_connection *conn) +{ + struct _many_bad_commands_client *ctx = conn->context; + struct smtp_reply *reply; + const char *error; + int ret; + + while ((ret=smtp_reply_parse_next(ctx->parser, FALSE, + &reply, &error)) > 0) { + if (debug) + i_debug("REPLY: %s", smtp_reply_log(reply)); + + switch (ctx->reply++) { + /* greeting */ + case 0: + i_assert(reply->status == 220); + break; + /* bad command reply */ + case 1: case 2: case 3: case 4: case 5: + case 6: case 7: case 8: case 9: case 10: + i_assert(reply->status == 500); + break; + case 11: + i_assert(reply->status == 421); + ctx->replied = TRUE; + io_loop_stop(ioloop); + connection_disconnect(&conn->conn); + return; + default: + i_unreached(); + } + } + + i_assert(ret >= 0); +} + +static void +test_many_bad_commands_client_connected(struct client_connection *conn) +{ + struct _many_bad_commands_client *ctx; + + ctx = p_new(conn->pool, struct _many_bad_commands_client, 1); + ctx->parser = smtp_reply_parser_init(conn->conn.input, (size_t)-1); + conn->context = ctx; + + switch (client_index) { + case 0: + o_stream_nsend_str(conn->conn.output, + "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"); + break; + case 1: + o_stream_nsend_str(conn->conn.output, + "a\r\nb\r\nc\r\nd\r\ne\r\nf\r\ng\r\nh\r\n" + "i\r\nj\r\nk\r\nl\r\nm\r\nn\r\no\r\np\r\n"); + break; + default: + i_unreached(); + } +} + +static void +test_many_bad_commands_client_deinit(struct client_connection *conn) +{ + struct _many_bad_commands_client *ctx = conn->context; + + i_assert(ctx->replied); + smtp_reply_parser_deinit(&ctx->parser); +} + +static void test_client_many_bad_commands(unsigned int index) +{ + test_client_input = test_many_bad_commands_client_input; + test_client_connected = test_many_bad_commands_client_connected; + test_client_deinit = test_many_bad_commands_client_deinit; + test_client_run(index); +} + +/* server */ + +struct _many_bad_commands { + struct istream *payload_input; + struct io *io; + + bool serviced:1; +}; + +static void +test_server_many_bad_commands_disconnect(void *context ATTR_UNUSED, + const char *reason) +{ + if (debug) + i_debug("Disconnect: %s", reason); + io_loop_stop(ioloop); +} + +static int +test_server_many_bad_commands_helo( + void *conn_ctx ATTR_UNUSED, struct smtp_server_cmd_ctx *cmd ATTR_UNUSED, + struct smtp_server_cmd_helo *data ATTR_UNUSED) +{ + test_assert(FALSE); + return 1; +} + +static int +test_server_many_bad_commands_rcpt( + void *conn_ctx ATTR_UNUSED, struct smtp_server_cmd_ctx *cmd ATTR_UNUSED, + struct smtp_server_recipient *rcpt ATTR_UNUSED) +{ + test_assert(FALSE); + return 1; +} + +static int +test_server_many_bad_commands_data_begin( + void *conn_ctx ATTR_UNUSED, struct smtp_server_cmd_ctx *cmd ATTR_UNUSED, + struct smtp_server_transaction *trans ATTR_UNUSED, + struct istream *data_input ATTR_UNUSED) +{ + test_assert(FALSE); + return 1; +} + +static void test_server_many_bad_commands +(const struct smtp_server_settings *server_set) +{ + server_callbacks.conn_disconnect = + test_server_many_bad_commands_disconnect; + + server_callbacks.conn_cmd_helo = + test_server_many_bad_commands_helo; + server_callbacks.conn_cmd_rcpt = + test_server_many_bad_commands_rcpt; + server_callbacks.conn_cmd_data_begin = + test_server_many_bad_commands_data_begin; + test_server_run(server_set); +} + +/* test */ + +static void test_many_bad_commands(void) +{ + struct smtp_server_settings smtp_server_set; + + test_server_defaults(&smtp_server_set); + smtp_server_set.max_client_idle_time_msecs = 1000; + smtp_server_set.max_bad_commands = 10; + + test_begin("many bad commands"); + test_run_client_server(&smtp_server_set, + test_server_many_bad_commands, + test_client_many_bad_commands, 2); + test_end(); +} + +/* * Long command */ @@ -2551,6 +2719,7 @@ static void (*const test_functions[])(void) = { test_slow_client, test_hanging_command_payload, test_bad_command, + test_many_bad_commands, test_long_command, test_big_data, test_bad_ehlo, -- 2.11.0 ++++++ 0013-lib-smtp-smtp-address-Don-t-return-NULL-from-smtp_ad.patch ++++++ >From b34002a4ca301ed94cd944ee3504287ed7e58031 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Sun, 22 Mar 2020 18:14:44 +0100 Subject: [PATCH 13/16] lib-smtp: smtp-address - Don't return NULL from smtp_address_clone*() unless the input is NULL. --- src/lib-smtp/smtp-address.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lib-smtp/smtp-address.c b/src/lib-smtp/smtp-address.c index 9487e2bc6c..dd045fb662 100644 --- a/src/lib-smtp/smtp-address.c +++ b/src/lib-smtp/smtp-address.c @@ -789,7 +789,7 @@ smtp_address_clone(pool_t pool, const struct smtp_address *src) size_t size, lpsize = 0, dsize = 0, rsize = 0; char *data, *localpart = NULL, *domain = NULL, *raw = NULL; - if (src == NULL || (smtp_address_isnull(src) && src->raw == NULL)) + if (src == NULL) return NULL; /* @UNSAFE */ @@ -857,7 +857,7 @@ struct smtp_address *smtp_address_clone_temp(const struct smtp_address *src) { struct smtp_address *new; - if (src == NULL || (smtp_address_isnull(src) && src->raw == NULL)) + if (src == NULL) return NULL; new = t_new(struct smtp_address, 1); -- 2.11.0 ++++++ 0014-lib-smtp-smtp-address-Don-t-recognize-an-address-wit.patch ++++++ >From 063462d588eaea6f266596fae5f5470792dcc98d Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Fri, 20 Mar 2020 13:35:19 +0100 Subject: [PATCH 14/16] lib-smtp: smtp-address - Don't recognize an address with empty localpart as <>. Depending on context, the addresses <""@domain.tld> and <""> are potentially valid non-null addresses. --- src/lib-smtp/smtp-address.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/lib-smtp/smtp-address.h b/src/lib-smtp/smtp-address.h index 64b6a009f4..326a6738f2 100644 --- a/src/lib-smtp/smtp-address.h +++ b/src/lib-smtp/smtp-address.h @@ -200,15 +200,14 @@ smtp_address_equals_icase(const struct smtp_address *address1, static inline bool ATTR_NULL(1) ATTR_PURE smtp_address_isnull(const struct smtp_address *address) { - return (address == NULL || address->localpart == NULL || - *address->localpart == '\0'); + return (address == NULL || address->localpart == NULL); } static inline bool ATTR_NULL(1) ATTR_PURE smtp_address_is_broken(const struct smtp_address *address) { return (address != NULL && - (address->localpart == NULL || *address->localpart == '\0') && + smtp_address_isnull(address) && (address->raw != NULL && *address->raw != '\0')); } -- 2.11.0 ++++++ 0015-lib-smtp-smtp-address-Only-produce-a-address-in-smtp.patch ++++++ >From cbab48f174580bfb8d49321d8d336f96a231b0cd Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Fri, 20 Mar 2020 13:37:04 +0100 Subject: [PATCH 15/16] lib-smtp: smtp-address - Only produce a <> address in smtp_address_clone() when that is the input. It also produced an effective null address when the localpart was empty. --- src/lib-smtp/smtp-address.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib-smtp/smtp-address.c b/src/lib-smtp/smtp-address.c index dd045fb662..1997535d51 100644 --- a/src/lib-smtp/smtp-address.c +++ b/src/lib-smtp/smtp-address.c @@ -795,7 +795,7 @@ smtp_address_clone(pool_t pool, const struct smtp_address *src) /* @UNSAFE */ size = sizeof(struct smtp_address); - if (src->localpart != NULL && *src->localpart != '\0') { + if (!smtp_address_isnull(src)) { lpsize = strlen(src->localpart) + 1; size = MALLOC_ADD(size, lpsize); } -- 2.11.0 ++++++ 0016-lmtp-lmtp-commands-Explicity-prohibit-empty-RCPT-pat.patch ++++++ >From 92d9690da195b6ceaa878ab1df6c7c31a75f63f8 Mon Sep 17 00:00:00 2001 From: Stephan Bosch <[email protected]> Date: Fri, 20 Mar 2020 13:38:41 +0100 Subject: [PATCH 16/16] lmtp: lmtp-commands - Explicity prohibit empty RCPT path. The empty path <""> will yield an empty username. --- src/lmtp/lmtp-commands.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/lmtp/lmtp-commands.c b/src/lmtp/lmtp-commands.c index c4d5989825..c368d6b668 100644 --- a/src/lmtp/lmtp-commands.c +++ b/src/lmtp/lmtp-commands.c @@ -66,10 +66,18 @@ int client_default_cmd_rcpt(struct client *client, char delim = '\0'; int ret; + i_assert(!smtp_address_isnull(rcpt->path)); + if (*rcpt->path->localpart == '\0' && rcpt->path->domain == NULL) { + smtp_server_recipient_reply( + rcpt, 550, "5.1.1", + "Unacceptable TO: Empty path not allowed"); + return -1; + } smtp_address_detail_parse_temp( client->unexpanded_lda_set->recipient_delimiter, rcpt->path, &username, &delim, &detail); + i_assert(*username != '\0'); /* Make user name and detail available in the recipient event. The mail_user event (for local delivery) also adds the user field, but -- 2.11.0 ++++++ dovecot-2.3-pigeonhole-0.5.3.tar.gz -> dovecot-2.3-pigeonhole-0.5.10.tar.gz ++++++ ++++ 59412 lines of diff (skipped) ++++++ dovecot-2.3-pigeonhole-0.5.3.tar.gz -> dovecot-2.3.10.tar.gz ++++++ /work/SRC/openSUSE:Leap:15.2/dovecot23/dovecot-2.3-pigeonhole-0.5.3.tar.gz /work/SRC/openSUSE:Leap:15.2/.dovecot23.new.2738/dovecot-2.3.10.tar.gz differ: char 5, line 1
