[PATCH] DSNs to follow RFCs more closely

[Tassilo Philipp]

Hi,

Find below the first stab at a final patch making generated bounce 
mails follow more closely the RFCs 3461, 3464 and 6522. The patch 
includes the patch from my previous mail in this thread.


The patch is based on opensmtp 7.5.0rc1 (w/ the additional patch in 
the rc1 announcement thread on this ml).


It does, in short:

- uses the correct content type(s) for DSNs, as specified in RFC3464 and
  RFC6522

- adds xtext_decode() utility func (following the style/interface of the
  base64_decode() utility func)

- extends the data structures of the bounce message and envelope, as
  necesary, to write the required fields outlined by RFC3461 and RFC3464
  into the machine readable sub mime part, namely, per-message field:

 * Original-Envelope-Id: (required if ENVID= was used, see RFC3461)

 and per recipiend fields:

 * Original-Recipient: (required if ORCPT= was used, see RFC3461)
 * Arrival-Date: (optional, but added it while at it...)
 * Diagnostic-Code: (optional, but added it while at it...)

- adds a check whether the SMTP ENVID= param is valid xtext (RFC3461)

- minor: removes a space in already set Final-Recipient: field
  (following more closely the example layout from RFC3464)


One thing I wasn't sure about what to do is *where* to handle the xtext 
decoding of the ENVID= and ORCPT= SMTP params. Right now the SMTP 
handling part just considers them "opaque text", as they are just handed 
down the relays, unmodified. When copying those into a DSN, they must be 
decoded, though, and the decoded result is considered being printable 
US-ASCII. From RFC3461 4.2 about ORCPT:


 `Due to limitations in the Delivery Status Notification format, the
  value of the original recipient address prior to encoding as "xtext"
  MUST consist entirely of printable (graphic and white space)
  characters from the US-ASCII repertoire.`

A functional identical wording is used in 4.4 for the ENVID= param.

The patch below keeps the SMTP behaviour to just treat those params as 
opaque strings, but checks if the decoded part is US-ASCII when 
generating a bounce - and if it's not, doesn't write it to the 
respective DSN field(s).


I think the SMTP handling might be the better place to check for that, 
and refuse such SMTP params outright, maybe?


Thanks


--- ./usr.sbin/smtpd/bounce.c
+++ ./usr.sbin/smtpd/bounce.c
@@ -57,6 +57,9 @@
char*report;
uint8_t  esc_class;
uint8_t  esc_code;
+   char*errorline;
+   /* decoded xtext SMTP ORCPT param, len always <= encoded */
+   char orcpt[DSN_ORCPT_LEN+1];
 };

 struct bounce_message {
@@ -68,6 +71,9 @@
char*to;
time_t   timeout;
TAILQ_HEAD(, bounce_envelope)envelopes;
+   time_t   creation;
+   /* decoded xtext SMTP ENVID param, len always <= encoded */
+   char envid[DSN_ENVID_LEN+1];
 };

 struct bounce_session {
@@ -120,6 +126,18 @@
}
 }
 
+static int

+count_printable_ascii_chars(const char *s)
+{
+   const char *p = s;
+   for (; *p; ++p) {
+   if (!isprint((unsigned char)*p))
+   break;
+   }
+   return p - s;
+}
+
+
 void
 bounce_add(uint64_t evpid)
 {
@@ -127,6 +145,7 @@
struct envelope  evp;
struct bounce_messagekey, *msg;
struct bounce_envelope  *be;
+   int  n;

bounce_init();
 
@@ -164,6 +183,14 @@

msg->msgid = key.msgid;
msg->bounce = key.bounce;
 
+		msg->creation = evp.creation; 
+		n = xtext_decode(evp.dsn_envid, msg->envid, sizeof(msg->envid));

+   if (n == -1 || n != count_printable_ascii_chars(msg->envid)) {
+   msg->envid[0] = '\0';
+   log_debug("debug: bounce: ill-formed ENVID param (decoded 
xtext not "
+   "printable ascii, not setting DSN Original-Envelope-Id: 
header");
+   }
+
TAILQ_INIT(>envelopes);

msg->smtpname = xstrdup(evp.smtpname);
@@ -187,11 +214,18 @@
be = xmalloc(sizeof *be);
be->id = evpid;
be->report = xstrdup(buf);
+   be->errorline = xstrdup(evp.errorline);
(void)strlcpy(be->dest.user, evp.dest.user, sizeof(be->dest.user));
(void)strlcpy(be->dest.domain, evp.dest.domain,
sizeof(be->dest.domain));
be->esc_class = evp.esc_class;
be->esc_code = evp.esc_code;
+   n = xtext_decode(evp.dsn_orcpt, be->orcpt, sizeof(be->orcpt));
+   if (n == -1 || n != count_printable_ascii_chars(be->orcpt)) {
+   be->orcpt[0] = '\0';
+   log_debug("debug: bounce: ill-formed ORCPT param (decoded xtext not 
"
+   "printable ascii, not setting DSN 

[PATCH] DSNs to follow more closely RFCs

[Tassilo Philipp]

Hi,

Find attached the first stab at a final patch making generated bounce 
mails follow more closely the RFCs 3461, 3464 and 6522. The attached 
file includes the patch from my previous mail in this thread.


The patch is based on opensmtp 7.5.0rc1 (w/ the additional patch in the 
rc1 announcement thread on this ml).


It does, in short:

- uses the correct content type(s) for DSNs, as specified in RFC3464 and
  RFC6522

- adds xtext_decode() utility func (following the style/interface of the
  base64_decode() utility func)

- extends the data structures of the bounce message and envelope, as
  necesary, to write the required fields outlined by RFC3461 and RFC3464
  into the machine readable sub mime part, namely, per-message field:

  * Original-Envelope-Id: (required if ENVID= was used, see RFC3461)

  and per recipiend fields:

  * Original-Recipient: (required if ORCPT= was used, see RFC3461)
  * Arrival-Date: (optional, but added it while at it...)
  * Diagnostic-Code: (optional, but added it while at it...)

- adds a check whether the SMTP ENVID= param is valid xtext (RFC3461)

- minor: removes a space in already set Final-Recipient: field
  (following more closely the example layout from RFC3464)


One thing I wasn't sure about what to do is *where* to handle the xtext 
decoding of the ENVID= and ORCPT= SMTP params. Right now the SMTP 
handling part just considers them "opaque text", as they are just handed 
down the relays, unmodified. When copying those into a DSN, they must be 
decoded, though, and the decoded result is considered being printable 
US-ASCII. From RFC3461 4.2 about ORCPT:


  `Due to limitations in the Delivery Status Notification format, the
   value of the original recipient address prior to encoding as "xtext"
   MUST consist entirely of printable (graphic and white space)
   characters from the US-ASCII repertoire.`

A functional identical wording is used in 4.4 for the ENVID= param.

The attached patch keeps the SMTP behaviour to just treat those params 
as opaque strings, but checks if the decoded part is US-ASCII when 
generating a bounce - and if it's not, doesn't write it to the 
respective DSN field(s).


I think the SMTP handling might be the better place to check for that, 
and refuse such SMTP params outright, maybe?


Thanks



On Tue, Mar 19, 2024 at 04:22:13PM +0100, Tassilo Philipp wrote:
Alright, find attached a first patch, fixing up some content-type 
headers, as outlined by RFC3464 and RFC6522 - in detail:


The patch's first hunk follows RFC3464, which specifies that a DSN 
should have a top-level type of "multipart/report" with a parameter 
"report-type=delivery-status"; the actual format is described in 
RFC6522, and is specified having two or three sub mime parts, a human 
readable message, a machine parsable part with content-type 
"message/delivery-status" and an optional 3rd part of either 
"text/rfc822-headers" or "message/rfc822".


The latter is part of the second hunk of the patch, b/c 
"text/rfc822-headers" was used in any case, even if the full message 
was returned in the DSN.


Also, the end of the second hunk removes a check, which I think is 
unintuitive to begin with, but unsure. Basically, it only returned the 
headers for NOTIFY=SUCCESS DSNs when RET=HDRS was also set (explicitly 
or implicitly).

Here's the section from RFC3461:

   When the value of the RET parameter is FULL, the full
   message SHOULD be returned for any DSN which conveys notification of
   delivery failure.  (However, if the length of the message is greater
   than some implementation-specified length, the MTA MAY return only
   the headers even if the RET parameter specified FULL.)  If a DSN
   contains no notifications of delivery failure, the MTA SHOULD return
   only the headers.

The original code does what the last line implies, but only when 
RET=HDRS is set, meaning it ignores it for anything but 
NOTIFY=SUCCESS, and returns always the full message.


I'm not sure what would be best to do here, all in all the RFC says 
'SHOULD'... either way, the check there should reflect the same logic 
used in the if...else... block of the second hunk, which is what the 
patch shoots for. I chose to keep it simple and simply do what RET= 
asks for.  So, if the original logic is kept, the second hunk's 
if...else... block would need an additional "s->msg->bounce.type == 
B_DELIVERED" check.


I tested the patch with different combinations of the RET= and NOTIFY= 
parameters, and so far it seems to work fine.


PS: I'm working on related other patches, at the moment, which I'll 
mail when they are done, namely adding a "Original-Envelope-ID" header 
in the DSN when the ENVID= param was present, as well as 
"Original-Recipient" for any ORCPT= param. Those are specified in 
RFC3461 section 6.3., but they are a bit more involved as the params 
need to be decoded a

Re: How to terminate smtpd filters?

[Tassilo Philipp]

I agree with Gilles, your filter should react on stdin closing, but not 
sure how your filter is set up.


Also, just a guess... are you running smtpd on Linux?

Linux doesn't kill children when the parent process dies, maybe that's
related? (To make it do that prctl(2) would need to be used w/
PR_SET_PDEATHSIG.)


On Tue, Apr 02, 2024 at 06:53:39AM +, gil...@poolp.org wrote:

April 2, 2024 4:47 AM, and...@tekrealm.net wrote:


What signals a termination for smtpd filters?

I'm using the code at 
https://github.com/Beutlin/howto-opensmtpd-filters-and-reports, 
Which works great, except for when smtpd gets shutdown. The script continues 
to run and
consumes up to 100% cpu time, while keeping the smtpd parent? process 
running.


I've tried adding a SIGTERM handler to the code which didn't work, as well 
as I saw
mentioned that the filter should exit on EOF, so I tried wrapping 
parser.dispatch() in

a try/except EOFError and using sys.exit. That didn't work either.

I've read smtpd-filters, and looked at various other filters and I am not 
understanding

what tells the filter to shutdown.



The filter is connected to smtpd through its stdin, so it can terminate when 
there's an EOF on stdin.


This is the proper way to do it and how all filters I wrote work but maybe a 
bug has crawled in the handling of filter termination and it went unnoticed, 
I don´t think I ever terminated smtpd in years besides system restarts.


What system are you running on ?

Re: [PATCH] Re: OpenSMTPD 7.5.0 RC1

[Tassilo Philipp]

Perfect, thanks! You are right, this didn't copy anything. I just 
noticed it now, b/c I need that param for the DSN work I'm currently 
working on.


And sorry for not spotting this earlier, when I tested that final 
version of the ORCPT patch, a while ago.


:)


On Wed, Mar 20, 2024 at 07:08:40PM +0100, Omar Polo wrote:

On 2024/03/20 17:36:01 +0100, Tassilo Philipp  
wrote:

Hi,

while working on the DSN patches mentioned in another thread, I came 
across an oversight in the final ORCPT patch that will be part of 7.5.0.


Find the patch attached - IMHO, this patch should make it into 7.5.0, as 
it's fixing an error writing to a wrong buffer, which could be abused 
(from a cursory review it looks safe as that wrong destination buffer 
big enough, but I haven't checked it thoroughly).


Thanks for spotting!  This has been committed and will be included in 
7.5 (both OpenBSD and -portable.)


I don't think this can be abused since the dsn_orcpt buffer is zeroed, 
so we're just going to truncate `opt', that we won't look at it again. 
In any case, this had to be fixed.

[PATCH] Re: OpenSMTPD 7.5.0 RC1

[Tassilo Philipp]

Hi,

while working on the DSN patches mentioned in another thread, I came 
across an oversight in the final ORCPT patch that will be part of 7.5.0.


Find the patch attached - IMHO, this patch should make it into 7.5.0, as 
it's fixing an error writing to a wrong buffer, which could be abused 
(from a cursory review it looks safe as that wrong destination buffer 
big enough, but I haven't checked it thoroughly).



On Fri, Mar 08, 2024 at 11:14:54AM +0100, Omar Polo wrote:
Today we're happy to announce the first release candidate for 
OpenSMTPD 7.5.0 that includes the latest developments on OpenBSD, 
briefly summarized below.  The 7.5.0 stable release will follow the 
OpenBSD release schedule and so will be available in around a month. 
It is our hope that the community will help testing this first 
release candidate and report any issue found.


Tarballs are available on the official mirror or on GitHub:

   https://opensmtpd.org/archives/opensmtpd-7.5.0rc1.tar.gz
   https://github.com/OpenSMTPD/OpenSMTPD/releases/tag/7.5.0rc1

Verify the tarball with signify(1) and the usual public key:

   https://opensmtpd.org/archives/opensmtpd-7.5.0rc1.sum.sig
   https://opensmtpd.org/archives/opensmtpd-20181026.pub


Changelog:

- run LMTP deliveires as the recipient user (again).
- do not execute commands from root's .forward file, nor allow expanding. 
- when an alternate delivery user is provided for a dispatcher, skip
 other users forward files. 
- reject invalid headers that start with blanks.

- relax ORCPT syntax validation.
- use smtpd' table parser in makemap(8) too.
- fix and improve the table(5) file format documentation.
- fixed handling of escaping inside quotes in From, To and Cc headers. 
- fix table lookups of IPv6 address.

- allow to use a key-pair table on various match constraints where only
 list tables were previously allowed. 
- allow inline tables and filter to span over multiple lines.
- enable DSN (Delivery Status Notification) for the implicit socket too. 
- add the `no-dsn' option to `listen on socket' too.


OpenSMTPD-portable specific changes:

- re-add ASR_IPV4_BEFORE_IPV6 compile-time knob to prefer connecting
 to IPv6 instead of IPv4. 
- update asr(3) and imsg with OpenBSD.

- configure: readd -R usage on NetBSD mistakenly dropped in previous
 release.



Thanks,

Omar Polo

--- ./usr.sbin/smtpd/smtp_session.c.orig2024-03-20 17:27:12.280717000 
+0100
+++ ./usr.sbin/smtpd/smtp_session.c 2024-03-20 17:27:18.240157000 +0100
@@ -2496,7 +2496,7 @@
 
if ((p = strchr(opt, ';')) == NULL ||
!valid_xtext(p + 1) ||
-   strlcpy(opt, tx->evp.dsn_orcpt, len) >= len) {
+   strlcpy(tx->evp.dsn_orcpt, opt, len) >= len) {
smtp_reply(tx->session,
"553 ORCPT address syntax error");
return;

Re: DSN message format: shouldn't this use multipart/report (as of RFC3464)

[Tassilo Philipp]

Alright, find attached a first patch, fixing up some content-type 
headers, as outlined by RFC3464 and RFC6522 - in detail:


The patch's first hunk follows RFC3464, which specifies that a DSN 
should have a top-level type of "multipart/report" with a parameter 
"report-type=delivery-status"; the actual format is described in 
RFC6522, and is specified having two or three sub mime parts, a human 
readable message, a machine parsable part with content-type 
"message/delivery-status" and an optional 3rd part of either 
"text/rfc822-headers" or "message/rfc822".


The latter is part of the second hunk of the patch, b/c 
"text/rfc822-headers" was used in any case, even if the full message was 
returned in the DSN.


Also, the end of the second hunk removes a check, which I think is 
unintuitive to begin with, but unsure. Basically, it only returned the 
headers for NOTIFY=SUCCESS DSNs when RET=HDRS was also set (explicitly 
or implicitly).

Here's the section from RFC3461:

When the value of the RET parameter is FULL, the full
message SHOULD be returned for any DSN which conveys notification of
delivery failure.  (However, if the length of the message is greater
than some implementation-specified length, the MTA MAY return only
the headers even if the RET parameter specified FULL.)  If a DSN
contains no notifications of delivery failure, the MTA SHOULD return
only the headers.

The original code does what the last line implies, but only when 
RET=HDRS is set, meaning it ignores it for anything but NOTIFY=SUCCESS, 
and returns always the full message.


I'm not sure what would be best to do here, all in all the RFC says 
'SHOULD'... either way, the check there should reflect the same logic 
used in the if...else... block of the second hunk, which is what the 
patch shoots for. I chose to keep it simple and simply do what RET= asks 
for.  So, if the original logic is kept, the second hunk's if...else... 
block would need an additional "s->msg->bounce.type == B_DELIVERED" 
check.


I tested the patch with different combinations of the RET= and NOTIFY= 
parameters, and so far it seems to work fine.


PS: I'm working on related other patches, at the moment, which I'll mail 
when they are done, namely adding a "Original-Envelope-ID" header in the 
DSN when the ENVID= param was present, as well as "Original-Recipient" 
for any ORCPT= param. Those are specified in RFC3461 section 6.3., but 
they are a bit more involved as the params need to be decoded and 
recoded, and I just haven't found the time, yet.


Thanks!




On Wed, Mar 13, 2024 at 12:04:24PM +0100, Tassilo Philipp wrote:
Ok, will get busy... I also noticed two other issues, one related to 
ENVID= (should be generated for DSNs), and some data in the DSN's 
message/delivery-status parts, that must be generated according to eh 
RFC, when ENVID= or ORCPT= are present, which are not, currently. Will 
look at those, as well. Might take a few days, though... will keep you 
posted.


Cheers

On Wed, Mar 13, 2024 at 11:00:51AM +, gil...@poolp.org wrote:

March 13, 2024 10:31 AM, "Tassilo Philipp"  wrote:


Hello,

I noticed that DSNs generated by OpenSMTPd use "Content-Type: 
multipart/mixed", instead of "Content-Type: multipart/report", as 
defined by RFC3461 (and described in RFC3464 and RFC3462). I 
wonder if there's a reason for that?


I haven only done a cursory review of the actual multiparts 
themselves, but they seem to (mostly) follow the mentioned RFCs 
with for example "Content-Type: text/rfc822-headers". However, if 
RET=FULL, the content type should be IMHO only "message/rfc822" 
and not "text/rfc822-headers", the latter seems to be currently 
used for both, RET=HDRS and RET=FULL. Need to read up on it more 
to get a clearer picture, though, I just started digging into it.


Either way... I think the RFCs should be followed, want me to prepare a patch?



Yes please, patch and RFC reference so this can be checked too

Thanks,


--- ./usr.sbin/smtpd/bounce.c
+++ ./usr.sbin/smtpd/bounce.c
@@ -452,7 +452,8 @@
 		"To: %s\r\n"
 		"Date: %s\r\n"
 		"MIME-Version: 1.0\r\n"
-		"Content-Type: multipart/mixed;"
+		"Content-Type: multipart/report;"
+			"report-type=delivery-status;"
 		"boundary=\"%16" PRIu64 "/%s\"\r\n"
 		"\r\n"
 		"This is a MIME-encapsulated message.\r\n"
@@ -534,19 +535,27 @@
 		break;
 
 	case BOUNCE_DATA_MESSAGE:
-		io_xprintf(s->io,
-		"--%16" PRIu64 "/%s\r\n"
-		"Content-Description: Message headers\r\n"
-		"Content-Type: text/rfc822-headers\r\n"
-		"\r\n",
-		s->boundary, s->smtpname);
+		if(s->msg->bounce.dsn_ret == DS

Re: DSN message format: shouldn't this use multipart/report (as of RFC3464)

[Tassilo Philipp]

Ok, will get busy... I also noticed two other issues, one related to 
ENVID= (should be generated for DSNs), and some data in the DSN's 
message/delivery-status parts, that must be generated according to eh 
RFC, when ENVID= or ORCPT= are present, which are not, currently. Will 
look at those, as well. Might take a few days, though... will keep you 
posted.


Cheers

On Wed, Mar 13, 2024 at 11:00:51AM +, gil...@poolp.org wrote:

March 13, 2024 10:31 AM, "Tassilo Philipp"  wrote:


Hello,

I noticed that DSNs generated by OpenSMTPd use "Content-Type: multipart/mixed", instead of 
"Content-Type: multipart/report", as defined by RFC3461 (and described in RFC3464 and RFC3462). I 
wonder if there's a reason for that?


I haven only done a cursory review of the actual multiparts themselves, but they seem to (mostly) 
follow the mentioned RFCs with for example "Content-Type: text/rfc822-headers". However, if 
RET=FULL, the content type should be IMHO only "message/rfc822" and not "text/rfc822-headers", the 
latter seems to be currently used for both, RET=HDRS and RET=FULL. 
Need to read up on it more to get a clearer picture, though, I just started digging into it.


Either way... I think the RFCs should be followed, want me to prepare a patch?



Yes please, patch and RFC reference so this can be checked too

Thanks,

DSN message format: shouldn't this use multipart/report (as of RFC3464)

[Tassilo Philipp]

Hello,

I noticed that DSNs generated by OpenSMTPd use "Content-Type: 
multipart/mixed", instead of "Content-Type: multipart/report", as 
defined by RFC3461 (and described in RFC3464 and RFC3462). I wonder if 
there's a reason for that?


I haven only done a cursory review of the actual multiparts themselves, 
but they seem to (mostly) follow the mentioned RFCs with for example 
"Content-Type: text/rfc822-headers". However, if RET=FULL, the content 
type should be IMHO only "message/rfc822" and not "text/rfc822-headers", 
the latter seems to be currently used for both, RET=HDRS and RET=FULL.  

Need to read up on it more to get a clearer picture, though, I just 
started digging into it.


Either way... I think the RFCs should be followed, want me to prepare a 
patch?


I came across this, b/c I have an edge case where I need to filter
DSN (don't ask, some client's requirement), and it would be much simpler 
to identify such mails with that Content-Type: multipart/report.

Re: Run VM with 16G or more?

[Tassilo Philipp]

Wrong ML? This is OpenSMTPD


On Sun, Dec 31, 2023 at 07:22:04PM +0100, Kirill A. Korinsky wrote:

Greetings,

How can I run a VM with more than 16G of memory?

A naive approach fails with error:


vmctl: start vm command failed: Cannot allocate memory



Yes, the host machine has that memory and much more.

--
wbr, Kirill

Re: [PATCH] relax 553 ORCPT address syntax error (was Re: EMails to "ORCPT=rfc822;u...@example.co

[Tassilo Philipp]

Thanks for looking into this! Also thanks for taking this a step further 
and removing unneeded complexity, given the train of thought to consider 
this param an opaque string.


I tried the patch in a real world scenario, having rolled it out to 
multiple machines and relays, ran manual tests with this parameter, and 
I didn't spot any issues, so far.




On Mon, Dec 11, 2023 at 08:56:08PM +0100, Omar Polo wrote:

[moving to tech@]

This seems to be a long-standing issue in opensmtpd.  There seem to be 
(at least) one fairly popular application that uses a, upon a first 
look, odd ORCPT parameter:



On Fri, Oct 28, 2022 at 04:16:36PM +0200, Tim Kuijsten wrote:
I have refined and more thoroughly tested a previous patch that 
relaxes the ORCPT address check.


Over the years mail has been rejected by senders that use RCPT 
TO commands like:

RCPT TO:
ORCPT=rfc822;groupwise-i...@example.com:0:0 or RCPT 
TO:
ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE


In the above the domain part of the ORCPT address resolves to 
example.com:0:0 which is rejected by smtpd with the message:
smtpd[20797]: 1a3a396cd4c57d05 smtp failed-command command="RCPT 
TO:
ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE" result="553 ORCPT address syntax error"


[...]


Quoting RFC 3461, the ORCPT parameter is defined as (¶ 4.2)

 orcpt-parameter = "ORCPT=" original-recipient-address
 original-recipient-address = addr-type ";" xtext
 addr-type = atom

  The "addr-type" portion MUST be an IANA-registered electronic mail
  address-type (as defined in [3]), while the "xtext" portion contains
  an encoded representation of the original recipient address using the
  rules in section 5 of this document.

  [...]

  The "addr-type" portion of the original-recipient-address is used to
  indicate the "type" of the address which appears in the ORCPT
  parameter value.  However, the address associated with the ORCPT
  keyword is NOT constrained to conform to the syntax rules for that
  "addr-type".

I'd like to emphasize the last sentence, which is the one I missed 
upon the first couple of look at this RFC: even if addr-type is 
rfc822, the xtext part (once decoded) is not constrained to conform to 
the syntax rules of rfc822.


That is, this groupwise thingy is actually confirming to the rfc when 
using a param of "rfc822;groupwise-i...@example.com:0:0".


Honesty, I fail to understand the meaning of the addr-type if any 
gargabe is allowed.


The "once decoded" part is important since the RFC allows for ANY 
character to be encoded, so rfc822;+45+78+62+6d+70+6c+65+40... 
(i.e. example@...) is still a valid ORCPT param.


Tassilo' diff (which is a slightly modified version of the one 
originally sent in the bug report by Tim) is available here


https://www.mail-archive.com/misc@opensmtpd.org/msg06054.html

and it relaxes the checks but IMHO it doesn't address the underlying 
issue: we expect a valid rfc822 address where it's not mandatory.


I think we should just keep the ORCPT address as on opaque string 
(modulo some validation) without trying to parse it.


I still have to test the diff below in a real-world scenario, but I'm 
sending it out so other can hopefully give it a spin.


(and I couldn't resist splitting a long line in the process.)


Thanks,

Omar Polo


diff /home/op/w/smtpd
commit - 45a7eac758c7b1e9c4f16ab2dfcb25672b49aad9
path + /home/op/w/smtpd
blob - c9611beb48feab47602b766061a7546c375160a8
file + envelope.c
--- envelope.c
+++ envelope.c
@@ -443,7 +443,8 @@ ascii_load_field(const char *field, struct envelope *e 
		return ascii_load_uint8(>dsn_notify, buf);


if (strcasecmp("dsn-orcpt", field) == 0)
-   return ascii_load_mailaddr(>dsn_orcpt, buf);
+   return ascii_load_string(ep->dsn_orcpt, buf,
+   sizeof(ep->dsn_orcpt));

if (strcasecmp("dsn-ret", field) == 0)
return ascii_load_dsn_ret(>dsn_ret, buf);
@@ -703,11 +704,8 @@ ascii_dump_field(const char *field, const struct envel 
	if (strcasecmp(field, "dsn-ret") == 0)

return ascii_dump_dsn_ret(ep->dsn_ret, buf, len);

-   if (strcasecmp(field, "dsn-orcpt") == 0) {
-   if (ep->dsn_orcpt.user[0] && ep->dsn_orcpt.domain[0])
-   return ascii_dump_mailaddr(>dsn_orcpt, buf, len);
-   return 1;
-   }
+   if (strcasecmp(field, "dsn-orcpt") == 0)
+   return ascii_dump_string(ep->dsn_orcpt, buf, len);

if (strcasecmp(field, "dsn-envid") == 0)
return ascii_dump_string(ep->dsn_envid, buf, len);
blob - f0bb42c53ffb8f98012d542209bb55ecd1ae
file + mta.c
--- mta.c
+++ mta.c
@@ -809,11 +809,8 @@ mta_handle_envelope(struct envelope *evp, const char * 
	if (strcmp(buf, e->dest))

e->rcpt = xstrdup(buf);
e->task = task;
-   if (evp->dsn_orcpt.user[0] && evp->dsn_orcpt.domain[0]) {
-   (void)snprintf(buf, sizeof buf, 

Re: Limit messages sent per unit time?

[Tassilo Philipp]

mta limit session-transaction-delay 10 
Is there any way to limit where this is implemented? Such as only 
sessions from a specific server (via tag, action, match, etc.)?


From my limited understanding of the few bits of code I read, around 
this and other mta options, those "limit" options seems to be global.


There are other (global) knobs that maybe do something to the scheduling 
per domain or host, but I haven't tried any of those and I think they 
rather apply to retries. Check out ./usr.sbin/smtpd/limit.c, function 
limit_mta_set().


I ran a few rough grep(1)s through the sources, by guessing related 
strings, but I didn't find anything related to action or match.



My current understanding is we could consider some different levels of 
compromise:


1. Only the service is compromised (e.g., PHP) - this would limit 
message sending to how the server is configured to send mail.

[...]
at least in the event an attacker only compromises a service I can 
still limit damage to the mail relay server IP address reputation.


Mh... good thinking, I agree. I appreciate you sharing your reasoning :)

Re: smtpd.conf questions

[Tassilo Philipp]

Not sure if this applies to you, but I just gave an answer to Paul 
Pace's "Limit messages sent per unit time?" question on this mailing 
list, which might also help you.



On Tue, Dec 05, 2023 at 10:32:20PM -0800, Sean Kamath wrote:

Hi.

I have a couple of (hopefully quick) questions about some max settings in 
smtpd.conf

Recently, I tried to email 13 email addresses on Google, and they throttled me. 
 I found that scheduling each envelop in the “mass mailing” individually went 
through just fine.  So I’m wondering if I can make opensmtpd send at most N 
messages to google (but allowing the other mails to be sent at a “safe and 
sane” rate of the defaults).

I found these three options, none of which I think will do what I’m hoping to 
do, namely set a max number of envelopes to be sent to a given MX at one time 
(I’ve had other MXes say “You’re sending too many recipients in one go” before, 
so it’s not just The Goog.)

mta max-deferred number
When delivery to a given host is suspended due to temporary
failures, cache at most number envelopes for that host such that
they can be delivered as soon as another delivery succeeds to
that host.  The default is 100.

I’m unclear on what happens to messages over the number?  Does it attempt to 
batch, at most, “number” envelopes at a time?  This kinda sounds like what I 
want, except I was limited at the very beginning, so there were no successful 
deliveries to later try.

smtp limit max-mails count
Limit the number of messages to count for each session.  The
default is 100.

It sounds like this is an incoming limit, where opensmtpd will accept at most 
count messages on one session, so no help here.

smtp limit max-rcpt count
Limit the number of recipients to count for each transaction.
The default is 1000.

This also sounds like an incoming limit, allowing up to “count” recipients to a 
single message.  Again, no help here.

Thus the only settings with “max” in them don’t appear to do what I need.  
Either I’m missing something, or some code would need to be written.

Sean

Re: Limit messages sent per unit time?

[Tassilo Philipp]

So your question made me curious to read more of the source... and there 
are actually multiple undocumented knobs, and there might be this one 
that maybe works for you, e.g. put this in you smtpd.conf for a minimum 
of 10s delay between MTA transactions:


mta limit session-transaction-delay 10

This would not start any new transaction before 10s passed since the 
last one. I just gave it a try - submit n mails, observe that the first 
one goes out right away, watch via smtpctl show queue that the other 
ones are delayed (or use the tracing and debug flags when running 
smtpd). If nothing was scheduled for longer than that, any new one gets 
sent right away.


There are quite a few more knobs, unsure why they aren't documented but 
this might be intentional, as you can probably seriously shoot yourself 
in the foot changing some of those.


Hope this helps, but no guarantees there are no side effects.

PS: can't help but point out that if I understand your use case 
correctly, this won't really prevent abuse, b/c if your system is 
compromised, the attacker might also just modify the config, no?




On Sat, Dec 09, 2023 at 07:09:16AM -0800, Paul Pace wrote:

On 2023-12-06 19:47, Paul Pace wrote:

Is there a way to limit messages sent per unit time?


Maybe it's just not possible in OpenSMTPD?

My use case is I have a small cluster of servers that use a single 
server as an SMTP relay. One of the servers hosts stuffs that have a 
somewhat higher probability of being compromised, so one method I 
would like to use to limit the potential damage done by a compromised 
server is to rate limit messages sent, ideally by some number of 
messages per unit time (probably 1 per 10 seconds, since it isn't 
expected to send much mail).


Thank you,


Paul

Re: [PATCH] relax 553 ORCPT address syntax error (was Re: EMails to "ORCPT=rfc822;u...@example.com" are rejected)

[Tassilo Philipp]

Sorry, some unnecessary whitespace change was part of the attached 
patch. Find attached a cleaned up version of the patch (functionally the 
same).


On Thu, Dec 07, 2023 at 01:34:42PM +0100, Tassilo Philipp wrote:
After a direct exchange with Omar Polo about the ORCPT patch, find 
attached a revised version of it. The changes compared to Tim's last 
patch are, just fyi:


1) valid_xtext() has been modified to avoid potential problems with 
sign extensions when using the ctype is*() functions. I also narrowed 
down the checks for hexchar letters to only allow ABCDEF (previously 
it just checked if it's an uppercase letter). A superflous string 
length check on hexchars was removed, b/c this is implicitly handled 
when checking the characters ('\0' fails the isdigit() check and 
related). While at it I reformatted a bit to be less verbose.


2) I reintroduced the check whether the entire ORCPT parameter exceeds 
the rfc3461 specified 500 char limit that Tim had in his previous 
patch.


3) I relaxed the check whether the ORCPT param is a valid mail 
address, which was done by text_to_mailaddr(). The reason is that it 
simply doesn't apply to the ORCPT parameter, as it's xtext encoded (so 
you could for example write '@' - which that function tries to find - 
as '+40'). It might also be a different address type altogether (e.g. 
X.400 as given as example by the rfc), etc.. so that 
text_to_mailaddr() logic might simply not apply to that param. So 
basically what's left is using that function only to see whether it's 
an uncommon ORCPT param (and printing that to the log), or not.


In theory, the xtext would have to be decoded, first, to be pendantic 
about this, which I skipped b/c it won't make a difference in this 
case for deliverability... not sure if this all should be simplified 
even more to just check for xtext, and that's it, not writing any 
"uncommon ORCPT param" to the log at all. In any case we leave the 
ORCPT param untouched. Thoughts?


4) The patch is now rebased against the latest version of opensmtpd.


Patch is attached

Cheers


On Sat, Nov 18, 2023 at 11:54:33AM +0100, Tassilo Philipp wrote:

Sorry for another bump of this patch: can it be merged?

I know the groupwise example in this thread is rare and doesn't 
affect a lot of smtpd users, but without it, it's unfortunately a 
bit of a blocker for some people. We personally apply this patch to 
our opensmtpd builds, but for others this might simply be a blocker 
to use opensmtpd, as they cannot control e.g. what a client uses to 
send them mail.


Such mails get rejected by any MTA on a route, no matter if the 
rejecting server is just a helper relay. The weird ORCP format used 
by groupwise is adhering to the RFC xtext spec, after all, and is 
valid.


If this cannot/won't be merged, please share the reasons for why not.

Thank you


On Thu, Jul 20, 2023 at 09:58:16AM +0200, Tassilo Philipp wrote:
Sorry to shamelessly "bump" this, but any way to get this 
integrated into upstream, eventually?


We used the original patch from Frank Scholl and then this 
improved one in production now for like a year, now, and didn't 
experience issues. In our case it is specifically needed for a 
client that uses GroupWise[0] internally to send mails (which 
seems to always generate mails with an "xtext" ORCPT param).


Thanks!

[0] https://www.microfocus.com/products/groupwise/


On Fri, Oct 28, 2022 at 04:16:36PM +0200, Tim Kuijsten wrote:
I have refined and more thoroughly tested a previous patch that 
relaxes the ORCPT address check.


Over the years mail has been rejected by senders that use RCPT 
TO commands like:
RCPT TO: 
ORCPT=rfc822;groupwise-i...@example.com:0:0 or RCPT 
TO: 
ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE


In the above the domain part of the ORCPT address resolves to 
example.com:0:0 which is rejected by smtpd with the message:
smtpd[20797]: 1a3a396cd4c57d05 smtp failed-command command="RCPT 
TO: 
ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE" result="553 ORCPT address syntax error"


I've studied RFC 3461 section 4 and 4.2 but it's not entirely 
clear to me if the above ORCPT command is valid or not. The 
encoding adheres to the spec, which says it must be valid xtext.


With this patch smtpd accepts any ORCPT that is valid xtext as 
defined in the RFC (and logs on informational message when it 
consists of an invalid user or domain name).


Cheers,

Tim

---
usr.sbin/smtpd/smtp_session.c | 22 ++ 
usr.sbin/smtpd/smtpd.h|  1 +
usr.sbin/smtpd/util.c | 32 
 3 files changed, 51 
insertions(+), 4 deletions(-)


diff --git a/usr.sbin/smtpd/smtp_session.c 
b/usr.sbin/smtpd/smtp_session.c index 72e13e8fd8d..c0c29d4a695 
100644

--- a/usr.sbin/smtpd/smtp_session.c
+++ b/usr.sbin/smtpd/smtp_session.c
@@ -2415,6 +2415,7 @@ smtp_tx_create_message(struct smtp_tx *tx) 
stat

Re: [PATCH] relax 553 ORCPT address syntax error (was Re: EMails to "ORCPT=rfc822;u...@example.com" are rejected)

[Tassilo Philipp]

After a direct exchange with Omar Polo about the ORCPT patch, find 
attached a revised version of it. The changes compared to Tim's last 
patch are, just fyi:


1) valid_xtext() has been modified to avoid potential problems with sign 
extensions when using the ctype is*() functions. I also narrowed down 
the checks for hexchar letters to only allow ABCDEF (previously it just 
checked if it's an uppercase letter). A superflous string length check 
on hexchars was removed, b/c this is implicitly handled when checking 
the characters ('\0' fails the isdigit() check and related). While at it 
I reformatted a bit to be less verbose.


2) I reintroduced the check whether the entire ORCPT parameter exceeds 
the rfc3461 specified 500 char limit that Tim had in his previous patch.


3) I relaxed the check whether the ORCPT param is a valid mail address, 
which was done by text_to_mailaddr(). The reason is that it simply 
doesn't apply to the ORCPT parameter, as it's xtext encoded (so you 
could for example write '@' - which that function tries to find - as 
'+40'). It might also be a different address type altogether (e.g. X.400 
as given as example by the rfc), etc.. so that text_to_mailaddr() logic 
might simply not apply to that param. So basically what's left is using 
that function only to see whether it's an uncommon ORCPT param (and 
printing that to the log), or not.


In theory, the xtext would have to be decoded, first, to be pendantic 
about this, which I skipped b/c it won't make a difference in this case 
for deliverability... not sure if this all should be simplified even 
more to just check for xtext, and that's it, not writing any "uncommon 
ORCPT param" to the log at all. In any case we leave the ORCPT param 
untouched. Thoughts?


4) The patch is now rebased against the latest version of opensmtpd.


Patch is attached

Cheers


On Sat, Nov 18, 2023 at 11:54:33AM +0100, Tassilo Philipp wrote:

Sorry for another bump of this patch: can it be merged?

I know the groupwise example in this thread is rare and doesn't affect 
a lot of smtpd users, but without it, it's unfortunately a bit of a 
blocker for some people. We personally apply this patch to our 
opensmtpd builds, but for others this might simply be a blocker to use 
opensmtpd, as they cannot control e.g. what a client uses to send them 
mail.


Such mails get rejected by any MTA on a route, no matter if the 
rejecting server is just a helper relay. The weird ORCP format used by 
groupwise is adhering to the RFC xtext spec, after all, and is valid.


If this cannot/won't be merged, please share the reasons for why not.

Thank you


On Thu, Jul 20, 2023 at 09:58:16AM +0200, Tassilo Philipp wrote:
Sorry to shamelessly "bump" this, but any way to get this integrated 
into upstream, eventually?


We used the original patch from Frank Scholl and then this improved 
one in production now for like a year, now, and didn't experience 
issues. In our case it is specifically needed for a client that uses 
GroupWise[0] internally to send mails (which seems to always 
generate mails with an "xtext" ORCPT param).


Thanks!

[0] https://www.microfocus.com/products/groupwise/


On Fri, Oct 28, 2022 at 04:16:36PM +0200, Tim Kuijsten wrote:
I have refined and more thoroughly tested a previous patch that 
relaxes the ORCPT address check.


Over the years mail has been rejected by senders that use RCPT TO 
commands like:
RCPT TO: 
ORCPT=rfc822;groupwise-i...@example.com:0:0 or RCPT 
TO: ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE


In the above the domain part of the ORCPT address resolves to 
example.com:0:0 which is rejected by smtpd with the message:
smtpd[20797]: 1a3a396cd4c57d05 smtp failed-command command="RCPT 
TO: ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE" result="553 ORCPT address syntax error"


I've studied RFC 3461 section 4 and 4.2 but it's not entirely 
clear to me if the above ORCPT command is valid or not. The 
encoding adheres to the spec, which says it must be valid xtext.


With this patch smtpd accepts any ORCPT that is valid xtext as 
defined in the RFC (and logs on informational message when it 
consists of an invalid user or domain name).


Cheers,

Tim

---
usr.sbin/smtpd/smtp_session.c | 22 ++ 
usr.sbin/smtpd/smtpd.h|  1 +
usr.sbin/smtpd/util.c | 32 
 3 files changed, 51 
insertions(+), 4 deletions(-)


diff --git a/usr.sbin/smtpd/smtp_session.c 
b/usr.sbin/smtpd/smtp_session.c index 72e13e8fd8d..c0c29d4a695 
100644

--- a/usr.sbin/smtpd/smtp_session.c
+++ b/usr.sbin/smtpd/smtp_session.c
@@ -2415,6 +2415,7 @@ smtp_tx_create_message(struct smtp_tx *tx) 
static void

smtp_tx_rcpt_to(struct smtp_tx *tx, const char *line)
{
+   struct mailaddr orcptaddr;
 char *opt, *p;
 char *copy;
 char tmp[SMTP_LINE_MAX]; @@ -2469,10 +2470,23 @@ 
sm

Re: Silently reject mails

[Tassilo Philipp]

Since you mentioned "all mails" and "without creating non deliverable 
notices", I think it's worth pointing out that the mail server will 
probably send delivery status notifications for successfully delivered 
mails, if the sender requested it, so also look for the no-dsn option 
you can set on the listener.


On Wed, Dec 06, 2023 at 09:43:22PM +0100, Omar Polo wrote:

On 2023/12/06 20:17:49 +0100, Hagen Bauer  wrote:

hi,

i have not found a way to silently drop all mails to a domain. (yes 
there is a use case to run a mail server that does not store any 
mails without creating non deliverable notices)


Is there a way to configure opensmpt for such a use case?


Haven't tried, but a mda command that writes to to /dev/null should be 
a way.


(not tested)

action "ignore" [...] mda "cat >/dev/null"


Cheers,

Omar Polo

[patch] default socket listener not advertising DSNs (but explicit ones do)

[Tassilo Philipp]

Hi,

I stumbled across an issue when trying to send mail with opensmtpd by 
submitting via the unix domain socket, when requesting any kind of DSN, 
(e.g. mailwrapper(8) based sendmail command's -N param).


Such mail gets refused by smtpd when the unix domain socket in use is 
the default one, instead of an explicitly specified one in 
smtpd.conf(5).

Explicitly specifying 'listen on socket' in the config works.

The man page implies that having this listen line in the config or not 
should come down to the same, but that isn't the case. The default 
creates a socket that doesn't advertise any DSN extension, whereas 
specifying it explicitly, does.


At first I thought this might be intentional, however, with an explicit 
config, it's not possible to disable DSNs, as the 'listen on socket' 
line doesn't support any no-dsn option, which exists for other listeners.


Find attached a one-line patch that makes the socket created by default 
behave the same way as when specifying it via the config, explicitly. I 
think the no-dsn option should also be available for such listeners, 
however I don't know exactly if this is wanted, so I left that out for 
now.


Thanks

--- ./usr.sbin/smtpd/parse.y.orig   2023-11-28 14:12:34.578898000 +0100
+++ ./usr.sbin/smtpd/parse.y2023-11-28 14:12:41.349092000 +0100
@@ -3136,6 +3136,7 @@
/* If the socket listener was not configured, create a default one. */
if (!conf->sc_sock_listener) {
memset(_opts, 0, sizeof listen_opts);
+   listen_opts.flags |= F_EXT_DSN;
create_sock_listener(_opts);
}
 

Re: [PATCH] relax 553 ORCPT address syntax error (was Re: EMails to "ORCPT=rfc822;u...@example.com" are rejected)

[Tassilo Philipp]

Sorry for another bump of this patch: can it be merged?

I know the groupwise example in this thread is rare and doesn't affect a 
lot of smtpd users, but without it, it's unfortunately a bit of a 
blocker for some people. We personally apply this patch to our opensmtpd 
builds, but for others this might simply be a blocker to use opensmtpd, 
as they cannot control e.g. what a client uses to send them mail.


Such mails get rejected by any MTA on a route, no matter if the 
rejecting server is just a helper relay. The weird ORCP format used by 
groupwise is adhering to the RFC xtext spec, after all, and is valid.


If this cannot/won't be merged, please share the reasons for why not.

Thank you


On Thu, Jul 20, 2023 at 09:58:16AM +0200, Tassilo Philipp wrote:
Sorry to shamelessly "bump" this, but any way to get this integrated 
into upstream, eventually?


We used the original patch from Frank Scholl and then this improved 
one in production now for like a year, now, and didn't experience 
issues. In our case it is specifically needed for a client that uses 
GroupWise[0] internally to send mails (which seems to always generate 
mails with an "xtext" ORCPT param).


Thanks!

[0] https://www.microfocus.com/products/groupwise/


On Fri, Oct 28, 2022 at 04:16:36PM +0200, Tim Kuijsten wrote:
I have refined and more thoroughly tested a previous patch that 
relaxes the ORCPT address check.


Over the years mail has been rejected by senders that use RCPT TO 
commands like:
RCPT TO: 
ORCPT=rfc822;groupwise-i...@example.com:0:0 or RCPT 
TO: ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE


In the above the domain part of the ORCPT address resolves to 
example.com:0:0 which is rejected by smtpd with the message:
smtpd[20797]: 1a3a396cd4c57d05 smtp failed-command command="RCPT 
TO: ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE" result="553 ORCPT address syntax error"


I've studied RFC 3461 section 4 and 4.2 but it's not entirely clear 
to me if the above ORCPT command is valid or not. The encoding 
adheres to the spec, which says it must be valid xtext.


With this patch smtpd accepts any ORCPT that is valid xtext as 
defined in the RFC (and logs on informational message when it 
consists of an invalid user or domain name).


Cheers,

Tim

---
usr.sbin/smtpd/smtp_session.c | 22 ++ 
usr.sbin/smtpd/smtpd.h|  1 +
usr.sbin/smtpd/util.c | 32  
3 files changed, 51 insertions(+), 4 deletions(-)


diff --git a/usr.sbin/smtpd/smtp_session.c 
b/usr.sbin/smtpd/smtp_session.c index 72e13e8fd8d..c0c29d4a695 
100644

--- a/usr.sbin/smtpd/smtp_session.c
+++ b/usr.sbin/smtpd/smtp_session.c
@@ -2415,6 +2415,7 @@ smtp_tx_create_message(struct smtp_tx *tx) 
static void

smtp_tx_rcpt_to(struct smtp_tx *tx, const char *line)
{
+   struct mailaddr orcptaddr;
  char *opt, *p;
  char *copy;
  char tmp[SMTP_LINE_MAX]; @@ -2469,10 +2470,23 @@ 
smtp_tx_rcpt_to(struct smtp_tx *tx, const char *line)

  if (strncasecmp(opt, "rfc822;", 7) == 0)
  opt += 7;

-   if (!text_to_mailaddr(>evp.dsn_orcpt, 
opt) || -
!valid_localpart(tx->evp.dsn_orcpt.user) || - 
(strlen(tx->evp.dsn_orcpt.domain) != 0 && - 
!valid_domainpart(tx->evp.dsn_orcpt.domain))) { +

if (!text_to_mailaddr(, opt)) {
+   smtp_reply(tx->session,
+   "553 ORCPT address syntax 
error"); +   return;

+   }
+
+   if (valid_localpart(orcptaddr.user) &&
+   (strlen(orcptaddr.domain) != 0 &&
+valid_domainpart(orcptaddr.domain))) { 
+   tx->evp.dsn_orcpt = orcptaddr;

+   } else if (valid_xtext(opt)) {
+   log_info("%016"PRIx64" smtp "
+   "uncommon ORCPT: \"%s\", 
u:\"%s\", d:\"%s\"",

+   tx->session->id,
+   opt, orcptaddr.user, 
orcptaddr.domain); +   tx->evp.dsn_orcpt 
= orcptaddr;

+   } else {
  smtp_reply(tx->session,
  "553 ORCPT address syntax error");
  return; diff --git 
a/usr.sbin/smtpd/smtpd.h b/usr.sbin/smtpd/smtpd.h

index 125a6a5dfbe..c59706885e2 100644
--- a/usr.sbin/smtpd/smtpd.h
+++ b/usr.sbin/smtpd/smtpd.h
@@ -1702,6 +1702,7 @@ int mailaddr_match(const struct mailaddr *, 
const struct mailaddr *);

int valid_localpart(const char *);
int valid_domainpart(const char *);
int valid_domainname(const char *);
+int valid_

Re: Connecting client at 587

[Tassilo Philipp]

I'm just guessing as you don't post your entire smtpd.conf:

auth needs a table (in your case "creds", but it misses the <>), which 
you have to define beforehand and point to your file via the "table" 
keyword.


At the end of smtpd.conf(5) you have examples.


On Sat, Oct 21, 2023 at 09:07:04PM +0200, Sagar Acharya wrote:

I am hosting with tls-require at port 587

listen on 0.0.0.0 tls-require port 587 pki pkname auth creds

In creds, I have for user foo

foo\t

But when I authenticate using client Monocles mail, I get invalid username or 
password, with STARTTLS.

Logs say

smtp authentication user=foo result=permfail
smtp failed-command command="AUTH PLAIN (...)" result="535 Authentication 
failed"

What is the issue?


Thanking you
Sagar Acharya
https://humaaraartha.in/selfdost/selfdost.html

Re: Example of smtpd-filters

[Tassilo Philipp]

You basically write a script that processes incoming lines on stdin, and 
writes back to stdout. The protocol is described in smtpd-filters(7).


This basic idea in lua would be something like:

for line in io.lines() do
if line == 'in' then
io.write('out')
end
end

Now handle the lines you want to handle, and respond to them how it's 
described in smtpd-filters(7).



On Fri, Oct 20, 2023 at 09:52:41PM +0200, Sagar Acharya wrote:

I'm simply unable to start.

I have to know where to get the mail from, format of text, which 
process to give the return to, in what format, i.e. are there any 
standard return values for accept or reject mail.

Thanking you
Sagar Acharya
https://humaaraartha.in/selfdost/selfdost.html



20 Oct 2023, 23:47 by tphil...@potion-studios.com:


Post your script that you got so far, explain where you get stuck/confused, and 
someone will probably be happy to help.


On Fri, Oct 20, 2023 at 02:05:13PM +0200, Sagar Acharya wrote:


Can you please help me with a sample script for accepting or rejecting mail in 
lua.

Say there is myfilter.lua within /etc/smtpd/

Can one please help with a sample whose pseudocode goes like

fetch(mail)if mailbody contains foo, reject,
else if mailbody contains bar, accept.

Such a sample script would help very much to support users write their own custom filters. Thanking you 
Sagar Acharya

https://humaaraartha.in/selfdost/selfdost.html

Re: Example of smtpd-filters

[Tassilo Philipp]

Post your script that you got so far, explain where you get 
stuck/confused, and someone will probably be happy to help.



On Fri, Oct 20, 2023 at 02:05:13PM +0200, Sagar Acharya wrote:

Can you please help me with a sample script for accepting or rejecting mail in 
lua.

Say there is myfilter.lua within /etc/smtpd/

Can one please help with a sample whose pseudocode goes like

fetch(mail)if mailbody contains foo, reject,
else if mailbody contains bar, accept.

Such a sample script would help very much to support users write their own custom filters. 
Thanking you

Sagar Acharya
https://humaaraartha.in/selfdost/selfdost.html

Re: Setting personal mailserver

[Tassilo Philipp]

Thanks for the link. They don't require it though, according to the doc, 
they don't even enforce it for gmail when using it with own domains.


It's certainly a good practice, though, that's true.



On Sat, Sep 09, 2023 at 01:54:48PM +0900, Pontus Stenetorp wrote:

On Sat 09 Sep 2023, Stuart Longland wrote:

On 9/9/23 01:28, Tassilo Philipp wrote:
[...] I didn't bother with DKIM until Google started mandating 
it for example [...[


Hm... do you have a reference for that? I don't have that 
experience with gmail servers. Also I don't find info about that 
being mandatory, online.


https://support.google.com/a/answer/174124?hl=en#hcfe-content

Sadly, I don't have any log messages to show, because I last had the 
problem in May 2021, and my log retention does not go back that far.


At least from my experience and from reading Google's documentation, Google 
does not *require* both DKIM and SPF, but has since late 2022 or early 2023 
started to randomly reject e-mails that has *neither*:

550-5.7.26 This mail is unauthenticated, which poses a security risk to 
the sender and Gmail users, and has been blocked. The sender must authenticate 
with at least one of SPF or DKIM. For this message, DKIM checks did not pass 
and SPF check for [example.com] did not pass with ip: [127.0.0.1]. The sender 
should visit https://support.google.com/mail/answer/81126#authentication for 
instructions on setting up authentication.

I doubt that DKIM ever hurts though if you have it set up.

Re: Setting personal mailserver

[Tassilo Philipp]

[...] I didn't bother with DKIM until Google started mandating it for 
example [...[


Hm... do you have a reference for that? I don't have that experience 
with gmail servers. Also I don't find info about that being mandatory, 
online.



On Fri, Sep 08, 2023 at 08:24:38AM +1000, Stuart Longland wrote:

On 7/9/23 20:44, Sagar Acharya wrote:

Let the mail providers have their setups. Is it possible to have a 
configuration where I have 2 servers, example.com example2.com where I can send 
and receive emails on ports say, 777 on plaintext, starttls optional and port 
778 with smtps?

Give me a configuration for such a thing.

humaaraartha.in.       TXT        "v=spf1 ipv4:{myipv4address} -all" 
humaaraartha.in.   TXT    "resports:777,778" 
humaaraartha.in. humaaraartha.in.       MX          10 humaaraartha.in. 
humaaraartha.in.       A              {myipv4address} 
That is all you have, nothing more for both servers. Can you help me send and recieve mails on ports 777,778 with just above DNS and smtpd? I can add SRV records for detection of ports 777, 778 if you want.


Okay, not quite sure what the "resports" TXT record is achieving (a 
quick search on the topic didn't reveal any documentation on how it 
was supposed to work or correct syntax).  I won't labour the point 
about outgoing port 25 traffic since others have covered this already.


You can of course use different ports between servers on an 
agreed-upon manner.  e.g. say we have a server, bnemx.vk4msl.com, 
running OpenSMTPD:



vk4msl-bne# cat /etc/mail/smtpd.conf
#   $OpenBSD: smtpd.conf,v 1.14 2019/11/26 20:14:38 gilles Exp $

# This is the smtpd server system-wide configuration file.
# See smtpd.conf(5) for more information.

#table aliases file:/etc/mail/aliases
table virtualdomains file:/etc/mail/virtualdomains
table virtualusers file:/etc/mail/virtualusers

pki bnemx cert "/etc/ssl/bnemx.vk4msl.com.fullchain.pem"
pki bnemx key "/etc/ssl/private/bnemx.vk4msl.com.key"
pki bnemx dhe auto

listen on socket
listen on all tls pki bnemx 

… etc, I won't post the full config.

Those `listen` lines are the key, from smtpd.conf manpage:

listen on interface [family] [options]
Listen on the interface for incoming connections, using the same
syntax as ifconfig(8).  The interface parameter may also be an
interface group, an IP address, or a domain name.  Listening can
optionally be restricted to a specific address family, which can
be either inet4 or inet6.


In amongst the options:

port [port]
Listen on the given port instead of the default port 25.


So if I chose to, I could add:

listen on all port 777

and then re-start smtpd, I'd now be listening on port 777.

You could then tell your SMTP server to send to port 777 when sending 
to my domain.


But doing so would be useless:
- no one else would bother using port 777/tcp: they would most likely 
use port 25
- you wouldn't be able to send to any other server, unless they too, 
chose to use port 777/tcp.


If you have a good proposal for how such alternative ports could be 
advertised (maybe via DNS TXT record), perhaps you could propose that 
as a Request For Comment to the Internet Engineering Task Force… and 
maybe if enough people thought it was a good idea, it would be adopted 
with its own official RFC number (like RFC-821, later replaced by 
RFC-2821 and RFC-5321).


That though, won't mean instant ability to pick your own port number. 
The "alternate port number" feature would then need to be added to the 
various SMTP servers out there.  Then sysadmins would need to install 
that version.


This may take years, or even never happen in some cases.  (Qmail is 
still IPv4-only because the author believes IPv6 is unnecessary.)


Regardless of what you think of spam or how to fight it, the truth is 
the small fish don't make the rules in this game.  You and I are small 
fish.  I've been mucking around with mail servers pretty much this 
whole century so far.


I started with trialling something over dial-up (ever seen a 56kbps 
modem screaming under the strain of an outbound mail queue stuffed 
with spam?  I have!)… moved to using Sendmail on an old Slackware 
server hosted on ADSL with 2GB SCSI disks and a self-signed HTTPS 
certificate for webmail in 2001.  Been running my own server ever 
since.


It's not impossible to do it yourself, and dealing with spam is a 
constant cat-and-mouse game.  Things have become more complex out of 
necessity (I didn't bother with DKIM until Google started mandating it 
for example), but even then, not overly difficult.


The minimum standard however has changed over the years as 
requirements changed.  That includes:


- outbound SMTP unblocked -- pretty much since forever since that's 
how TCP/IP works
- static IPv4 -- dynamic IPv4 has not been possible since ~2004 or so 
- SPF DNS records -- since ~2010 or so

- DKIM signing and DMARC policies -- 

Re: Setting personal mailserver

[Tassilo Philipp]

Give me a configuration for such a thing. 


I think several people mentioned by now reading the doc and getting 
familiar with email. This is not trying to be mean, I think you really 
would set it up faster by learning and using it, then also understanding 
it b/c you have to maintain it, than asking over and over for some 
config lines on here.



On Thu, Sep 07, 2023 at 12:44:07PM +0200, Sagar Acharya wrote:

I get you, I get you.

Let the mail providers have their setups. Is it possible to have a 
configuration where I have 2 servers, example.com example2.com where I can send 
and receive emails on ports say, 777 on plaintext, starttls optional and port 
778 with smtps?

Give me a configuration for such a thing. 

humaaraartha.in.       TXT        "v=spf1 ipv4:{myipv4address} -all" 
humaaraartha.in.   TXT    "resports:777,778"humaaraartha.in. humaaraartha.in.       MX          10 humaaraartha.in.  
humaaraartha.in.       A              {myipv4address} 
That is all you have, nothing more for both servers. Can you help me send and recieve mails on ports 777,778 with just above DNS and smtpd? I can add SRV records for detection of ports 777, 778 if you want. 
Thanking you

Sagar Acharya
https://humaaraartha.in



7 Sept 2023, 15:33 by gil...@poolp.org:


September 7, 2023 11:44 AM, "Sagar Acharya"  wrote:

In today's times of mature NLP, you will not be able to differentiate human mail from bot mail or 
spam. Only in person verification is trustworthy.
No. Are you saying that only people who control the network should send mails? Well DNS exactly is 
for that. If you find I send spams, you can easily easily block mails from my domain 
humaaraartha.in but it is not wise nor ethical to by default not allow people to mail.


That issue lies because hardware is not mapped to people. There is no technological solution for 
trust hopping between machines. ssh should be discouraged and each machine, denoted by single IP 
address should be mapped to a human. So humaaraartha.in is run by Sagar Acharya.


My configuration of whitelisting does exactly that. In today's world where each grain can 
potentially have an IPv6, I accept requests only from whitelist or at the very least accept from 
everyone and prioritize the whitelist.


Well, what action should be implemented for sending emails. I don't get a sending action. I have 
changed conf to


action "send" relay helo humaaraartha.inmatch from any for any action "send" 
Thanking you

Sagar Acharya
https://humaaraartha.in



As many people told you, domestic connections are no longer suitable for sending mail, wether you 
like it or not this is the actual state of the SMTP network and will remain like this because the 
big mailer corps control most of the e-mail address space and have decided so. If you ignore this 
then you'll be blocked from most recipients, you decide if it's acceptable for you.



Then, if you're domestic connection has outgoing port 25 filtered, you can't work around this and 
need a relay host somewhere else that can accept mail on a different port with unfiltered port 25 
for outgoing trafic. You can't just switch to a different port and expect it to work this shows a 
misunderstanding of how networking, internet and SMTP works.


There's nothing that can be changed in your config that will fix this because the problem isn't a 
configuration issue but an issue with understanding both what you're allowed and trying to do.

Re: Setting personal mailserver

[Tassilo Philipp]

I tested all of the IPs from your output, and all of them listen on port 
25 and a smtp server is answering. So if you are relaying to those via 
port 25, and you get a network error (I guess a timeout), then I guess 
your outgoing port 25 is blocked. This is relatively common with 
residential uplinks, ask your ISP to open port 25 for you.


That said... I'm only guessing here.


On Sat, Sep 02, 2023 at 03:52:37PM +0200, Sagar Acharya wrote:

I made some progress. I am able to receive mails now but when I send mail from 
u...@mydomain.com to sagaracha...@tutanota.com using mutt , I get,

result="TempFail" stat="Network error on destination MXs"
smtp-out: Enabling route [] <-> 81.3.6.162 (w1.tutanota.de) 
smtp-out: Enabling route [] <-> 185.205.69.211 (185.205.69.211) 
smtp-out: Enabling route [] <-> 81.3.6.165 (w4.tutanota.de)

mta error reason=Connection timeout


DNS

mydomain.com.    86400      IN        MX        10 mail.mydomain.com.

Thanking you
Sagar Acharya
https://humaaraartha.in



2 Sept 2023, 05:45 by bub...@live.de:


Hello, pls show your config file.

Mit freundlichen Grüßen, V.Bubnov


01.09.2023, в 21:43, Sagar Acharya  написал(а):

To enable being able to send mails from my server, I added tls certs.

Now when I send from this email id to u...@mydomain.com , I get the error below.

530
5.5.1 Invalid command: Must issue an AUTH command first (in reply to MAIL 
FROM command)


Since STARTTLS is working on 25, I think things should go smoothly but it isn't so. Please help. 
Thanking you

Sagar Acharya
https://humaaraartha.in



1 Sept 2023, 20:52 by sagaracha...@tutanota.com:


I used mutt for accessing mail. I still am unable to send mail using my server. 
I can receive mails.

I also completed the whitelist. How can I do this?

I want to allow access only upto 25MB attachments from whitelisted emails and 
allow only 1email (only text based) per day from non-whitelisted emails. How do 
I do that?

How do I limit overall size of mailbox and auto-delete old mails?

Thanking you
Sagar Acharya
https://humaaraartha.in



1 Sept 2023, 14:04 by tphil...@potion-studios.com:


From the doc (smtpd.conf(5)):

maildir [pathname [junk]]
Deliver the message to the maildir in pathname if
specified, or by default to ~/Maildir.

So given your config, you seem to get exactly what you configured.

For your "whitelist", create the match rules for your domains, and for 
everything else use a reject rule at the end.





On Fri, Sep 01, 2023 at 09:59:31AM +0200, Sagar Acharya wrote:

I got a mail, which lies in Maildir, however no mailbox is configured. Is there 
a default mailbox in alpine and how do I access the mail contents in

~/Maildir

My mails are under

~/Maildir/new/

Also, how do I whitelist email ids, say, I want mails only from

f...@bar.com
f...@bar2.com
f...@bar3.com

That's it, no other mails.
Thanking you
Sagar Acharya
https://humaaraartha.in



1 Sept 2023, 12:42 by sagaracha...@tutanota.com:


How do I do that? What CLI tool do I use?

While starting the daemon, the configuration is OK as given in prompt.

With the DNS configuration I have, where can I send a mail, at 
u...@mydomain.com or at u...@mail.mydomain.com ?

Thanking you
Sagar Acharya
https://humaaraartha.in



31 Aug 2023, 01:06 by stu...@gathman.org:




On Wed, 30 Aug 2023, Sagar Acharya wrote:


I'm facing an issue similar to a person a while ago available on archive. I use 
alpine, and the conf is as below

There is nothing in the mailbox.



Are you looking with alpine, or with CLI tools like ls?  Use CLI tools to check 
that you've configured smtpd to store incoming mail where you think you have.

I go so far as to use raw IPv6 for personal mailbox on various overlay mesh 
vpns like Cjdns and Yggdrasil (giving you personal authenticated IPs 
independent of any ISP).  I just caught up with an online friend that moved 
from Hawaii to New York.  Still works despite changes in ISP and ICANN domains.

Re: Setting personal mailserver

[Tassilo Philipp]

If you want to apply the logic you are asking for to emails that are 
transmitted, I think you need to work with custom filters for specific 
logic (e.g. looking only at attachment size, per day limits, etc.).


There is no "here you go" answer to that, or simple smtpd.conf 
statements that would do what you want.  For overall message size you 
can use max-message-size (see smtpd.conf(5)), but this doesn't sound 
like it fits your requirement.


That said: I also think you need to dig deeper into how email works, 
in general, and read the docs, thoroughly. Your questions here are 
either very broad, or they simply don't apply to this mailing list.


For example you are asking here how to limit the overall size of the 
mailbox and auto-delete old mails: the answer to that is, that it's 
simply not the SMTP server's job to do any of that, as SMTP is about 
email transmission, not about mailboxes.


In your case (from your original config example), your mailbox is a 
maildir on disk, which you could limit with file system tools, but you 
could use other message delivery agents (MDA) to have finer control of 
what ends up where. Read up on them. Either way, this is the wrong 
mailing list for that.




On Fri, Sep 01, 2023 at 05:22:20PM +0200, Sagar Acharya wrote:
I used mutt for accessing mail. I still am unable to send mail using 
my server. I can receive mails.


I also completed the whitelist. How can I do this?

I want to allow access only upto 25MB attachments from whitelisted 
emails and allow only 1email (only text based) per day from 
non-whitelisted emails. How do I do that?


How do I limit overall size of mailbox and auto-delete old mails? 
Thanking you

Sagar Acharya
https://humaaraartha.in



1 Sept 2023, 14:04 by tphil...@potion-studios.com:


From the doc (smtpd.conf(5)):

 maildir [pathname [junk]]
 Deliver the message to the maildir in pathname if
 specified, or by default to ~/Maildir.

So given your config, you seem to get exactly what you configured.

For your "whitelist", create the match rules for your domains, and for 
everything else use a reject rule at the end.




On Fri, Sep 01, 2023 at 09:59:31AM +0200, Sagar Acharya wrote:


I got a mail, which lies in Maildir, however no mailbox is configured. Is there 
a default mailbox in alpine and how do I access the mail contents in

~/Maildir

My mails are under

~/Maildir/new/

Also, how do I whitelist email ids, say, I want mails only from

f...@bar.com
f...@bar2.com
f...@bar3.com

That's it, no other mails.
Thanking you
Sagar Acharya
https://humaaraartha.in



1 Sept 2023, 12:42 by sagaracha...@tutanota.com:


How do I do that? What CLI tool do I use?

While starting the daemon, the configuration is OK as given in prompt.

With the DNS configuration I have, where can I send a mail, at 
u...@mydomain.com or at u...@mail.mydomain.com ?

Thanking you
Sagar Acharya
https://humaaraartha.in



31 Aug 2023, 01:06 by stu...@gathman.org:




On Wed, 30 Aug 2023, Sagar Acharya wrote:


I'm facing an issue similar to a person a while ago available on archive. I use 
alpine, and the conf is as below

There is nothing in the mailbox.



Are you looking with alpine, or with CLI tools like ls?  Use CLI tools to check 
that you've configured smtpd to store incoming mail where you think you have.

I go so far as to use raw IPv6 for personal mailbox on various overlay mesh 
vpns like Cjdns and Yggdrasil (giving you personal authenticated IPs 
independent of any ISP).  I just caught up with an online friend that moved 
from Hawaii to New York.  Still works despite changes in ISP and ICANN domains.

Re: Setting personal mailserver

[Tassilo Philipp]

The error is clear: whatever you connect to requires authentification, 
before submitting the mail. So, provide that info.


Authentication is not TLS encryption, so no idea what you mean by you 
added TLS certs and that STARTTLS should make it work: it's simply not 
authentication.



On Fri, Sep 01, 2023 at 09:42:17PM +0200, Sagar Acharya wrote:

To enable being able to send mails from my server, I added tls certs.

Now when I send from this email id to u...@mydomain.com , I get the error below.

530
5.5.1 Invalid command: Must issue an AUTH command first (in reply to MAIL 
FROM command)


Since STARTTLS is working on 25, I think things should go smoothly but it isn't so. Please help. 
Thanking you

Sagar Acharya
https://humaaraartha.in



1 Sept 2023, 20:52 by sagaracha...@tutanota.com:


I used mutt for accessing mail. I still am unable to send mail using my server. 
I can receive mails.

I also completed the whitelist. How can I do this?

I want to allow access only upto 25MB attachments from whitelisted emails and 
allow only 1email (only text based) per day from non-whitelisted emails. How do 
I do that?

How do I limit overall size of mailbox and auto-delete old mails?

Thanking you
Sagar Acharya
https://humaaraartha.in



1 Sept 2023, 14:04 by tphil...@potion-studios.com:


From the doc (smtpd.conf(5)):

maildir [pathname [junk]]
Deliver the message to the maildir in pathname if
specified, or by default to ~/Maildir.

So given your config, you seem to get exactly what you configured.

For your "whitelist", create the match rules for your domains, and for 
everything else use a reject rule at the end.




On Fri, Sep 01, 2023 at 09:59:31AM +0200, Sagar Acharya wrote:


I got a mail, which lies in Maildir, however no mailbox is configured. Is there 
a default mailbox in alpine and how do I access the mail contents in

~/Maildir

My mails are under

~/Maildir/new/

Also, how do I whitelist email ids, say, I want mails only from

f...@bar.com
f...@bar2.com
f...@bar3.com

That's it, no other mails.
Thanking you
Sagar Acharya
https://humaaraartha.in



1 Sept 2023, 12:42 by sagaracha...@tutanota.com:


How do I do that? What CLI tool do I use?

While starting the daemon, the configuration is OK as given in prompt.

With the DNS configuration I have, where can I send a mail, at 
u...@mydomain.com or at u...@mail.mydomain.com ?

Thanking you
Sagar Acharya
https://humaaraartha.in



31 Aug 2023, 01:06 by stu...@gathman.org:




On Wed, 30 Aug 2023, Sagar Acharya wrote:


I'm facing an issue similar to a person a while ago available on archive. I use 
alpine, and the conf is as below

There is nothing in the mailbox.



Are you looking with alpine, or with CLI tools like ls?  Use CLI tools to check 
that you've configured smtpd to store incoming mail where you think you have.

I go so far as to use raw IPv6 for personal mailbox on various overlay mesh 
vpns like Cjdns and Yggdrasil (giving you personal authenticated IPs 
independent of any ISP).  I just caught up with an online friend that moved 
from Hawaii to New York.  Still works despite changes in ISP and ICANN domains.

Re: Setting personal mailserver

[Tassilo Philipp]

From the doc (smtpd.conf(5)):

   maildir [pathname [junk]]
 Deliver the message to the maildir in pathname if
 specified, or by default to ~/Maildir.

So given your config, you seem to get exactly what you configured.

For your "whitelist", create the match rules for your domains, and for 
everything else use a reject rule at the end.





On Fri, Sep 01, 2023 at 09:59:31AM +0200, Sagar Acharya wrote:

I got a mail, which lies in Maildir, however no mailbox is configured. Is there 
a default mailbox in alpine and how do I access the mail contents in

~/Maildir

My mails are under

~/Maildir/new/

Also, how do I whitelist email ids, say, I want mails only from

f...@bar.com
f...@bar2.com
f...@bar3.com

That's it, no other mails.
Thanking you
Sagar Acharya
https://humaaraartha.in



1 Sept 2023, 12:42 by sagaracha...@tutanota.com:


How do I do that? What CLI tool do I use?

While starting the daemon, the configuration is OK as given in prompt.

With the DNS configuration I have, where can I send a mail, at 
u...@mydomain.com or at u...@mail.mydomain.com ?

Thanking you
Sagar Acharya
https://humaaraartha.in



31 Aug 2023, 01:06 by stu...@gathman.org:




On Wed, 30 Aug 2023, Sagar Acharya wrote:

I'm facing an issue similar to a person a while ago available on 
archive. I use alpine, and the conf is as below


There is nothing in the mailbox.



Are you looking with alpine, or with CLI tools like ls?  Use CLI tools 
to check that you've configured smtpd to store incoming mail where you 
think you have.


I go so far as to use raw IPv6 for personal mailbox on various overlay 
mesh vpns like Cjdns and Yggdrasil (giving you personal authenticated 
IPs independent of any ISP).  I just caught up with an online 
friend that moved from Hawaii to New York.  Still works despite changes 
in ISP and ICANN domains.

Re: [PATCH] relax 553 ORCPT address syntax error (was Re: EMails to "ORCPT=rfc822;u...@example.com" are rejected)

[Tassilo Philipp]

Sorry to shamelessly "bump" this, but any way to get this integrated 
into upstream, eventually?


We used the original patch from Frank Scholl and then this improved one 
in production now for like a year, now, and didn't experience issues. In 
our case it is specifically needed for a client that uses GroupWise[0] 
internally to send mails (which seems to always generate mails with an 
"xtext" ORCPT param).


Thanks!

[0] https://www.microfocus.com/products/groupwise/


On Fri, Oct 28, 2022 at 04:16:36PM +0200, Tim Kuijsten wrote:
I have refined and more thoroughly tested a previous patch that 
relaxes the ORCPT address check.


Over the years mail has been rejected by senders that use RCPT TO 
commands like:
RCPT TO: ORCPT=rfc822;groupwise-i...@example.com:0:0 or 
RCPT TO: ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE


In the above the domain part of the ORCPT address resolves to 
example.com:0:0 which is rejected by smtpd with the message:
smtpd[20797]: 1a3a396cd4c57d05 smtp failed-command command="RCPT 
TO: ORCPT=rfc822;groupwise-i...@example.com:0:0 
NOTIFY=SUCCESS,FAILURE" result="553 ORCPT address syntax error"


I've studied RFC 3461 section 4 and 4.2 but it's not entirely clear to 
me if the above ORCPT command is valid or not. The encoding adheres to 
the spec, which says it must be valid xtext.


With this patch smtpd accepts any ORCPT that is valid xtext as defined 
in the RFC (and logs on informational message when it consists of an 
invalid user or domain name).


Cheers,

Tim

---
usr.sbin/smtpd/smtp_session.c | 22 ++ 
usr.sbin/smtpd/smtpd.h|  1 +
usr.sbin/smtpd/util.c | 32  
3 files changed, 51 insertions(+), 4 deletions(-)


diff --git a/usr.sbin/smtpd/smtp_session.c b/usr.sbin/smtpd/smtp_session.c 
index 72e13e8fd8d..c0c29d4a695 100644

--- a/usr.sbin/smtpd/smtp_session.c
+++ b/usr.sbin/smtpd/smtp_session.c
@@ -2415,6 +2415,7 @@ smtp_tx_create_message(struct smtp_tx *tx) 
static void

smtp_tx_rcpt_to(struct smtp_tx *tx, const char *line)
{
+   struct mailaddr orcptaddr;
   char *opt, *p;
   char *copy;
   char tmp[SMTP_LINE_MAX]; 
@@ -2469,10 +2470,23 @@ smtp_tx_rcpt_to(struct smtp_tx *tx, const char 
*line)

   if (strncasecmp(opt, "rfc822;", 7) == 0)
   opt += 7;

-   if (!text_to_mailaddr(>evp.dsn_orcpt, opt) || 
-   !valid_localpart(tx->evp.dsn_orcpt.user) || 
-   (strlen(tx->evp.dsn_orcpt.domain) != 0 && 
-!valid_domainpart(tx->evp.dsn_orcpt.domain))) { 
+   if (!text_to_mailaddr(, opt)) {

+   smtp_reply(tx->session,
+   "553 ORCPT address syntax error"); 
+   return;

+   }
+
+   if (valid_localpart(orcptaddr.user) &&
+   (strlen(orcptaddr.domain) != 0 &&
+valid_domainpart(orcptaddr.domain))) { 
+   tx->evp.dsn_orcpt = orcptaddr;

+   } else if (valid_xtext(opt)) {
+   log_info("%016"PRIx64" smtp "
+   "uncommon ORCPT: \"%s\", u:\"%s\", 
d:\"%s\"",

+   tx->session->id,
+   opt, orcptaddr.user, orcptaddr.domain); 
+   tx->evp.dsn_orcpt = orcptaddr;

+   } else {
   smtp_reply(tx->session,
   "553 ORCPT address syntax error");
   return; 
diff --git a/usr.sbin/smtpd/smtpd.h b/usr.sbin/smtpd/smtpd.h

index 125a6a5dfbe..c59706885e2 100644
--- a/usr.sbin/smtpd/smtpd.h
+++ b/usr.sbin/smtpd/smtpd.h
@@ -1702,6 +1702,7 @@ int mailaddr_match(const struct mailaddr *, 
const struct mailaddr *);

int valid_localpart(const char *);
int valid_domainpart(const char *);
int valid_domainname(const char *);
+int valid_xtext(const char *s);
int valid_smtp_response(const char *);
int secure_file(int, char *, char *, uid_t, int);
int  lowercase(char *, const char *, size_t);
diff --git a/usr.sbin/smtpd/util.c b/usr.sbin/smtpd/util.c
index feb663cc61e..0c4d0015fa4 100644
--- a/usr.sbin/smtpd/util.c
+++ b/usr.sbin/smtpd/util.c
@@ -515,6 +515,38 @@ valid_domainname(const char *str)
   return 1; 
}


+int
+valid_xtext(const char *s)
+{
+   while (*s != '\0') {
+   if (*s == '=')
+   return 0;
+
+   if (*s < '\x21' || *s > '\x7e')
+   return 0;
+
+   if (*s == '+') {
+   /* expect hexchar "+XX" RFC 3461 4. */
+   if (strnlen(s, 3) != 3)
+   return 0;
+
+   s++;
+
+   if 

Re: smtpctl: lookup_record: %{i}._spf.mta.salesforce.com contains macros and can't be resolved

[Tassilo Philipp]

If I understand correctly, this is part of the SPF spec. Those things 
are defined hard, e.g. %{i} is the source IP of the message, %{ir} is 
the same reversed, etc.. There are more, but it's a short list.


So, I guess this is simply not implemented in smtpctl spfwalk.

Btw... this is the first time I ever see this used in the wild, thanks 
for the pointer.


On Thu, Jan 26, 2023 at 01:58:45PM +0100, Harald Dunkel wrote:

Hi folks,

smtpctl spfwalk returns messages like

smtpctl: lookup_record: %{i}._spf.mta.salesforce.com contains macros and can't be resolved 
smtpctl: lookup_record: %{ir}.%{v}.%{d}.spf.has.pphosted.com contains macros and can't be resolved 
smtpctl: lookup_record: %{i}._spf.mta.salesforce.com contains macros and can't be resolved 
smtpctl: lookup_record: %{ir}.%{v}.%{d}.spf.has.pphosted.com contains macros and can't be resolved 
smtpctl: lookup_record: %{i}._spf.mta.salesforce.com contains macros and can't be resolved 
smtpctl: lookup_record: %{i}._spf.mta.salesforce.com contains macros and can't be resolved 
smtpctl: lookup_record: %{i}._spf.mta.salesforce.com contains macros and can't be resolved 
smtpctl: lookup_record: %{i}._spf.corp.salesforce.com contains macros and can't be resolved 
smtpctl: lookup_record: %{i}._spf.mta.salesforce.com contains macros and can't be resolved


is this something the owner of the SPF record in DNS has to fix, is 
this not implemented yet, or what is the story here?



Every insightful comment is highly appreciated

Harri

Re: sysupdate and space check

[Tassilo Philipp]

I think you wrote to the wrong mailing list, this is OpenSMTPd. Maybe 
you wanted to write to the OpenBSD ML?.



On Mon, Oct 24, 2022 at 04:59:23PM +, Peter Fraser wrote:
I make a stupid mistake; I didn't check partition sizes before doing a sysupgrade. 
sysupgrade ran out of space or /usr in the middle of the upgrade.
I know I should have checked first but it would be nice if sysupgrade did warn me. 
The site was a 20-minute drive away, and their down time was a lot longer then I expected.

Re: Failure to check FCrDNS with long DNS replies?

[Tassilo Philipp]

On 21. Nov 2020, at 10:44, Tassilo Philipp  wrote:


FYI, I run into the same issue with a different provider:
relay.yourmailgateway.de which also has a large number of A records.

Trying to reproduce and digging deeper now, by adding debug logs etc.


Interesting... thanks for checking and having thought of my report. I 
for myself didn't have any issues anymore, however, I barely ever 
receive any mail from sfr. Also, given the random order of IPs in the 
DNS reply, I simply might have had luck if it's in any case related 
to the IP order. I have no evidence for, but when I was having 
problems, the IP in question was among the last ones in the reply.


I'm curious what you'll find…


FYI, after digging deeper into this, I figured out that this was an issue 
with the DNS forwarders/resolver I was using (unfortunately not under 
my control) on this particular mail server: The forwarder is not able 
to resolve relay.yourmailgateway.de <http://relay.yourmailgateway.de/>

at all, likely due to the large number of A records 52 A + 13  records.

I believe there is a limit in BIND suite (32) and OpenBSD libc (35) 
and others, which restricts older gethostbyname() calls with struct 
hostent results down to that 30-something number. Likely the used 
resolver was using these old/obsolete libc functions…


But OpenSMTPD and filter FCrDNS and OpenBSD ASR all doing fine here, 
because using getaddrinfo() alike under the hood with dynamic struct 
addrinfo result allocation, which does not expose any such limits and 
resolves all 65 A and  records just fine.


Thanks for the feedback, that sounds like a fitting analysis. So if I 
follow your thought, the resolver basically truncates the list and what 
opensmtpd gets to see at the hand sometimes misses the entry it tries to 
verify? Sounds like the culprit indeed.


I personally did not observe this issue anymore, unsure why, some update 
might have fixed it on some upstream resolver or dunno...
How are you dealing with this, given you don't control the resolver? I 
guess you just switched it?


Thanks again for digging into this more

Re: Capturing the log output of opensmtpd

[Tassilo Philipp]

Yes, a bounce message is an email, so it won't be on stdout or stderr. 
Either way, Martijn's answer probably points out a better approach for 
your case, anyways.



On Sat, Oct 15, 2022 at 02:34:52PM +0100, Simon Harrison wrote:

On Sat, 15 Oct 2022 14:44:01 +0200
Tassilo Philipp  wrote:

I guess your python subprocess is the submission agent? In that case 
you get the error in theory automatically, but it would be in form of 
a bounce message. If your submission agent cannot receive bounces 
they will be lost (probably hanging in the next MTA's queue for a 
while until they hit a timeout).


On Sat, Oct 15, 2022 at 01:21:15PM +0100, Simon Harrison wrote:
Hello. I'm using Python subprocess to send mails on a linux server. 
For outlook.com addresses I get the following error in 
/var/log/mail.log:


Oct 14 11:41:22 myhost smtpd[1846073]: f01b467faa967988 mta 
delivery evpid=d9b3ae9518ff979a from=
to= rcpt=<-> source="server.ip.address" 
relay="104.47.56.161 (104.47.56.161)" delay=2s result="PermFail" 
stat="550 5.7.1 Unfortunately, messages from [server.ip.address] 
weren't sent. Please contact your Internet service provider since 
part of their network is on our block list (S3140). You can also 
refer your provider to 
http://mail.live.com/mail/troubleshooting.aspx#errors. 
[CO1NAM11FT072.eop-nam11.prod.protection.outlook.com]"


So my question is, is there any way to get that output rather than 
ssh-ing into the server and checking the log? Can opensmtpd return 
that output as well as logging it? Python subprocess.stdout 
suggests that opensmtpd does not return anything over than 0 or 1.


Cheers,

Simon





stdout and stderr are both empty strings unfortunately.

Thanks,

Simon

Re: Capturing the log output of opensmtpd

[Tassilo Philipp]

I guess your python subprocess is the submission agent? In that case you 
get the error in theory automatically, but it would be in form of a 
bounce message. If your submission agent cannot receive bounces they 
will be lost (probably hanging in the next MTA's queue for a while until 
they hit a timeout).


On Sat, Oct 15, 2022 at 01:21:15PM +0100, Simon Harrison wrote:
Hello. I'm using Python subprocess to send mails on a linux server. For 
outlook.com addresses I get the following error in /var/log/mail.log:


Oct 14 11:41:22 myhost smtpd[1846073]: f01b467faa967988 mta delivery 
evpid=d9b3ae9518ff979a from= to=
rcpt=<-> source="server.ip.address" relay="104.47.56.161 
(104.47.56.161)" delay=2s result="PermFail" stat="550 5.7.1 
Unfortunately, messages from [server.ip.address] weren't sent. Please 
contact your Internet service provider since part of their network is 
on our block list (S3140). You can also refer your provider to 
http://mail.live.com/mail/troubleshooting.aspx#errors. 
[CO1NAM11FT072.eop-nam11.prod.protection.outlook.com]"


So my question is, is there any way to get that output rather than 
ssh-ing into the server and checking the log? Can opensmtpd return that 
output as well as logging it? Python subprocess.stdout suggests that 
opensmtpd does not return anything over than 0 or 1.


Cheers,

Simon

Re: Does OpenBSD support Receive Side Scaling (also called: multi-queue receiving)

[Tassilo Philipp]

I think you wrote to the wrong mailing list, this is OpenSMTPd, not 
OpenBSD.



On Fri, Oct 14, 2022 at 09:49:05PM +0900, Gabor LENCSE wrote:

Dear All,

I am a researcher and I would like to benchmark the stateful NAT64 
performance of OpenBSD PF.


I use a 32-core server as DUT (Device Under Test). When I use Linux 
for benchmarking other stateful NAT64 implementations, I use the 
"ethtool -N enp5s0f1 rx-flow-hash udp4 sdfn" command to include also 
the source and destination port numbers (not only the source and 
destination IP addresses) into the hash function to distribute the 
interrupts caused by packet arrivals evenly among all the CPU cores.


I tried to find a similar solution under OpenBSD, but I could not. (I 
used search expressions like: OpenBSD RSS receive side scaling multi 
queue receiving) Perhaps it is called differently under OpenBSD, or 
maybe there is no such solution at all?


Could you advise me please?

Thank you very much for your help in advance!

Best regards,

Gabor Lencse

Re: issue with lower case utf-8 when in quotations; RFC 2047?

[Tassilo Philipp]

3): or it's the side that composes the mail that writes bad mail headers


On Tue, Sep 13, 2022 at 07:58:48PM +0200, Tassilo Philipp wrote:

Well:

1) if it's really 'Content-Type: Type: text/plain; charset=“utf-8”', 
it's also wrong, the other "Type:" shouldn't be there, and the quotes 
(") aren't ascii 0x22h quotes, but unicode ones (the quote thing is 
probably the result of your client fancying things up pointlessly or 
copy and paste or dunno...)


2) none of this should have to do anything with OpenSMTPd, it's not 
parsing your mails and trying to make sense of UTF-8 or so; this means 
if it's not readable it's most likely your client not getting it right


hth

On Tue, Sep 13, 2022 at 10:08:18AM -0700, Ethan Ongstad wrote:
Nice catch. Actually that’s how it was in the message ( 
Content-Type: Type: text/plain; charset=“utf-8”  ). After looking at 
it again, the unreadable messages do have a base64 encoding, should 
this be an issue?



On Sep 13, 2022, at 08:58, Tassilo Philipp  wrote:

not sure if related, but I noticed your line says "text:plain" instead of 
"text/plain" (which should be used according to rfc2045)


On Mon, Sep 12, 2022 at 10:20:38PM -0700, Ethan Ongstad wrote:

The messages I receive that have the line ‘ Content-Type: text:plain; charset="utf-8” ‘ 
are not readable, but it is human readable if the "utf-8" bit is capitalized or if 
there are no quotation marks. Is this a opensmtpd bug?

section 2 of RFC 2047 makes it sound like capitalization should not be relevant. 
https://www.rfc-editor.org/rfc/rfc2047#section-2 
<https://www.rfc-editor.org/rfc/rfc2047#section-2>

Re: issue with lower case utf-8 when in quotations; RFC 2047?

[Tassilo Philipp]

Well:

1) if it's really 'Content-Type: Type: text/plain; charset=“utf-8”', 
it's also wrong, the other "Type:" shouldn't be there, and the quotes 
(") aren't ascii 0x22h quotes, but unicode ones (the quote thing is 
probably the result of your client fancying things up pointlessly or 
copy and paste or dunno...)


2) none of this should have to do anything with OpenSMTPd, it's not 
parsing your mails and trying to make sense of UTF-8 or so; this means 
if it's not readable it's most likely your client not getting it right


hth

On Tue, Sep 13, 2022 at 10:08:18AM -0700, Ethan Ongstad wrote:
Nice catch. Actually that’s how it was in the message ( Content-Type: 
Type: text/plain; charset=“utf-8”  ). After looking at it again, the 
unreadable messages do have a base64 encoding, should this be an 
issue?



On Sep 13, 2022, at 08:58, Tassilo Philipp  wrote:

not sure if related, but I noticed your line says "text:plain" instead of 
"text/plain" (which should be used according to rfc2045)


On Mon, Sep 12, 2022 at 10:20:38PM -0700, Ethan Ongstad wrote:

The messages I receive that have the line ‘ Content-Type: text:plain; charset="utf-8” ‘ 
are not readable, but it is human readable if the "utf-8" bit is capitalized or if 
there are no quotation marks. Is this a opensmtpd bug?

section 2 of RFC 2047 makes it sound like capitalization should not be relevant. 
https://www.rfc-editor.org/rfc/rfc2047#section-2 
<https://www.rfc-editor.org/rfc/rfc2047#section-2>

Re: issue with lower case utf-8 when in quotations; RFC 2047?

[Tassilo Philipp]

not sure if related, but I noticed your line says "text:plain" instead 
of "text/plain" (which should be used according to rfc2045)



On Mon, Sep 12, 2022 at 10:20:38PM -0700, Ethan Ongstad wrote:

The messages I receive that have the line ‘ Content-Type: text:plain; charset="utf-8” ‘ 
are not readable, but it is human readable if the "utf-8" bit is capitalized or if 
there are no quotation marks. Is this a opensmtpd bug?

section 2 of RFC 2047 makes it sound like capitalization should not be relevant. 
https://www.rfc-editor.org/rfc/rfc2047#section-2 

Re: certificate verification when using multiple relay hosts

[Tassilo Philipp]

This is what we're doing, but it's not directly under our control, so 
there is some back and forth, etc.. possible, yes, but the question 
still remains.
I personally lean more and more towards thinking it would be nice to be 
able to specify multiple relay hosts, explicitly.



On Thu, Sep 08, 2022 at 12:35:04AM +0200, Tobias Fiebig wrote:

Heho,
Why don't you add mailrelays.domain as a DNSAltName to the certs of these 
hosts? Or are they not under your control?

With best regards,
Tobias


-Original Message-
From: Tassilo Philipp 
Sent: Wednesday, 7 September 2022 11:31
To: misc@opensmtpd.org
Subject: certificate verification when using multiple relay hosts

Hello,

I'd like to pick y'all's brains about a TLS enabled multi-relay-host setup, 
where I'm not sure about what is right, or should maybe be possible.

The setup in question is an OpenSMTPd box that is configured to relay to multiple, explicitly specified, redundant hosts, the crucial config line 
is:


 action "relay_out" relay host smtps://mailrelays.domain

(note: whether it's using smtps or smtp w/ starttls, etc. isn't important, it 
comes down to the same)

Multiple A records are entered for the domain mailrelays.domain, so it resolves 
to multiple IPs.

This DNS-based multi-A-records setup is the only way I found to tell OpenSMTPd 
to use a list of relay hosts, and this works nicely. I verified given the logs 
and traces, that it keeps a list of them all, selects what it thinks the best 
connector is, handles a connector becoming unavailable, gracefully, etc.. Great!

However, this DNS based multi-host setup complicates matters when verifying 
certificates. Imagine that mailrelays.domain points to 1.2.3.1 and 1.2.3.2. 
Also, let's say there are specific A records pointing to those IPs, as well as 
their respective PTR records, so the full list is:

mailrelays.domain.   1.2.3.1
mailrelays.domain.   1.2.3.2

mailrelay01.domain.  1.2.3.1
mailrelay02.domain.  1.2.3.2

1.3.2.1.in-addr.arpa.mailrelay01.domain.
2.3.2.1.in-addr.arpa.mailrelay02.domain.


Also, let's say both relay hosts present certificates which only have their own 
respective DNS names listed, but *not* "mailrelays.domain".

In this case the cert verification fails when relaying mail fails, b/c 
OpenSMTPd checks whether the cert of each box has mailrelays.domain listed, 
which they do not, they only list their specific, number-suffixed domains.


By itself one could argue that this is to be expected, and I kinda agree. 
However, one could also argue that maybe it should do a PTR lookup, first, and 
use that DNS name for verification.


Taking a step back, I think the question essentially boils down to: how to 
specify multiple relay hosts (e.g. for redundancy) *by DNS name*, so the cert 
verification would work per relay host?

The problems I encountered to get this set up are:

- multiple CNAME records for a domain isn't possible in DNS

- one cannot make use of MX records, either, as the relay host line
  seems to only resolve A records in this case

- there is seemingly no way to list multiple relay hosts in smtpd.conf,
  explicitly, but maybe I'm missing something


Thoughts? I'm not sure what's right or wrong here, in some ways it behaves like 
it should, but then again it also makes it hard to specify multiple relay 
hosts, conveniently. I obviously might also totally miss something, in which 
case I would be grateful to get some feedback.

Thanks!

certificate verification when using multiple relay hosts

[Tassilo Philipp]

Hello,

I'd like to pick y'all's brains about a TLS enabled multi-relay-host 
setup, where I'm not sure about what is right, or should maybe be 
possible.


The setup in question is an OpenSMTPd box that is configured to relay to 
multiple, explicitly specified, redundant hosts, the crucial config line 
is:


 action "relay_out" relay host smtps://mailrelays.domain

(note: whether it's using smtps or smtp w/ starttls, etc. isn't 
important, it comes down to the same)


Multiple A records are entered for the domain mailrelays.domain, so it 
resolves to multiple IPs.


This DNS-based multi-A-records setup is the only way I found to tell 
OpenSMTPd to use a list of relay hosts, and this works nicely. I 
verified given the logs and traces, that it keeps a list of them all, 
selects what it thinks the best connector is, handles a connector 
becoming unavailable, gracefully, etc.. Great!


However, this DNS based multi-host setup complicates matters when 
verifying certificates. Imagine that mailrelays.domain points to 1.2.3.1 
and 1.2.3.2. Also, let's say there are specific A records pointing to 
those IPs, as well as their respective PTR records, so the full list is:


mailrelays.domain.   1.2.3.1 
mailrelays.domain.   1.2.3.2


mailrelay01.domain.  1.2.3.1 
mailrelay02.domain.  1.2.3.2


1.3.2.1.in-addr.arpa.mailrelay01.domain. 
2.3.2.1.in-addr.arpa.mailrelay02.domain.



Also, let's say both relay hosts present certificates which only have 
their own respective DNS names listed, but *not* "mailrelays.domain".


In this case the cert verification fails when relaying mail fails, b/c 
OpenSMTPd checks whether the cert of each box has mailrelays.domain 
listed, which they do not, they only list their specific, 
number-suffixed domains.



By itself one could argue that this is to be expected, and I kinda 
agree. However, one could also argue that maybe it should do a PTR 
lookup, first, and use that DNS name for verification.



Taking a step back, I think the question essentially boils down to: how 
to specify multiple relay hosts (e.g. for redundancy) *by DNS name*, so 
the cert verification would work per relay host?


The problems I encountered to get this set up are:

- multiple CNAME records for a domain isn't possible in DNS

- one cannot make use of MX records, either, as the relay host line
  seems to only resolve A records in this case

- there is seemingly no way to list multiple relay hosts in smtpd.conf,
  explicitly, but maybe I'm missing something


Thoughts? I'm not sure what's right or wrong here, in some ways it 
behaves like it should, but then again it also makes it hard to specify 
multiple relay hosts, conveniently. I obviously might also totally miss 
something, in which case I would be grateful to get some feedback.


Thanks!

Re: Forward from to another MTA

[Tassilo Philipp]

Good thinking Reio! Indeed, there's only a relay line for auth'ed mail, 
but none for this type of forwarded, local mail. You probably need a 
further, specific match line. The debug output should help you spot this 
and write a rule accordingly.


Thanks Reio


On Tue, Aug 16, 2022 at 12:39:34PM +0300, Reio Remma wrote:

Hello!

I may be wrong, but list.domain.tld might be routed back to the same 
server (domain.tld)? Maybe you should be relaying to the Sympa server?


Good luck
Reio

On 16.08.2022 12:16, thiery wrote:

On 2022-08-16 10:13, Tassilo Philipp wrote:

I might misunderstand your question, but I noticed that your line:

  match for rcpt-to  action "mailinglist"

does not specify a "from" option, so it defaults to "from local". This 
means it won't match for non-local IPs. Maybe that's the culprit?


hth


Hello,

Oh you right now I have another problem :
---
Aug 16 10:34:13 leeds smtpd[17062]: 039b2f6018e9c7ea smtp 
failed-command command="RCPT TO:" result="524 5.2.4 
Mailing list expansion problem: "

---

Errata:
---
My previous error was :
Aug 15 17:50:00 leeds smtpd[5281]: 7dae3f5b0d6ff768 smtp 
failed-command command="RCPT TO:" result="550 
Invalid recipient: "

---

Let me try to rephrase my question :

I have two server, one with OpenSMTPd who manage email for my end 
users, another manage mailing lists with Sympa.


All incoming emails on OpenSMTPd are in @domain.tld. The mailing 
list software expect something in @list.domain.tld but they are 
aliased on the OpenSMTPd server as @domain.tld.


The aliases in  contain value like this :

mailinglist1: mailinglist1 @list.domain.tld (Without the space before @)

When my users send emails to mailinglist1 @domain.tld, I want 
OpenSMTPd to forward/relay them as mailinglist1 @list.domain.tld to 
the mailing list server and to do the same for all aliases in 
 table.


How can I achieve that ? :)

Let me know if you need more clarity.

Best regards,
Yan

Re: Forward from to another MTA

[Tassilo Philipp]

I'm not sure, I'm afraid I cannot help you further with this. I guess 
the forward seems to match and work now, as your new problem seems to be 
related to theor "RCPT TO:" stuff, which isn't 
list.domain.tld..., glad we got that sorted at least.


In order to further debug this, I would recommend you start smtpd with 
-v and enable some of the traces (either through -T options or 
smtpctl(8)). IIRC, you need -v in order for any of the tracing to work. 
Then you'll get a detailed output of what match rule is used, what 
action is triggered, etc..


Good luck

On Tue, Aug 16, 2022 at 11:16:00AM +0200, thiery wrote:

On 2022-08-16 10:13, Tassilo Philipp wrote:

I might misunderstand your question, but I noticed that your line:

 match for rcpt-to  action "mailinglist"

does not specify a "from" option, so it defaults to "from local". This 
means it won't match for non-local IPs. Maybe that's the culprit?


hth


Hello,

Oh you right now I have another problem :
---
Aug 16 10:34:13 leeds smtpd[17062]: 039b2f6018e9c7ea smtp 
failed-command command="RCPT TO:" result="524 5.2.4 
Mailing list expansion problem: "

---

Errata:
---
My previous error was :
Aug 15 17:50:00 leeds smtpd[5281]: 7dae3f5b0d6ff768 smtp 
failed-command command="RCPT TO:" result="550 Invalid 
recipient: "

---

Let me try to rephrase my question :

I have two server, one with OpenSMTPd who manage email for my end 
users, another manage mailing lists with Sympa.


All incoming emails on OpenSMTPd are in @domain.tld. The mailing list 
software expect something in @list.domain.tld but they are aliased on 
the OpenSMTPd server as @domain.tld.


The aliases in  contain value like this :

mailinglist1: mailinglist1 @list.domain.tld (Without the space before @)

When my users send emails to mailinglist1 @domain.tld, I want 
OpenSMTPd to forward/relay them as mailinglist1 @list.domain.tld to 
the mailing list server and to do the same for all aliases in  
table.


How can I achieve that ? :)

Let me know if you need more clarity.

Best regards,
Yan

Re: Forward from to another MTA

[Tassilo Philipp]

I might misunderstand your question, but I noticed that your line:

  match for rcpt-to  action "mailinglist"

does not specify a "from" option, so it defaults to "from local". This 
means it won't match for non-local IPs. Maybe that's the culprit?


hth

On Tue, Aug 16, 2022 at 10:07:02AM +0200, thiery wrote:

Hello,

I have some difficulties to create rule sets for a specific use case.

I use an OpenSMPTD/Dovecot as main mail server and Sympa ( 
https://www.sympa.org/index) running on an other server to manage 
mailing list.


On the main mail server, I have two sets of aliases:

table aliases db:/etc/aliases.db
table sympa db:/etc/mail/sympa/aliases.db

The first one contain list of aliases for local accounts.
e.g. anAlias: account

The second one is a list of alias which point to another MTA dedicated 
to a mailing list.

e.g. list: l...@list.domain.tld

I want to forward all incoming emails targeting the Sympa table to the 
mailing list server.


Unfortunately when I send an email to, let's say, 
l...@list.domain.tld, I got this result :


Aug 16 09:56:36 mx-01 smtpd[16029]: 624b10db2fc80050 smtp 
failed-command command="RCPT TO:" result="550 
Invalid recipient: "


Here the smtpd.conf:

pki mx-01.domain.tld cert 
"/etc/letsencrypt/live/mx-01.domain.tld/cert.pem"
pki mx-01.domain.tld key 
"/etc/letsencrypt/live/mx-01.domain.tld/privkey.pem"


table aliases db:/etc/aliases.db
table sympa db:/etc/mail/sympa/aliases.db
table ldap ldap:/etc/mail/ldap.conf

filter check_dyndns phase connect match rdns regex { '.*\.dyn\..*', 
'.*\.dsl\..*' } \

   disconnect "550 no residential connections"

filter check_rdns phase connect match !rdns \
   disconnect "550 no rDNS"

filter check_fcrdns phase connect match !fcrdns \
   disconnect "550 no FCrDNS"

filter "rspamd" proc-exec "filter-rspamd"

smtp max-message-size "20M"

listen on enp1s0 tls pki mx-01.domain.tld auth-optional filter { 
check_dyndns, check_rdns, check_fcrdns, rspamd }
listen on enp1s0 port submission tls-require pki mx-01.domain.tld auth 
filter rspamd


# Maybe I misunderstand the `forward-only` in my use case.
action "mailinglist" forward-only alias 
action "inbound" maildir junk userbase  alias 
action "outbound" relay

match for rcpt-to  action "mailinglist"
match from any for domain "domain.tld" action "inbound"
match from auth for any action "outbound"


Any ideas ?

Have a nice day,
Yan

Re: EMails to "ORCPT=rfc822;u...@example.com" are rejected

[Tassilo Philipp]

Hello,

it looks indeed like you are facing a different issue. I helped Frank 
with that ORCPT issue he was facing, and when I saw your post I thought 
"oh, sounds familiar" and told Frank about it, and he shared his 
work/patch.
So yeah, my bad that I misinterpreted your question and didn't check 
more thoroughly if it is indeed the same problem.





On Tue, May 31, 2022 at 11:50:39AM +0200, Harald Dunkel wrote:

Hi Frank,

I am not sure if I got this correctly, but AFAIU you assume some unusual 
chars in the recipient address. There are none, according to /var/log/maillog. 
The chars in the recipient address are between 0x33 and 0x7e. And there is 
neither a '+' nor a '='.


I have the impression that the recipient address is valid.


Regards
Harri


On 2022-05-30 18:30:04, opensmtpd.open...@xpoundit.com wrote:

Hi Harri,

we had issues with e-mails containing ORCPT as well and fixed the 
rejection with a patch. Originally, we created the patch when 6.7 was 
out and applied it to the version of OpenSMTPD available in the FreeBSD 
ports.


As of today, OpenSMTPD 6.8 is available in the FreeBSD ports system. 
The attached patches can be applied to this version (if you are on 
FreeBSD, just put them into ports/mail/opensmtpd/files).


If needed, I can massage the patches so they can be applied against the 
OpenBSD base as well, where OpenSMTPD resides in nowadays (we mainly 
use it on FreeBSD). I did not do this yet, since I wanted to provide a 
quick answer.


In our case, the above mentioned groupware introduced characters in the 
ORCPT field (colons.., 0x3a), that led smtp_tx_rcpt_to() 
(usr.sbin/smtpd/smtp_session.c) to return with "553 ORCPT address 
syntax error".


RFC3461 led us to the solution we are using today.

In section 4.2 [1], the ABNF of ORCPT is defined as:


orcpt-parameter = "ORCPT=" original-recipient-address
  original-recipient-address = addr-type ";" xtext
  addr-type = atom


The log you see is:

May 27 08:42:30 mymta smtpd[10310]: f06a752b657b4a05 smtp failed- 
command command="RCPT TO:

ORCPT=rfc822;u...@example.com" result="550 Invalid recipient:
"


According to the ABNF of ORCPT, everything after addr-type ";" 
("rfc822;" in your case) is supposed to be xtext, which is described a 
bit earlier in the introductory part of section 4 [2].



xtext = *( xchar / hexchar )

xchar = any ASCII CHAR between "!" (33) and "~" (126) inclusive,
except for "+" and "=".

; "hexchar"s are intended to encode octets that cannot appear
; as ASCII characters within an esmtp-value.

hexchar = ASCII "+" immediately followed by two upper case
  hexadecimal digits


smtp_tx_rcpt_to() in usr.sbin/smtpd/smtp_session.c tries to convert the 
text of the ORCPT DSN into an e-mail address and wants to check the 
validity of the local and the domain part. We replaced this check, are 
validating the xtext portion as specified above and are replying with a 
more precise error message if this check fails:


 if (strncasecmp(opt, "rfc822;", 7) == 0)
 opt += 7;

-if (!text_to_mailaddr(>evp.dsn_orcpt, opt) ||
-!valid_localpart(tx->evp.dsn_orcpt.user) ||
-(strlen(tx->evp.dsn_orcpt.domain) != 0 &&
- !valid_domainpart(tx->evp.dsn_orcpt.domain))) {
+if (!valid_xtext(opt)) {
 smtp_reply(tx->session, 
-"553 ORCPT address syntax error");

+"553 ORCPT xtext syntax error");
 return;
 }


In usr.sbin/smtpd/util.c we added valid_xtext():

+int
+valid_xtext(const char *s)
+{
+while (*s != '\0') {
+if(*s == '\x2b' || *s == '\x3d') {
+return 0;
+} else if(*s <= '\x21' || *s >= '\x7e') {
+return 0;
+} else {
+s++;
+continue;
+}
+return 0;
+}
+return 1;
+}

I hope this helps to narrow down your issue a bit. What kind of non-e- 
mailish characters do you see in the ORCPT?


In our case, the xtext portion of the ORCPT quite often contained valid 
e-mail addresses, but sometimes, it did not. As far as we understood 
the RFCs, xtext doas not necessarily need to be an e-mail address. This 
is why we decided to replace the original check. The above mentioned 
groupware used colons as field separators inside the xtext portion to 
keep track of the communication belonging to certain thread or, well, 
recipients.


What do the others think of the way we are handling the ORCPT?

Cheers

Frank



[1] https://datatracker.ietf.org/doc/html/rfc3461#section-4.2
[2] https://datatracker.ietf.org/doc/html/rfc3461#section-4




On Mon, 2022-05-30 at 09:04 +0200, Harald Dunkel wrote:

Hi folks,

my MTA (opensmtpd on OpenBSD 7.0) rejects a few EMails with
a message like

May 27 08:42:30 mymta smtpd[10310]: f06a752b657b4a05 smtp failed- 
command command="RCPT TO:

ORCPT=rfc822;u...@example.com" result="550 Invalid recipient:
"

in /var/log/maillog. The EMails to 

Re: What are the limitations for the queue encryption key?

[Tassilo Philipp]

Looks to me as it has to have exactly 32 chars.
From crypto.c:

  #define KEY_SIZE32

  ...

  static struct crypto_ctx {
  unsigned char   key[KEY_SIZE];
  } cp;

  ...

  int
  crypto_setup(const char *key, size_t len)
  {
  if (len != KEY_SIZE)
  return 0;
  ...
  }


I only had a cursory look, so no maybe there are other checks somewhere.

hth


On Wed, Apr 20, 2022 at 03:52:38PM +0100, Josey Smith wrote:

Hi all.

I'm on OpenSMTPD 7.0.0 and am trying out queue encryption.

Almost any key that I try errors with "smtpd: crypto_setup:invalid key 
for queue encryption".


If I use "openssl rand -hex 16" (which I found in an example on 
Gilles's site - 
https://poolp.org/posts/2013-04-26/opensmtpd-table_proc-queue_proc-crypto-queue-and-other-stuff/) 
it always seems to work, but if I increase the number it often fails.


So, mostly out of curiosity, I was wondering what are the limitations 
for a valid queue encryption key?


As a side note, if I check my config (smtpd -n) while queue encryption 
is set to "-" or "stdin" I get the same error message (although the 
server still seems to work). Is that a bug?


Josey

Re: filter testing

[Tassilo Philipp]

This is very cool, thank you!
One question about the fflush() you mention: I use awk filters a lot and 
never had any need to explicitly flush, but you probably did. Mind 
sharing some details on your use case?

Thank you!

On Sat, Apr 16, 2022 at 01:04:21PM -0500, Edgar Pettijohn wrote:

I've written a perl script to help test filters. It can be found at:

http://www.pettijohn-web.com/filter-test.pl

Its not 100% complete but it should be helpful enough in finding 
common problems. Such as forgetting fflush() in awk filters or 
swapping the session id and opaque tokens.


If you run into any problems please send examples of what you expected 
vs what you got. Or patches would be great.



Edgar

Re: msgid and sending via email providers

[Tassilo Philipp]

About your original question:

From your message I deduce that the message-id is added by your client 
in the first place? In that case, no, I don't think OpenSMTPd can be 
told to basically *replace* the message id for you.


That said, OpenSMTPd as a standard compliant mail server will add or 
replace a message-id, *iff*:


- this is a submission on port 587
- and if it is missing or an incorrect (syntax) one

This is specified in [0] (RFC6409) under 8.3.

In that very case, I think the "hostname" setting of the "listen" 
directive will be used for the message-id (the handling is around line 
2750 in smtp_session.c), but might be wrong, haven't tested that. Either 
way, this wouldn't help you, I think.


Also note that the MSA adding a missing message-id is only to ensure 
compliance with standards for other MTAs, though, b/c there are too many 
broken clients out there that don't add one to begin with. This byself 
isn't ideal actually (e.g. they give examples in that RFC under 8, like 
if this is added later on, it might mess with the validity of message 
signatures).


Long story short, IMHO:
- make sure your client adds such a header
- make sure that the client adds it in the format you want


Overall, I don't think that the content of the message-id header is used 
for spam detection... but who knows, mailer corps do all kinds of weird 
stuff...



... and speaking off weird stuff, about Leo's argument about t-online: 
what he writes is actually documented as-is by t-online's guide for 
server operators [1]. They say the "mail" in the hostname is 
"recommended", however, the part about the website with imprint sounds 
like a requirement, under 4.1 of [1]:


"In particular, we recommend choosing a host name that indicates its 
usage as a mail server (eg. mail.example.com) and to ensure the host's 
domain leads to a website providing full contact details."


I personally think this is messed up and agree with Leo here. But then 
again, the big mailers do whatever they want anyways, and always make it 
harder for the small and private ones, in one way or another, for the 
sake of whatever they claim. Anyone surprised here?


[0] https://datatracker.ietf.org/doc/html/rfc6409
[1] https://postmaster.t-online.de/index.en.html



On Wed, Nov 03, 2021 at 06:14:20PM +, Manfred Lotz wrote:

On Wed, 3 Nov 2021 14:59:15 +0100
Leo Unglaub  wrote:


Hey,
i am not sure if that is directly related to you problem, but 
t-online.de is one of the worst email providers i have ever seen. 
They eandomly block you for no reason. Last month i got bocked 
because the hostname of my email server did not contain the word 
"mail". They demandet that all of the sudden. Then they only accepted 
emails from my servers if on the same IP as the mailserver there was 
a websever that served a valid impress/legal notce. Totally nuts that 
t-online.de company.




I have no problem with t-online.

Re: Pluses in addresses do not work as expected

[Tassilo Philipp]

Hello,

I cannot help you with your specific config, and don't know if opensmtp 
has problems resolving the subaddressing for 'maildir' delivery.


However, I guess you use dovecot (which I assume, given the vultr link 
you posted), so you could use LMTP to deliver the messages to dovecot's 
lmtp socket. This is what I use, and I can confirm that subaddressing 
works (not 100% certain on the details, but I think it's the MDA that 
resolves this - in this case dovecot).


hope this helps


On Sat, Jan 30, 2021 at 12:38:06PM +0100, Péter Bertalan Zoltán wrote:

Hello,

Here is my current configuration (only the relevant parts):


smtpd.conf:
---
pki foo.bp99.eu cert "/etc/ssl/foo.bp99.eu.crt"
pki foo.bp99.eu key "/etc/ssl/private/foo.bp99.eu.key"

table domains { foo.bp99.eu }
table aliases file:/etc/mail/aliases
table passwd passwd:/etc/mail/passwd
table virtuals file:/etc/mail/virtual

listen on all tls pki foo.bp99.eu
listen on all port submission tls-require pki foo.bp99.eu \
   auth 

action "local" mbox alias 
action "domain" \
   maildir "/var/vmail/%{dest.domain}/%{dest.user:lowercase|strip}" \
   virtual 
action "relay" relay helo foo.bp99.eu

match from any for domain  action "domain"
match from local for local action "local"
match from any auth for any action "relay"
---


/etc/mail/passwd:
- 
b...@foo.bp99.eu:[hash]:vmail:2000:2000:/var/vmail/foo.bp99.eu/bp99:userdb_mail=maildir:/var/vmail/foo.bp99.eu/bp99 
-



/etc/mail/virtual:
-
abuse   bp99
hostmaster  bp99
postmaster  bp99
webmaster   bp99

bp99vmail
bertalan.peter  bp99
-


My problem is that emails sent to eg  are not 
delivered to , but go into the maildir 
/var/vmail/foo.bp99.eu/bp99+sometag/.


Could you tell me what is wrong with my setup? I find all these tables 
mildly confusing. I can’t even find a mention of any 
`passwd:/path/to/file' in smtpd.conf(5). I found that on Vultr 
(https://www.vultr.com/docs/an-openbsd-e-mail-server-using-opensmtpd-dovecot-rspamd-and-rainloop). 
I think I need this nonstandard (?) credentials file so that I can point 
Dovecot to the same file for webmail authentication.



Thank you very much for your help in advance
Bertalan


--
Bertalan Z. Péter 
PGP: FB9B 34FE 3500 3977 92AE  4809 935C 3BEB 44C1 0F89





signature.asc
Description: PGP signature

Re: Failure to check FCrDNS with long DNS replies?

[Tassilo Philipp]

FYI, I run into the same issue with a different provider:
relay.yourmailgateway.de which also has a large number of A records.

Trying to reproduce and digging deeper now, by adding debug logs etc.


Interesting... thanks for checking and having thought of my report. I 
for myself didn't have any issues anymore, however, I barely ever 
receive any mail from sfr. Also, given the random order of IPs in the 
DNS reply, I simply might have had luck if it's in any case related to 
the IP order. I have no evidence for, but when I was having problems, 
the IP in question was among the last ones in the reply.


I'm curious what you'll find...

Thanks!

Re: Failure to check FCrDNS with long DNS replies?

[Tassilo Philipp]

Mhmm… but they returned different results, for dig vs OpenSMTPd filter lookup? 


Not sure, as I don't log the replies, but I don't think so.


May cache TTL have expired and record re-fetched with your local test? 
What’s your local cache software, is it able to handle large A record lists? 


It's "unbound", so yes, it should handle that just fine. The result I 
pasted was also queried through it.



In regards to the dnscrypt servers, are you sure you hit the same 
recursive resolver with dig as with OpenSMTPd filter before?


Absolutely, I enforced a single one for a test, namely soltysiak.


If you can reproduce, this would indeed point to an issue in the 
filter or local cache.  But that case should be easy test by just 
sending some test-mails from a sfr.fr  account? 


Correct. Unfortunately I don't know anyone with such an account, and 
wanted to setup just a local test, faking it, to at least be able to 
exclude opensmtp as a culprit.


In the end I just disabled the check, as one email user was desperately 
waiting for a mail that was affected, and I saw it in the logs being 
rejected over and over again, and I hade some others spinning plates to 
handle at that time. Now I have a bit more time and headspace again and 
can look into it more.



Maybe someone subscribed to this list has such an account and could send 
you a test mail?


That would be terrific!

Re: Failure to check FCrDNS with long DNS replies?

[Tassilo Philipp]

Thanks for the reply and your thoughts.


There should be nothing limit FCrDNS here, despite that
these are a lot of records.

But have you tried the dig lookup below from the actual mail
server at the time (or shortly after) the time of the failure?


Yes, that was the first thing I tried, and I had those delivery failures 
before and after that test. (In fact, I changed the error message to one 
specific to the fcrdns check, restarted opensmtp and waited for the next 
delivery attempt).


After that I started looking into the sources of OpenSMTPd and all I 
found was a loop running over all records in the reply, so yeah, no 
limitation there.




While the DNS record seems to be there and correct:
At the time of the connect your mail server was not be able to 
resolve the record through whatever you have configured as 
forwarder/lookup/recursive DNS servers.

Reasons can vary from local provider network hiccup to
global BGP issues.
Your mail server may use a different route and different
lookup servers than your local client you test dig command with.


It's a local DNS cache which forwards to some dnscrypt servers. I 
verified from the logs that my manual name resolution test I did, and 
the lookup from OpenSMTPd did use the same resolution.




I have often seen local ISP forwarding DNS servers being
blocked by other large ISP DNS servers already,
e.g. Hetzner DNS recursive Forwarders (and even
whole Hetzner netblocks) are blocked by Telekom
authoritative DNS servers, due to abuse reasons. 


Yeah, I hear you, I had similar experiences in the past, one also with 
Telekom btw., in our case it was for an online game, and we ultimately 
needed to tell players that were affected to "tunnel around" some hop in 
Frankfurt, as it was impossible to get in contact with anyone at 
Telekom that was able or willing to get us in contact with the 
technicians, there. After like 2 months that route was fine again.




I also have similar experiences the other way around with
other ISPs blocking Telekom forwarders, etc.

Sometimes you may be able to contact abuse/tech addresses
to get the relevant IPs unblocked, but often this is just
temporary anyways.

Not everything is reachable from everywhere as it should be.
This happens all the time. This is the Internet.


This is very true, this is the internet.

While looking into this I was just really surprised to see the long list 
of A records this resolves to and it felt like this was maybe the 
culprit... everything else worked fine and no other mail server 
connecting was rejected that way.


I'll reenable the fcrdns check again, and see what happens. It was 
disabled now for a while b/c of a user depending on some mails coming 
from SFR.


Thanks for the feedback!

Re: Virtual user and domain setup

[Tassilo Philipp]

I might be misunderstanding what you are trying to achieve, but it 
sounds to me that you need mail aliases.


Check the "alias" option of "action", and also the section "Aliasing 
tables" in table(5).


hth

On Sun, Apr 05, 2020 at 02:24:01PM +0200, Leo Unglaub wrote:

Hey,
first of all i want to thank you all for your work on OpenSMTPD over 
all those years. It has powered my one domain very well over all those 
years. But now i have a problem with setting up virtual domains and 
users.


My goal is the following. I have the following domains:


foo.com
bar.com


With those domains i have the following email addresses:


us...@foo.com
us...@foo.com
us...@bar.com
us...@bar.com


But all those users dont exist on my machine as real users. I just 
want to recieve emails for those accounts and process them via lmtp to 
dovecot. So my action basically looks like that:



action "local_lmtp_deliver" lmtp "/var/dovecot/lmtp"


But when i trace the lookup from the smtpd i get the following:


b4e62ea90ed6c91d smtp connected address=local host=foo.com
lookup: match "local" as NETADDR in table static: -> true 
lookup: match "foo.com" as DOMAIN in table static: -> true 
rule #1 matched: match from any for any action local_lmtp_deliver 
lookup: lookup "user1" as USERINFO in table getpwnam: -> none 
b4e62ea90ed6c91d smtp failed-command command="RCPT 
TO: " result="550 Invalid recipient: " 
b4e62ea90ed6c91d smtp disconnected reason=disconnect

debug: control -> client: pipe closed
debug: clearing p=client, fd=11, pid=0


For some reason the user1 part is still getting resolved as a real 
user on the system. I read on the man page and found the "user 
username" option for the action. I did the following:


action "local_lmtp_deliver" lmtp "/var/dovecot/lmtp" rcpt-to user "dovecot-worker" 
dovecot-worker is the account used by dovecot to handle all the email 
storage in /var/vmail. But i get the same error.


So i guess i am doing it all wrong. Could someone please be so kind 
and give me a hint in the right direction how the virtual user stuff 
is working in OpenSMTPD. Because i think i am lost here. I am doing 
something completely wrong.


I am on the latest OpenBSD release (including all syspatch).

Thanks so much!
Greetings
Leo

Re: often (but not always) two envelopes per mail in queue

[Tassilo Philipp]

Ok, I have a better idea now...


a- an envelope is created for each RCPT TO in a transaction...
b- ... and additional envelopes may be created by aliases _during_ that RCPT TO


Thanks a ton for those two pointers, I'll investigate and write what I 
can figure out.


You correctly guessed a), turns out that my MUA (msmtp, but only when 
triggered by mutt) somehow doubles all RCPT TO addresses (but not 
always, as said). I'm not sure at the moment why, but I can exclude 
OpenSMTPd from being the culprit.


I guess this was already going on for months, but previously used MSAs 
weren't under my control, so I never noticed it. Oops.


Thanks again for your precise answer which pointed me into the right 
direction, and sorry that I didn't check it more thoroughly from the get 
go.

Re: often (but not always) two envelopes per mail in queue

[Tassilo Philipp]

There is something that needs to be investigated, this is not normal no.

I won't rule out a bug in OpenSMTPD but given how envelopes are created, it is 
very unlikely:

a- an envelope is created for each RCPT TO in a transaction...
b- ... and additional envelopes may be created by aliases _during_ that RCPT TO


Thanks a ton for those two pointers, I'll investigate and write what I 
can figure out.



You provided only a partial smtpd.conf so it's hard to know if b- is possible here, 
as relaying doesn't go through aliasing which would hint at a- being the explanation. 
smtpd.conf has to be read as a whole otherwise troubleshooting is not possible.


Yes, I omitted the lines I thought were either not important (like 
pointing to the cert files), which I thought were unlikely to create 
problems (like rdns check filter regexes), ... I understand that this 
wasn't helpful. If I cannot find the issue by trying to hunt down what 
you pointed out, above, I'll break it down to the simplest possible 
config and share that one in it's entirety.



You can run smtpd with `-T smtp` to inspect SMTP transactions as they are submitted, 
you can also use `-T rules` to check which rule was matched and make sure it is that 
match auth and not another one that got hit first.


Thanks, this will come in useful!

often (but not always) two envelopes per mail in queue

[Tassilo Philipp]

Hello,

I noticed that for most emails I submit through my instance of OpenSMTP, 
there are most of the time (but interestingly not always) two envelopes 
in the queue. However, mail delivery works fine, and only one copy 
arrives at the recipient.


To illustrate the case, sending a mail (sensitive info replaced), 
results in the queue to be before relay:


# smtpctl show
10da45aca8ae47a5|inet4|mta|auth|sen...@snd.com|recei...@rcv.com|recei...@rcv.com|1580120435|1580120435|1580120436|0|inflight|2|
10da45acc22f61bd|inet4|mta|auth|sen...@snd.com|recei...@rcv.com|recei...@rcv.com|1580120435|1580120435|1580120436|0|inflight|2|

More info on both envelopes:

# smtpctl show envelope 10da45aca8ae47a5
version: 3
dispatcher: relay
tag: MSA
type: mta
smtpname: mx.ownmailserver.com
helo: localhost
hostname: 
sockaddr: 
sender: sen...@snd.com
rcpt: recei...@rcv.com
dest: recei...@rcv.com
ctime: 1580120435
last-try: 0
last-bounce: 0
ttl: 0
retry: 0
flags: authenticated
dsn-notify: 0


# smtpctl show envelope 10da45acc22f61bd
version: 3
dispatcher: relay
tag: MSA
type: mta
smtpname: mx.ownmailserver.com
helo: localhost
hostname: 
sockaddr: 
sender: sen...@snd.com
rcpt: recei...@rcv.com
dest: recei...@rcv.com
ctime: 1580120435
last-try: 0
last-bounce: 0
ttl: 0
retry: 0
flags: authenticated
dsn-notify: 0


Looks like they are exact duplicates, but with their distinct envelope 
IDs.  Here's the relay log:


Jan 27 10:20:43 schacht01 smtpd[50381]: 11490b9d19a2b420 mta delivery evpid=10da45aca8ae47a5 from= to= 
rcpt=<-> source="" relay="" delay=8s result="Ok" stat="250 2.6.0 
<2020015810049931962...@snd.com> [InternalId=3259974] Queued mail for delivery"
Jan 27 10:20:43 schacht01 smtpd[50381]: 11490b9d19a2b420 mta delivery evpid=10da45acc22f61bd from= to= 
rcpt=<-> source="" relay="" delay=8s result="Ok" stat="250 2.6.0 
<2020015810049931962...@snd.com> [InternalId=3259974] Queued mail for delivery"


So, both are relayed, it seems, however, on the receiving side only one 
copy arrives, and no errors reported. I guess that if there is a double 
relay, the receiver might still only accept one copy, b/c the Message-Id 
being the same? Or isn't there a double relay to begin with?


I'm also a bit puzzled why in rare cases, only one envelope is in the 
queue - what I would expect - but I cannot find why or what the 
difference is.



I have only one thing listening on the submission port:

# netstat -4an | grep -F .587
tcp4   0  0 .587 *.*LISTEN


The corresponding smtp.conf's config file line is, as well as the used 
action/match pair:


listen on $listen_ext port submission tls-require pki mx_pki auth hostname $mx_domain senders 
 mask-src tag "MSA"

action "relay" relay helo $mx_domain
match auth from any for any action "relay"



So I guess my question is very simple in nature: Is this normal 
behaviour, or is there something going on that needs to be investigated?


Thank you

getting username from a client certificate field?

[Tassilo Philipp]

Hello,

thank you for writing OpenSMTPD, it's working great!

Is it possible to somehow get a username from a client certificate? This would
allow for user specific auth based on a certificate, instead of providing a
username via standard SMTP auth.

The use case I am thinking of would be using a "listen" line with "tls-require
verify", but without "auth", to relay mail from connections that provided
valid client certificates. The problem is that without "auth" there is no
username, so it's not possible to make use of "senders " to
restrict submission to only select client certificates.

Maybe something like "tls-require verify username-from CN" could be a possible
extension to be able to use (or override) the username and select the cert's
field the username is to be picked from.

Some other have this, e.g. dovecot's auth_ssl_username_from_cert, for example.

Or am I missing something and this is already possible somehow?

Thank you