Re: Virtual alias address class and no_address_mappings
On Mon, Dec 30, 2019 at 06:27:30PM +1300, Peter wrote: > > http://www.postfix.org/DEBUG_README.html#mail > > > > * postconf -nf content_filter = scan:127.0.0.1:10025 inet_interfaces = all receive_override_options = no_address_mappings virtual_alias_domains = [domain].com virtual_alias_maps = pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf hash:/etc/postfix/virtual_alias_maps With "ldap", "pgsql" and "mysql" it is generally a good idea to use "proxy:ldap", "proxy:pgsql", ... > > * postconf -Mf smtp inet n - - - - smtpd scan unix - - n - 10 smtp -o smtp_send_xforward_command=yes -o disable_mime_output_conversion=yes -o smtp_generic_maps= localhost:10026 inet n - n - 10 smtpd -o content_filter= -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks -o smtpd_helo_restrictions= -o smtpd_client_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o mynetworks=127.0.0.0/8 -o smtpd_authorized_xforward_hosts=127.0.0.0/8 > > * logs showing all processing of a rejected message Dec 30 00:59:13 li1145-60 postfix/smtpd[4934]: 84E583E814: client=mail-vs1-f52.google.com[209.85.217.52] Dec 30 00:59:13 li1145-60 postfix/cleanup[4939]: 84E583E814: message-id= Dec 30 00:59:13 li1145-60 postfix/qmgr[4761]: 84E583E814: from=, size=2491, nrcpt=1 (queue active) Dec 30 00:59:13 li1145-60 postfix/error[4940]: 84E583E814: to=, relay=none, delay=0.2, delays=0.2/0/0/0, dsn=5.1.1, status=bounced (User unknown in virtual alias table) Dec 30 00:59:13 li1145-60 postfix/bounce[4941]: 84E583E814: sender non-delivery notification: 8ECE03EA80 Dec 30 00:59:13 li1145-60 postfix/qmgr[4761]: 84E583E814: removed with "XFORWARD" the logs could be misleading, this could be the processing of the message south of the content filter, and perhaps the recipient is not listed in virtual_alias_maps. Is there any other logging for "84E583E814" or the message-id in question? If the "content_filter" was set as reported, the message would not have been handed off to the error transport. The presence of a filter: https://github.com/vdukhovni/postfix/blob/master/postfix/src/qmgr/qmgr_message.c#L1105-L1115 preëmpts recipient-specific resolution: https://github.com/vdukhovni/postfix/blob/master/postfix/src/qmgr/qmgr_message.c#L1121-L1127 that (for example) bounces unknown recipients in virtual alias domains. The reported symptoms are not consistent with content_filter being set for the message. -- Viktor.
Re: Virtual alias address class and no_address_mappings
On 30/12/19 5:15 pm, Viktor Dukhovni wrote: On Mon, Dec 30, 2019 at 04:37:32PM +1300, Peter wrote: If someone uses virtual_address_domains and has "receive_override_options = no_address_mappings", then postfix will kick back an error of "User unknown in virtual alias table" for any recipient in that domain. Addresses in virtual_alias_domains resolve to the error transport, unless rewritten to a "real domain", typically via virtual_alias_maps. That makes sense. I'm seeing the error transport in the logs. But, content_filter bypasses transport resolution, since all recipients go to the filter transport, so the below is not what typically happens. But if that's the case then it shouldn't be going to the error transport. User has a content filter configured which re-injects to postfix on another port which does not have no_address_mappings set. The intent here is for address mappings to be ignored and the content filter run initially then upon re-injection for the address mappings to be processed and the message routed as appropriate. How is the content filter specified? Is this a multi-instance configuration with "default_transport" set to the filter, or a single instance configuration with "-o content_filter=" in the pre-filter master.cf SMTP listener? Config and logs linked below. Because no_address_mappings is set and the recipient address is in the virtual alias address class, however, the message instead gets rejected initially with the above mentioned error. That is not what happens when "content_filter" is specified. Then I got it wrong. I tried to follow it in the source but got lost very quickly and gave up. Is this behavior intentional or just a side effect of how no_address_mappings is implemented? http://www.postfix.org/DEBUG_README.html#mail Post: Ok, I needed to get permission from the OP to repost his info * postconf -nf * postconf -Mf https://pastebin.com/raw/cfiTsNJD (sorry pastebin.com was the ops choice but I gave the raw link at least) * logs showing all processing of a rejected message https://dpaste.org/FxmW/raw Peter
Re: Virtual alias address class and no_address_mappings
On Mon, Dec 30, 2019 at 04:37:32PM +1300, Peter wrote: > If someone uses virtual_address_domains and has > "receive_override_options = no_address_mappings", then postfix will > kick back an error of "User unknown in virtual alias table" for any > recipient in that domain. Addresses in virtual_alias_domains resolve to the error transport, unless rewritten to a "real domain", typically via virtual_alias_maps. But, content_filter bypasses transport resolution, since all recipients go to the filter transport, so the below is not what typically happens. > User has a content filter configured which re-injects to postfix on > another port which does not have no_address_mappings set. The intent > here is for address mappings to be ignored and the content filter run > initially then upon re-injection for the address mappings to be > processed and the message routed as appropriate. How is the content filter specified? Is this a multi-instance configuration with "default_transport" set to the filter, or a single instance configuration with "-o content_filter=" in the pre-filter master.cf SMTP listener? > Because no_address_mappings is set and the recipient address is in the > virtual alias address class, however, the message instead gets > rejected initially with the above mentioned error. That is not what happens when "content_filter" is specified. > Is this behavior intentional or just a side effect of how > no_address_mappings is implemented? http://www.postfix.org/DEBUG_README.html#mail Post: * postconf -nf * postconf -Mf * logs showing all processing of a rejected message -- Viktor.
Virtual alias address class and no_address_mappings
I'm not entirely certain if this is intentional or not, but I ran across this one with someone in IRC just now. If someone uses virtual_address_domains and has "receive_override_options = no_address_mappings", then postfix will kick back an error of "User unknown in virtual alias table" for any recipient in that domain. This is probably because postfix doesn't load or access virtual_alias_maps if no_address_mappings is set. This can be problematic in the following scenario: User has a content filter configured which re-injects to postfix on another port which does not have no_address_mappings set. The intent here is for address mappings to be ignored and the content filter run initially then upon re-injection for the address mappings to be processed and the message routed as appropriate. Because no_address_mappings is set and the recipient address is in the virtual alias address class, however, the message instead gets rejected initially with the above mentioned error. A better approach might be to read the virtual_alias_maps in order to allow the message to pass the recipient check but to still not use it for address mappings if no_address_mappings is set. Is this behavior intentional or just a side effect of how no_address_mappings is implemented? Peter
Re: what's an encrypted mail service?
Eliza Chan: > Does it mean messages get encrypted in tranfer process and message > storage is encrypted in rest? Every provider can TLS encrypt your SMTP mail in transit, but the provider still has access to the plaintext that goes into and that comes out of TLS-over-SMTP. Many providers encrypt mail at rest. The main differentiator is who can decrypt your mail at rest. Wietse
Re: what's an encrypted mail service?
On Mon, Dec 30, 2019 at 10:28:03AM +0800, Eliza Chan wrote: > Does it mean messages get encrypted in transfer process and message storage is > encrypted in rest? That's a plausible definition, but transfer may not always be encrypted, since the provided typically would not refuse inbound mail that did not use STARTTLS. The responsibility to use STARTTLS falls on the SMTP client. > Why so many encrypted mail providers like protonmail, tunanota say > they have privacy advantages than traditional mail providers (for > example, gmail)? If they also encrypt messages at rest, then it is harder for unauthorised parties to access message plaintext. But you can read their documentation for any details that substantiate the marketing blurbs. -- Viktor.
what's an encrypted mail service?
Does it mean messages get encrypted in tranfer process and message storage is encrypted in rest? Why so many encrypted mail providers like protonmail, tunanota say they have privacy advantages than traditional mail providers (for example, gmail)? Thanks for any guide. Eliza
Re: smtpd_policy_service_timeout question
Hi Scott On Sun, Dec 29, 2019 at 10:04:39AM -0500, Scott Kitterman wrote: > For the policy server in question, the 100s default should be more than 2x > the > maximum time the policy request can take, even if DNS is very slow, The policy service is supposed to do proper timeouts for everything it requests. The timeouts in Postfix should be considered as last-resort, for when the policy service itself broke. Bastian -- Another dream that failed. There's nothing sadder. -- Kirk, "This side of Paradise", stardate 3417.3
Re: Postfix Maildir problems - SOLVED(?)
Op 29-12-19 om 16:29 schreef Wietse Venema: Richard Rasker: So here is a striking difference between the old and the new machine that I don't understand. Is there a way to figure out which process is actually dumping the mail in /var/spool/mail? Because if it is still maildrop, it isn't logging anything as it should. Postfix logs all deliveries. Here is an example that you posted earlier: Dec 28 21:54:08 mail postfix/local[2755]: 26F111E0377: to=, relay=local, delay=0.01, delays=0/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to command: /usr/bin/maildrop) Look for "status=sent" lines in your log. That will show whether /usr/bin/maildrop is involved. All logged mail events (sending, receiving) have "status=sent" in the second-last logged line in mail.log (the last line saying that it's removed from the queue). But some progress happened, even though I don't really understand how. Up until two hours ago, mail *always* ended up in /var/spool/mail, even when I deleted "/usr/bin/maildrop" after mailbox_command = in main.cf, and the log entries also confirming that it was delivered to the maildir: ... status=sent (delivered to maildir) (I can match these entries with messages in /var/spool/mail, not with any messages in any Maildir directory as I would expect). Yet now, all of a sudden, new test messages show up in the correct Maildir folders. The one thing I did apart from changing the mailbox_command parameter back and forth was making the file /var/spool/mail/[username] read-only (chmod 400). This never resulted in any error messages BTW. (And yes, main.cf specified home_mailbox = Maildir/ all the time -- as already stated in my very first message here.) So I have no idea what fixed at least the behaviour of postfix, and maildrop still refuses to work properly (i.e. it still sends mail to /var/spool/mail, and refuses to log anything), but at least I have a functional maildir system -- also after completely rebooting the box just to make sure. I'll try and sort out the maildrop conundrum later -- and as I never actually used its filtering capabilities in the first place, I might even leave it at this. Thank you once again for your hints and support! Best regards, Richard
Re: Postfix Maildir problems
Richard Rasker: > So here is a striking difference between the old and the new machine > that I don't understand. Is there a way to figure out which process is > actually dumping the mail in /var/spool/mail? Because if it is still > maildrop, it isn't logging anything as it should. Postfix logs all deliveries. Here is an example that you posted earlier: Dec 28 21:54:08 mail postfix/local[2755]: 26F111E0377: to=, relay=local, delay=0.01, delays=0/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to command: /usr/bin/maildrop) Look for "status=sent" lines in your log. That will show whether /usr/bin/maildrop is involved. Wietse
Re: smtpd_policy_service_timeout question
On Sunday, December 29, 2019 9:01:12 AM EST Wietse Venema wrote: > Scott Kitterman: > > If Postfix smtpd is waiting for a response to an in progress request > > An SMTP client request or policy request? Policy request. > > and the smtpd_policy_service_timeout is reached, does Postfix keep > > the pipe open until that request completes or does it close it > > right away? > > smtpd_policy_service_timeout is the limit for connecting to, writing > to, or receiving from a policy server. > > - If this expires while the Postfix SMTP server is waiting for an > SMTP client request, nothing happens. > > - If this expires while writing to, or receiving from a policy > server, the Postfix SMTP server will disconnect from the policy > server, and new connection, and retry the policy request. > > > Is it sufficient to set it to slightly higher than the maximum > > time the policy service is supposed to run for a single request > > are should it be substantially higher than that? > > I would not recommend to set timeouts just a little higher than > needed. If, not when, some system becomes overloaded, having 'tight' > timeouts will just make things worse. > > Wietse For the policy server in question, the 100s default should be more than 2x the maximum time the policy request can take, even if DNS is very slow, but there are configuration options users might select that could make it a much closer thing. That clarifies it for me enough that I believe I can get my documentation correct. Thanks, Scott K
Re: Postfix Maildir problems
Hello, Op 29-12-19 om 05:18 schreef @lbutlr: On 28 Dec 2019, at 14:54, Richard Rasker wrote: Everything went very smooth, and everything works (sending mail, receiving mail, authentication, certificates, IMAP folders showing in the mail client (Thunderbird)) -- except for the very last thing: received mail ends up in /var/spool/mail/[username] instead of the users' Maildir folders. Postfix logs what happens to the message, and postfix says the message was delivered. Postfix its at that point out of the picture, and it is your courier setup that is not correct. This may well be the case. Also you say “identical setup” but obviously it cannot be actually identical. Why do you think it is identical? I checked the following configuration files on both the old and the new server, and they are literally identical: /etc/default/courier: # This file is automatically generated by maintainer scripts. # You may modifiy this file, but additional values and format # modifications will not be preserved. MAILDIR="Maildir" /etc/default/maildroprc: logfile "/var/log/maildrop" VERBOSE="5" log "==" /etc/courier/authdaemonrc: ##VERSION: $Id: authdaemonrc.in,v 1.13 2005/10/05 00:07:32 mrsam Exp $ [etc.] /etc/courier/imapd: ##VERSION: $Id: imapd.dist.in,v 1.41 2008/06/21 16:01:23 mrsam Exp $ [etc.] Postfix' main.cf file has some minor differences with regard to tls and smtpd_client_restrictions, but those should not have any bearing on the final delivery of mail. I could not locate any other configuration files -- and if there are any more, I never touched them on either machine. It’s been a long time since I used Courier, but commenting out the mail drop line in main.cf and reloading postfix should disable mail drop. That is not going to fix your problem, but postfix will deliver the mail to your home_mailbox, assuming your users have a $HOME. At one point, I had the entry mailbox_command=/usr/bin/maildrop in main.cf, and when I executed # postconf -e "home_mailbox = Maildir/" both mail.log and syslog threw this warning (and in multiple instances from several active processes): syslog.1:Dec 28 21:07:05 mail configure-instance.sh[5021]: postconf: warning: /etc/postfix/main.cf, line 81: overriding earlier entry: mailbox_command=/usr/bin/maildrop syslog.1:Dec 28 21:07:05 mail postmulti[5087]: warning: /etc/postfix/main.cf, line 81: overriding earlier entry: mailbox_command=/usr/bin/maildrop syslog.1:Dec 28 21:07:05 mail postmulti[5087]: postsuper: warning: /etc/postfix/main.cf, line 81: overriding earlier entry: mailbox_command=/usr/bin/maildrop [etc] At the moment, these warnings no longer appear when restarting postfix, regardless of the main.cf entry wrt. home_mailbox -- I don't know why. When you say no *significant* difference in conf files, that implies there is some difference. Your problem likely lies in those differences. I haven't been able to identify any differences so far, except the ones in main.cf that I mentioned already. Does maildrop log its actions? It should -- but here I find a strange difference: the maildrop log on the new machine stays empty. Receiving a message on the old machine results in the following maildrop log entry: == Date: Sun Dec 29 14:44:11 2019 From: r...@linetec.nl Subj: File: /home/rr/./Maildir (806) Yet nothing is logged at all in /var/log/maildrop on the new machine -- suggesting that maildrop isn't working properly there. Still, combing through syslog and other logs for 'maildrop' or even 'mail' does not reveal any relevant errors or warnings that may relate to this. So here is a striking difference between the old and the new machine that I don't understand. Is there a way to figure out which process is actually dumping the mail in /var/spool/mail? Because if it is still maildrop, it isn't logging anything as it should. Would it be safe to (temporarily) change something about /var/spool/mail (e.g. write permissions) to force an error message? I could provide all configuration files (main.cf, master.cf), but I'm not sure which ones may be relevant, so I won't dump it all here right away. Postfix configuration files are unlikely to help as the log line you posted shows postfix delivers the message to maildrop. More likely you need help from a courier list? That may be a very good idea. Thank you (and the others here) for your suggestions and support. I'll try to find out what is happening -- although after almost two days of fruitless searching and tinkering, I am tempted to make a back-up of the current situation, totally purge postfix, courier, amavisd-new etc. from the machine, and start with a fresh installation of those packages. I must have done something wrong early on, messing up or overlooking one tiny setting or so that I haven't found yet, but for the life of me I can't figure out
Re: smtpd_policy_service_timeout question
Scott Kitterman: > If Postfix smtpd is waiting for a response to an in progress request An SMTP client request or policy request? > and the smtpd_policy_service_timeout is reached, does Postfix keep > the pipe open until that request completes or does it close it > right away? smtpd_policy_service_timeout is the limit for connecting to, writing to, or receiving from a policy server. - If this expires while the Postfix SMTP server is waiting for an SMTP client request, nothing happens. - If this expires while writing to, or receiving from a policy server, the Postfix SMTP server will disconnect from the policy server, and new connection, and retry the policy request. > Is it sufficient to set it to slightly higher than the maximum > time the policy service is supposed to run for a single request > are should it be substantially higher than that? I would not recommend to set timeouts just a little higher than needed. If, not when, some system becomes overloaded, having 'tight' timeouts will just make things worse. Wietse
