Re: lost connection while sending end of data

[Wietse Venema]

Christos Chatzaras:
> I use dovecot lmtp, dovecot quota plugin and postfix.
> 
> When I send e-mail to 2 recipients (or more) at the same time and if one of 
> them is over quota (or under quota and the message I send is bigger than his 
> free space) mailq shows:
> 
> -Queue ID-  --Size-- Arrival Time -Sender/Recipient---
> 20B03336F2226099 Thu Apr 19 18:02:47  supp...@example.com
> (lost connection with server25.example.org[private/dovecot-lmtp] while 
> sending end of data -- message may be sent more than once)
> us...@example.com
> us...@example.com
> 
> E-mails sent from the same domain on same server so it's a local delivery.
> 
> If I send the e-mail to the over quota user ( only him on To: ) then I get a 
> bounce that says that user is over quota which is the correct behaviour.
> 
> By changing the postfix main.cf setting from the default:
> 
> default_destination_recipient_limit = 50
> 
> to:
> 
> default_destination_recipient_limit = 1
> 
> it solves the issue.
> 
> Is this a known bug?

Please see the instructions in the mailing list welcome message.

Wietse

Re: automatic email account configuration, postfix pipelining restriction

[Wietse Venema]

David Mehler:
> Hi,
> 
> It's Thunderbird 52.7. Is there a workaround to make this work?

Yes, do nothing. In particular, do not use the Postfix
reject_unauth_pipelining feature, because that would trigger
a REJECT response.

Wietse
 
> On 4/20/18, Viktor Dukhovni  wrote:
> >
> >
> >> On Apr 20, 2018, at 4:52 PM, David Mehler  wrote:
> >>
> >> I'm atempting to configure email autoconfig and autodiscover services
> >> for Mozilla and Microsoft clients. I'm using Postfix 3.3. At first I
> >> thought I was dealing with either an Apache or Dovecot issue, now I'm
> >> thinking it's an error with my Postfix configuration.
> >>
> >> Whenever I atempt a connection I'm getting this in my postfix error log
> >> file:
> >>
> >> Apr 20 14:37:00 hostname postfix/submission/smtpd[92360]: improper
> >> command pipelining after EHLO from Connecting-Machine-Hostname-And-IP:
> >> QUIT\r\n
> >
> > This client does not implement SMTP correctly.  There's nothing wrong
> > with the Postfix configuration.  The client MUST wait for the EHLO
> > response *before* sending QUIT.
> >
> > --
> > Viktor.
> >
> >
> 

Re: automatic email account configuration, postfix pipelining restriction

[David Mehler]

Hi,

It's Thunderbird 52.7. Is there a workaround to make this work?

Thanks.
Dave.


On 4/20/18, Viktor Dukhovni  wrote:
>
>
>> On Apr 20, 2018, at 4:52 PM, David Mehler  wrote:
>>
>> I'm atempting to configure email autoconfig and autodiscover services
>> for Mozilla and Microsoft clients. I'm using Postfix 3.3. At first I
>> thought I was dealing with either an Apache or Dovecot issue, now I'm
>> thinking it's an error with my Postfix configuration.
>>
>> Whenever I atempt a connection I'm getting this in my postfix error log
>> file:
>>
>> Apr 20 14:37:00 hostname postfix/submission/smtpd[92360]: improper
>> command pipelining after EHLO from Connecting-Machine-Hostname-And-IP:
>> QUIT\r\n
>
> This client does not implement SMTP correctly.  There's nothing wrong
> with the Postfix configuration.  The client MUST wait for the EHLO
> response *before* sending QUIT.
>
> --
>   Viktor.
>
>

Re: automatic email account configuration, postfix pipelining restriction

[Viktor Dukhovni]

> On Apr 20, 2018, at 4:52 PM, David Mehler  wrote:
> 
> I'm atempting to configure email autoconfig and autodiscover services
> for Mozilla and Microsoft clients. I'm using Postfix 3.3. At first I
> thought I was dealing with either an Apache or Dovecot issue, now I'm
> thinking it's an error with my Postfix configuration.
> 
> Whenever I atempt a connection I'm getting this in my postfix error log file:
> 
> Apr 20 14:37:00 hostname postfix/submission/smtpd[92360]: improper
> command pipelining after EHLO from Connecting-Machine-Hostname-And-IP:
> QUIT\r\n

This client does not implement SMTP correctly.  There's nothing wrong
with the Postfix configuration.  The client MUST wait for the EHLO
response *before* sending QUIT.

-- 
Viktor.

Re: Postfix, milters and quarantine actions

[Viktor Dukhovni]

> On Apr 20, 2018, at 5:40 PM, J Doe  wrote:
> 
> Hello,
> 
> I had some questions regarding milters in general, with the questions 
> initially focused on the OpenDKIM milter (version 2.10.3), on Postfix 3.1.0

Look for the word "quarantine" in http://www.postfix.org/MILTER_README.html

-- 
Viktor.

lost connection while sending end of data

[Christos Chatzaras]

I use dovecot lmtp, dovecot quota plugin and postfix.

When I send e-mail to 2 recipients (or more) at the same time and if one of 
them is over quota (or under quota and the message I send is bigger than his 
free space) mailq shows:

-Queue ID-  --Size-- Arrival Time -Sender/Recipient---
20B03336F2226099 Thu Apr 19 18:02:47  supp...@example.com
(lost connection with server25.example.org[private/dovecot-lmtp] while sending 
end of data -- message may be sent more than once)
us...@example.com
us...@example.com

E-mails sent from the same domain on same server so it's a local delivery.

If I send the e-mail to the over quota user ( only him on To: ) then I get a 
bounce that says that user is over quota which is the correct behaviour.

By changing the postfix main.cf setting from the default:

default_destination_recipient_limit = 50

to:

default_destination_recipient_limit = 1

it solves the issue.

Is this a known bug?

Postfix, milters and quarantine actions

[J Doe]

Hello,

I had some questions regarding milters in general, with the questions initially 
focused on the OpenDKIM milter (version 2.10.3), on Postfix 3.1.0

In man 5 opendkim.conf, under the CaptureUnknownErrors parameter, it specifies:

When set, and on systems where MTA quarantine is available, the 
filter will request quarantine of a message that results in an internal
error or resource exhaustion.

My questions are:

1. Is this supported on Postfix 3.1.0 and later ?

2. If it is supported on Postfix 3.1.0, is this because quarantine 
functionality is exposed to milters ?
(ie: I know quarantine functionality is not a part of SMTP, so it’s not the 
milter talking via
SMTP to Postfix)

3. If it is supported, is this available for other milters that request 
quarantine (ie: OpenDMARC, etc.) ?

Thanks,

- J

automatic email account configuration, postfix pipelining restriction

[David Mehler]

Hello,

I'm atempting to configure email autoconfig and autodiscover services
for Mozilla and Microsoft clients. I'm using Postfix 3.3. At first I
thought I was dealing with either an Apache or Dovecot issue, now I'm
thinking it's an error with my Postfix configuration.

Whenever I atempt a connection I'm getting this in my postfix error log file:

Apr 20 14:37:00 hostname postfix/submission/smtpd[92360]: improper
command pipelining after EHLO from Connecting-Machine-Hostname-And-IP:
QUIT\r\n

Suggestions welcome.
Thanks.
Dave.

If it helps here's my postfix master.cf and main.cf files:
#cat master.cf
smtp  inet  n   -   n   -   -   smtpd
#smtp  inet  n   -   n   -   1   postscreen
 #-o smtpd_sasl_auth_enable=no
#smtpd pass  -   -   n   -   -   smtpd
dnsblog   unix  -   -   n   -   0   dnsblog
tlsproxy  unix  -   -   n   -   0   tlsproxy
# Submission port 587 for client connection / sending mails from
authenticated users
submission inet n   -   n   -   -   smtpd
 -o syslog_name=postfix/submission
 # for opportunistic smtpd
  #-o smtpd_tls_security_level=may
 # Encrypt by default
  -o smtpd_tls_dh1024_param_file=/etc/ssl/dhparam.pem
 -o smtpd_tls_security_level=encrypt
 -o smtpd_sasl_auth_enable=yes
 -o smtpd_sasl_type=dovecot
 -o smtpd_sasl_path=private/auth
 -o smtpd_sasl_security_options=noanonymous
 -o smtpd_client_restrictions=$mua_client_restrictions
 -o smtpd_sender_restrictions=$mua_sender_restrictions
 -o smtpd_relay_restrictions=$mua_relay_restrictions
 -o 
smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject
 -o smtpd_sender_login_maps=mysql:/usr/local/etc/postfix/db/sender-login-maps.cf
 -o tls_preempt_cipherlist=yes
#smtps inet  n   -   n   -   -   smtpd
  #-o syslog_name=postfix/smtps
  #-o smtpd_tls_wrappermode=yes
  #-o smtpd_sasl_auth_enable=yes
  #-o smtpd_reject_unlisted_recipient=no
  #-o 
smtpd_recipient_restrictions=permit_sasl_authenticated,permit_mynetworks,reject
  #-o tls_preempt_cipherlist=yes
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
#628   inet  n   -   n   -   -   qmqpd
pickupunix  n   -   n   60  1   pickup
cleanup   unix  n   -   n   -   0   cleanup
qmgr  unix  n   -   n   300 1   qmgr
#qmgr unix  n   -   n   300 1   oqmgr
tlsmgrunix  -   -   n   1000?   1   tlsmgr
rewrite   unix  -   -   n   -   -   trivial-rewrite
bounceunix  -   -   n   -   0   bounce
defer unix  -   -   n   -   0   bounce
trace unix  -   -   n   -   0   bounce
verifyunix  -   -   n   -   1   verify
flush unix  n   -   n   1000?   0   flush
proxymap  unix  -   -   n   -   -   proxymap
proxywrite unix -   -   n   -   1   proxymap
smtp  unix  -   -   n   -   -   smtp
relay unix  -   -   n   -   -   smtp
#   -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq unix  n   -   n   -   -   showq
error unix  -   -   n   -   -   error
retry unix  -   -   n   -   -   error
discard   unix  -   -   n   -   -   discard
local unix  -   n   n   -   -   local
virtual   unix  -   n   n   -   -   virtual
lmtp  unix  -   -   n   -   -   lmtp
anvil unix  -   -   n   -   1   anvil
scacheunix  -   -   n   -   1   scache

# for SPF support
spf-policy unix -   n   n   -   0   spawn
  user=vmail argv=/usr/local/bin/perl
/usr/local/libexec/postfix-policyd-spf-perl

dfilt unix-   n   n   -   -   pipe
flags=Rq user=filter argv=/usr/local/etc/postfix/disclaimer -f
${sender} -r ${recipient}

# scan service for clamsmtpd
scan unix -   -   n   -   16   smtp
   -o smtp_data_done_timeout=1200
   -o smtp_send_xforward_command=yes
   -o disable_dns_lookups=yes

127.0.0.1:10026 inet n   -   n   -   16   smtpd
   -o content_filter=
   -o local_recipient_maps=
   -o relay_recipient_maps=
   -o smtpd_restriction_classes=
   -o smtpd_client_restrictions=
   -o smtpd_helo_restrictions=
   -o smtpd_sender_restrictions=
   -o smtpd_recipient_restrictions=permit_mynetworks,reject
   -o mynetworks_style=host

Re: Read Only account

[Viktor Dukhovni]

> On Apr 20, 2018, at 3:40 PM, @lbutlr  wrote:
> 
> How would I configure a user so that they could only read mail and not send 
> any mail (even to local users).

If you accept mail from strangers on port 25, and the user can reach
port 25 on your inbound MX host, then you can't prevent him from 
impersonating some stranger.  Authentication on port 25 is not
required.  You could firewall-off inbound port 25 from hosts on
your network, forcing the user to go off-site to send the "forbidden"
email.

-- 
-- 
Viktor.

Re: Read Only account

[/dev/rob0]

On Fri, Apr 20, 2018 at 03:53:17PM -0400, Kevin A. McGrail wrote:
> On 4/20/2018 3:40 PM, @lbutlr wrote:
> > How would I configure a user so that they could only read mail 
> > and not send any mail (even to local users).
> >
> Different auth for POP or IMAP vs SMTP?

Or in the SASL backend, have this user's credentials not be valid for 
SMTP, or in Postfix a check_sasl_access lookup to reject this user's 
mail.  Lots of ways.  These will also necessitate that you require 
your users to AUTH; no permit_mynetworks nor similar IP-based relay 
allowances.

If perchance this is a shell user, don't forget to exclude him/her 
from your authorized_submit_users, to prevent sendmail submission.
-- 
  http://rob0.nodns4.us/
  Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:

Re: Read Only account

[Kevin A. McGrail]

On 4/20/2018 3:40 PM, @lbutlr wrote:
> How would I configure a user so that they could only read mail and not send 
> any mail (even to local users).
>
Different auth for POP or IMAP vs SMTP?

Read Only account

[@lbutlr]

How would I configure a user so that they could only read mail and not send any 
mail (even to local users).

Re: Mails stuck in queue until inflow stops

[Stephen Satchell]

On 04/20/2018 11:12 AM, Bastian Blank wrote:

If your application eats up all the memory, then you won't get any
useful message rate outgoing.



And worse, if you overflow DRAM, you now add swap load to the disk, 
which further slows things down.  One MUST avoid going into swap if 
possible, or have an extremely fast backing store (like SSD).

Re: Mails stuck in queue until inflow stops

[Viktor Dukhovni]

> On Apr 20, 2018, at 2:12 PM, Bastian Blank 
>  wrote:
> 
>> bounce_queue_lifetime = 5d
> 
> I thought this is a system that should move mails as fast as possible
> outgoing.  Why would it ever handle bounces?

The application may want to collect bounces for list management.

>> debug_peer_level = 2
> 
> Don't use that.

This is in the upstream default main.cf, provided debug_peer_list is empty, it 
is harmless.

>> inet_interfaces = 127.0.0.1
> 
> Only listen on localhost.  So you actually have the application and
> postfix battling on resources on this system?  Why?

If the application sends the *same* message to many users, it would
be dramatically more efficient to send many users in a single envelope,
with VERP if per-user bounces are wanted.
> 
>> lmtp_destination_concurrency_limit = 30
> 
> And there you have lmtp.

Harmless if unused.

> 
>> lmtp_line_length_limit = 990
> 
> You can't send such long lines, ever!

Ditto.

>> mynetworks = /etc/postfix/mynetworks
> 
> This does not make sense.

This is a Debian feature.

>> qmgr_message_active_limit = 20
>> qmgr_message_recipient_limit = 20
> 
> 200k messages at the same time?  But also 3k recipient in the same mail?

Fine if memory permits.

>> smtpd_recipient_restrictions =  permit_mynetworks,
>> permit_sasl_authenticated, check_client_access
>> cidr:/etc/postfix/relay_allowedips, reject
> 
> What?

Looks fine to me, ideally the CIDR table is not too large.

>> transport_maps = 
>> cdb:/etc/postfix/bounce_transport,cdb:/etc/postfix/suppresslist,hash:/etc/postfix/transport,regexp:/etc/postfix/transport_regex,hash:/etc/postfix/emm_transport
> 
> What are you doing?

Some custom transport rules, regexp transports can be difficult to get right, 
but if the number of rules is small, and they simple enough this is fine.  That 
said two CDB tables in a row should really be combined into a single table, and 
having a "hash" table after that looks silly.  The qmgr(8) will do much less 
work if there's just one table replacing the first three.

-- 
Viktor.

Re: Mails stuck in queue until inflow stops

[Bastian Blank]

On Fri, Apr 20, 2018 at 06:42:03PM +0530, Ram wrote:
> Is there a contention on the queue manager when the inflow is too quick ?

Well, show some evidence.

> bounce_queue_lifetime = 5d

I thought this is a system that should move mails as fast as possible
outgoing.  Why would it ever handle bounces?

> broken_sasl_auth_clients = yes

Authentication?

> debug_peer_level = 2

Don't use that.

> inet_interfaces = 127.0.0.1

Only listen on localhost.  So you actually have the application and
postfix battling on ressources on this system?  Why?

> lmtp_destination_concurrency_limit = 30

And there you have lmtp.

> lmtp_line_length_limit = 990

You can't send such long lines, ever!

> mailbox_size_limit = 52783082

I though this system should send, not retrieve mails?

> mynetworks = /etc/postfix/mynetworks

This does not make sense.

> qmgr_message_active_limit = 20
> qmgr_message_recipient_limit = 20

200k messages at the same time?  But also 3k recipient in the same mail?

> smtpd_recipient_restrictions =  permit_mynetworks,
> permit_sasl_authenticated, check_client_access
> cidr:/etc/postfix/relay_allowedips, reject

What?

> transport_maps = 
> cdb:/etc/postfix/bounce_transport,cdb:/etc/postfix/suppresslist,hash:/etc/postfix/transport,regexp:/etc/postfix/transport_regex,hash:/etc/postfix/emm_transport

What are you doing?

Your story does not fit the configuration.  This looks like a generic
purpose relay with authentication, but you told something different.

If your application eats up all the memory, then you won't get any
useful message rate outgoing.

Bastian

-- 
It is a human characteristic to love little animals, especially if
they're attractive in some way.
-- McCoy, "The Trouble with Tribbles", stardate 4525.6

Re: integrating p0f with postfix

[Wietse Venema]

David Mehler:
> Hello,
> 
> I was hoping to avoid something so heavy weight, are there any other options?

As of Postfix 3.2, the policy delegation protocol will send the
server and client port and address. You can use this to query the
p0f cache.

http://www.postfix.org/SMTP_POLICY_README.html

This also has pointers to a simple policy daemon.

Wietse

Re: Mails stuck in queue until inflow stops

[Viktor Dukhovni]

> On Apr 20, 2018, at 9:12 AM, Ram  wrote:
> 
> I have a very busy postfix server that acts as a relay. It gets mails from an 
> application and then forwards the mails to the delivery servers on local LAN
> 
> The application can send mails at rate of  upto 600 mails per second
> Postfix has been configured to accept mails all that quickly, but the 
> delivery is very poor until inflow stops. Only around 20-50 mails per s
> Once the app completes the inflow, then the mails are cleared at a rate of 
> 1000 mails per second

If you can get the application to send to multiple IP addresses, you
could run multiple Postfix instances with multiple queues, each
handling a fraction of the load.  More queue manager processes
might get a larger fraction of the CPU pie.

If the local DNS does round-robin A records, and the application
calls getaddrinfo() for each connection, a multi-homed Postfix
hostname might suffice.

Make sure that your transport table is as fast as possible,
perhaps none at all, or a texthash file.  While the flood
is coming in determine whether the congestion is in the
"incoming" or "active" queue.  I expect the former, but
this needs to be confirmed before considering further
tuning.

-- 
-- 
Viktor.

Re: undisclosed-recipients

[Noel Jones]

On 4/20/2018 3:30 AM, Karel wrote:
> Hello,
> 
> is it legitimate to use "To: undisclosed-recipients", or is to only
> (mainly) used by spammers ?

It is legit. I don't see many spammers using this anymore, probably
because of the incorrect perception that it indicates spam.

It is no different from a "To: (not you)" header, such as this email.

> 
> Seems to me, if I get an email, I should also know who else was the
> email sent to.
> Its like having a conference call and you don't know who's participating.

No, it's like watching a television show and having everyone else
know you're watching and where you live; it protects YOUR privacy
and your email address.

True story: I once attended a conference that the organizer sent
notice emails with the 1000 or so attendees all listed in To: and
CC: lines.  I probably got 200 mails from people hitting reply-all
and then replying to the reply -- does the hotel have a pool, how's
your mother doing after her surgery, etc. ad nauseam.

> 
> If I wanted to block emails without any recipient, what would be the
> best way to do it ?

Use header_checks.  But I don't recommend it.

> 
> thanks,
> Karel
> 


  -- Noel Jones

Re: integrating p0f with postfix

[David Mehler]

Hello,

I was hoping to avoid something so heavy weight, are there any other options?

Thanks.
Dave.


On 4/20/18, Matus UHLAR - fantomas  wrote:
> On 19.04.18 22:25, David Mehler wrote:
>>Does anyone have p0f going with postfix? I'm wanting to add a header
>>for email connecting OS.
>
> I think amavis supports p0f, so any way of integrating amavis into postfix
> should allow this functionality (and many others).
> --
> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> There's a long-standing bug relating to the x86 architecture that
> allows you to install Windows.   -- Matthew D. Fuller
>

Re: user unknown in virtual mailbox table

[Alfredo De Luca]

Hi all. I had a run with postmap and these are the founding

so we have mydomain1.com which is the original domain.and mydomain2.com
which is the actual domanin of our company.
So when I do the following

- postmap -q arel...@mydomain1.comregexp:./domain_rewriting ldap:./
ldap-virtual-maps.cf
   areluca basically doesn't exist with my mydomain1.com so...I get a
message back with *user unknown*

- postmap -q arel...@mydomain2.comregexp:./domain_rewriting ldap:./
ldap-virtual-maps.cf
returns arel...@mydomain1.com..which DOESN\t exist. but cause it
find a result anyway I dont get any mail back saying *user unknown*

So it's something in the ldap that I need to add or trigger.

Maybe mailacceptinggeneralid will do the job accordingly to
*http://www.postfix.org/LDAP_README.html#config?
??*


Thanks









On Fri, Apr 20, 2018 at 4:03 PM, Viktor Dukhovni  wrote:

>
>
> > On Apr 20, 2018, at 8:03 AM, @lbutlr  wrote:
> >
> > The biggest issue between regex (POSIX) and PCRE is that POSIX regex is
> greedy. that is, it matches the longest possible left, while PCRE matches
> the shortest possible left.
>
> That's false (example uses a Bash in-line file):
>
>$ postmap -q aaa pcre:<(printf '%s\n' '/(a*)(a)/ $1:$2')
>aa:a
>
> however, PCRE does also provide non-greedy "*" and "+" variants:
>
>   $ postmap -q aaa pcre:<(printf '%s\n' '/(a+?)(a)/ $1:$2')
>   a:a
>
>   $ postmap -q aaa pcre:<(printf '%s\n' '/(a*?)(a)/ $1:$2')
>   :a
>
> --
> Viktor.
>
>


-- 
*Alfredo*

Re: Mails stuck in queue until inflow stops

[Durga Prasad Malyala]

Hi,
We achieved considerable improvement in delivery speed and thereby negligible 
queues by shifting the mail spool to a faster disk. 

Rgds/DP

Sent from my iPhone. Pls excuse brevity and typos if any. 

> On 20-Apr-2018, at 8:10 PM, Stephen Satchell  wrote:
> 
>> On 04/20/2018 06:44 AM, Wietse Venema wrote:
>> No, there is contention for the file system.
>> If you disabled in_flow_delay, turn it back on, please. This allows
>> the queue manager to push back, though it works only for clients
>> that make few parallel connections.
> 
> Looking at master.cf, there is the column "command + args".
> 
> Question:  would "nice -n 5 " work?
> 
> for example:
>  pickupunix  n   -   n   60  1   nice -5 pickup
>  smtp  inet  n   -   n   -   5   nice -5 smtpd
>  smtps inet  n   -   n   -   -   nice -5 smtpd
>-o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes
>  (submission, too?)
> 
> This means that, on disk contention, the other processes have an edge over 
> the above-listed processes.  As I recall, when you have multiple processes 
> contending for I/O on a device, the process with the higher working priority 
> gets selected.  For writes, though, most of the requests end up being 
> buffered and the writes actually performed by a daemon or triggered by 
> close(2) or fsync(2).
> 
> (All this goes out the window when one moves to solid-state disk.  That 
> assumes that the system running PostFix can keep up with the inflow; if not, 
> then my suggestion may still have merit even with SSD.)
> 
> In TUNING_README.html, you suggest running a DNS server on the local machine. 
>  Excellent advice.  When I configured an instance of Postfix as a smart relay 
> for a double-handful of CPanel/PLESK servers, I found that a local DNS server 
> with a large DNS cache had a profound positive effect on clearing out the 
> mail queue in a timely manner.  The computer running this PostFix instance 
> had eight gigabytes of DRAM, which also let the Linux file system's cache 
> reduce the accesses to the disk drive.
> 
> For high-traffic endpoints (aol.com, gmail.com, yahoo.com, ) I also had 
> dedicated senders defined so that I could throttle mail to those endpoints to 
> prevent triggering anti-spam controls, with the rest going out the regular 
> way.
> 

Re: Mails stuck in queue until inflow stops

[Stephen Satchell]

On 04/20/2018 06:44 AM, Wietse Venema wrote:

No, there is contention for the file system.

If you disabled in_flow_delay, turn it back on, please. This allows
the queue manager to push back, though it works only for clients
that make few parallel connections.


Looking at master.cf, there is the column "command + args".

Question:  would "nice -n 5 " work?

for example:
  pickupunix  n   -   n   60  1   nice -5 pickup
  smtp  inet  n   -   n   -   5   nice -5 smtpd
  smtps inet  n   -   n   -   -   nice -5 smtpd
-o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes
  (submission, too?)

This means that, on disk contention, the other processes have an edge 
over the above-listed processes.  As I recall, when you have multiple 
processes contending for I/O on a device, the process with the higher 
working priority gets selected.  For writes, though, most of the 
requests end up being buffered and the writes actually performed by a 
daemon or triggered by close(2) or fsync(2).


(All this goes out the window when one moves to solid-state disk.  That 
assumes that the system running PostFix can keep up with the inflow; if 
not, then my suggestion may still have merit even with SSD.)


In TUNING_README.html, you suggest running a DNS server on the local 
machine.  Excellent advice.  When I configured an instance of Postfix as 
a smart relay for a double-handful of CPanel/PLESK servers, I found that 
a local DNS server with a large DNS cache had a profound positive effect 
on clearing out the mail queue in a timely manner.  The computer running 
this PostFix instance had eight gigabytes of DRAM, which also let the 
Linux file system's cache reduce the accesses to the disk drive.


For high-traffic endpoints (aol.com, gmail.com, yahoo.com, ) I also 
had dedicated senders defined so that I could throttle mail to those 
endpoints to prevent triggering anti-spam controls, with the rest going 
out the regular way.

Re: Mails stuck in queue until inflow stops

[Wietse Venema]

Ram:
> 
> 
> On 04/20/2018 07:14 PM, Wietse Venema wrote:
> > Ram:
> >> I have a very busy postfix server that acts as a relay. It gets mails
> >> from an application and then forwards the mails to the delivery servers
> >> on local LAN
> >>
> >> The application can send mails at rate of? upto 600 mails per second
> >> Postfix has been configured to accept mails all that quickly, but the
> >> delivery is very poor until inflow stops. Only around 20-50 mails per s
> >> Once the app completes the inflow, then the mails are cleared at a rate
> >> of 1000 mails per second
> >>
> >> Why ?
> >>
> >> Is there a contention on the queue manager when the inflow is too quick ?
> > No, there is contention for the file system.
> >
> > If you disabled in_flow_delay, turn it back on, please. This allows
> > the queue manager to push back, though it works only for clients
> > that make few parallel connections.
> >
> > Otherwise, you need a faster disk. SSDs have become quite affordable,
> > even the 'enterprise' ones that have some extra capacitors to prevent
> > data corruption after power failure.
> I am using spool dir on /dev/shm
> 
> in flow delay .. slows down smtp connections which the application can 
> not handle
> That is why I have disabled

If you can't use the Postfix safety mechanism, then I can't help you.

Wietse

Re: Mails stuck in queue until inflow stops

[Ram]

On 04/20/2018 07:14 PM, Wietse Venema wrote:

Ram:

I have a very busy postfix server that acts as a relay. It gets mails
from an application and then forwards the mails to the delivery servers
on local LAN

The application can send mails at rate of? upto 600 mails per second
Postfix has been configured to accept mails all that quickly, but the
delivery is very poor until inflow stops. Only around 20-50 mails per s
Once the app completes the inflow, then the mails are cleared at a rate
of 1000 mails per second

Why ?

Is there a contention on the queue manager when the inflow is too quick ?

No, there is contention for the file system.

If you disabled in_flow_delay, turn it back on, please. This allows
the queue manager to push back, though it works only for clients
that make few parallel connections.

Otherwise, you need a faster disk. SSDs have become quite affordable,
even the 'enterprise' ones that have some extra capacitors to prevent
data corruption after power failure.

I am using spool dir on /dev/shm

in flow delay .. slows down smtp connections which the application can 
not handle

That is why I have disabled







Wietse




https://netcore.in/20-years-journey/?utm_source=email-disclaimer_medium=email_campaign=netcore-turns-20

Re: user unknown in virtual mailbox table

[Viktor Dukhovni]

> On Apr 20, 2018, at 8:03 AM, @lbutlr  wrote:
> 
> The biggest issue between regex (POSIX) and PCRE is that POSIX regex is 
> greedy. that is, it matches the longest possible left, while PCRE matches the 
> shortest possible left.

That's false (example uses a Bash in-line file):

   $ postmap -q aaa pcre:<(printf '%s\n' '/(a*)(a)/ $1:$2')
   aa:a

however, PCRE does also provide non-greedy "*" and "+" variants:

  $ postmap -q aaa pcre:<(printf '%s\n' '/(a+?)(a)/ $1:$2')
  a:a

  $ postmap -q aaa pcre:<(printf '%s\n' '/(a*?)(a)/ $1:$2')
  :a

-- 
Viktor.

Re: Mails stuck in queue until inflow stops

[Wietse Venema]

Ram:
> I have a very busy postfix server that acts as a relay. It gets mails 
> from an application and then forwards the mails to the delivery servers 
> on local LAN
> 
> The application can send mails at rate of? upto 600 mails per second
> Postfix has been configured to accept mails all that quickly, but the 
> delivery is very poor until inflow stops. Only around 20-50 mails per s
> Once the app completes the inflow, then the mails are cleared at a rate 
> of 1000 mails per second
> 
> Why ?
> 
> Is there a contention on the queue manager when the inflow is too quick ?

No, there is contention for the file system.

If you disabled in_flow_delay, turn it back on, please. This allows
the queue manager to push back, though it works only for clients
that make few parallel connections.

Otherwise, you need a faster disk. SSDs have become quite affordable,
even the 'enterprise' ones that have some extra capacitors to prevent
data corruption after power failure.

Wietse

Mails stuck in queue until inflow stops

[Ram]

I have a very busy postfix server that acts as a relay. It gets mails 
from an application and then forwards the mails to the delivery servers 
on local LAN


The application can send mails at rate of  upto 600 mails per second
Postfix has been configured to accept mails all that quickly, but the 
delivery is very poor until inflow stops. Only around 20-50 mails per s
Once the app completes the inflow, then the mails are cleared at a rate 
of 1000 mails per second


Why ?

Is there a contention on the queue manager when the inflow is too quick ?



Postfix version 3.0.1 on Centos 7.2


postconf -n

alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
always_add_missing_headers = yes
bounce_queue_lifetime = 5d
bounce_template_file = /etc/postfix/bounce.cf.default
broken_sasl_auth_clients = yes
command_directory = /usr/sbin
compatibility_level = 2
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
default_destination_concurrency_limit = 500
default_process_limit = 500
disable_mime_input_processing = yes
disable_vrfy_command = yes
hash_queue_depth = 1
hash_queue_names = deferred, defer, hold
header_checks = regexp:/etc/postfix/header_checks
html_directory = no
in_flow_delay = 0s
inet_interfaces = 127.0.0.1
inet_protocols = all
lmtp_destination_concurrency_limit = 30
lmtp_line_length_limit = 990
mail_owner = postfix
mailbox_size_limit = 52783082
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
maximal_queue_lifetime = 5d
message_size_limit = 52783082
meta_directory = /etc/postfix
minimal_backoff_time = 30s
mydestination = XXX
myhostname = XXX
mynetworks = /etc/postfix/mynetworks
newaliases_path = /usr/bin/newaliases.postfix
qmgr_message_active_limit = 20
qmgr_message_recipient_limit = 20
queue_directory = /dev/shm/postfix
readme_directory = /usr/share/doc/postfix-3.0.1/README_FILES
relayhost = [X]
sample_directory = /usr/share/doc/postfix-3.0.1/samples
setgid_group = postdrop
shlib_directory = /usr/lib64/postfix
smtp_connection_cache_on_demand = yes
smtp_connection_cache_time_limit = 300s
smtp_line_length_limit = 990
smtpd_client_connection_count_limit = 0
smtpd_client_connection_rate_limit = 0
smtpd_recipient_limit = 3000
smtpd_recipient_restrictions =  permit_mynetworks, 
permit_sasl_authenticated, check_client_access 
cidr:/etc/postfix/relay_allowedips, reject

smtpd_restriction_classes = check_env_from
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = yes
smtpd_sasl_local_domain =
smtpd_sasl_security_options = noanonymous
smtpd_sender_login_maps = hash:/etc/postfix/smtpd_sender_login_maps
smtpd_sender_restrictions = permit_mynetworks, check_client_access 
cidr:/etc/postfix/permit_sender_ip, reject_sender_login_mismatch, permit
transport_maps = 
cdb:/etc/postfix/bounce_transport,cdb:/etc/postfix/suppresslist,hash:/etc/postfix/transport,regexp:/etc/postfix/transport_regex,hash:/etc/postfix/emm_transport

unknown_hostname_reject_code = 550
unknown_local_recipient_reject_code = 550
virtual_alias_maps = hash:/etc/postfix/vmap
virtual_mailbox_base = /var/spool/mail


https://netcore.in/20-years-journey/?utm_source=email-disclaimer_medium=email_campaign=netcore-turns-20

Re: user unknown in virtual mailbox table

[@lbutlr]

On 2018-04-20 (05:07 MDT), Wietse Venema  wrote:
> 
> Also, be aware that many examples 'on the web' use PCRE which
> is subtly different from regexp.

The biggest issue between regex (POSIX) and PCRE is that POSIX regex is greedy. 
that is, it matches the longest possible left, while PCRE matches the shortest 
possible left.

The O'Reilly Media book on regex covers this in great detail



-- 
It was intended that when Newspeak had been adopted once and for all and
Oldspeak forgotten, a heretical thought...should be literally
unthinkable, at least so far as thought is dependent on words.

Re: integrating p0f with postfix

[Matus UHLAR - fantomas]

On 19.04.18 22:25, David Mehler wrote:

Does anyone have p0f going with postfix? I'm wanting to add a header
for email connecting OS.


I think amavis supports p0f, so any way of integrating amavis into postfix
should allow this functionality (and many others).
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
There's a long-standing bug relating to the x86 architecture that
allows you to install Windows.   -- Matthew D. Fuller

Re: user unknown in virtual mailbox table

[Wietse Venema]

Alfredo De Luca:
> Hi all. Any clue/suggestions?
> > virtual_mailbox_domains = $config_directory/vdomains.txt
> > virtual_mailbox_maps = regexp:$config_directory/domain_rewriting

Test your regexp table like this:

$ postmap -q u...@example.com regexp:$config_directory/domain_rewriting

Or better, experiment with a file that isn't used by Postfix.
Also, be aware that many examples 'on the web' use PCRE which
is subtly different from regexp.

Wietse

Re: user unknown in virtual mailbox table

[Alfredo De Luca]

Hi all. Any clue/suggestions?

Cheers


On Thu, Apr 19, 2018 at 2:51 PM, Alfredo De Luca 
wrote:

> Hi all. Here is my postfix config.of course with domains and ip
> changed.
>
> I had a look also on the ldap section and given what are the
> reccommandation here (http://www.postfix.org/LDAP_README.html#config) it
> seems to be that ldap queries when we received an unknow user with a
> different domain from our main we don\t get an email back.
> Any thoughts?
>
> POSTFINGER output
> 
> --System Parameters--
> mail_version = 2.10.1
> hostname = dovecot
> uname = Linux dovecot 3.10.0-693.11.6.el7.x86_64 #1 SMP Thu Jan 4 01:06:37
> UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
>
> --Packaging information--
> looks like this postfix comes from RPM package: postfix-2.10.1-6.el7.x86_64
>
> --main.cf non-default parameters--
> alias_maps = hash:/etc/aliases
> debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd
> $daemon_directory/$process_name $process_id & sleep 5
> maildrop_destination_recipient_limit = 1
> mailq_path = /usr/bin/mailq.postfix
> manpage_directory = /usr/share/man
> message_size_limit = 26214400
> mydomain = mydomain1.com
> myhostname = smtp.mydomain1.com
> mynetworks = 127.0.0.0/8 10.10.10.251/32 [::1]/128 [fe80::]/64
> 10.20.20.20/32<-- IP CHANGED
> myorigin = $mydomain
> newaliases_path = /usr/bin/newaliases.postfix
> readme_directory = /usr/share/doc/postfix-2.10.1/README_FILES
> relayhost = [mx2.mydomain1.com]
> remote_header_rewrite_domain = $mydomain
> sample_directory = /usr/share/doc/postfix-2.10.1/samples
> sendmail_path = /usr/sbin/sendmail.postfix
> smtpd_banner = mail.mydomain1.com
> smtpd_client_connection_count_limit = 10
> smtpd_client_connection_rate_limit = 60
> smtpd_recipient_restrictions = permit_mynetworks,reject_non_
> fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_
> authenticated,reject_unauth_destination,check_policy_service
> inet:localhost:12340
> smtpd_sasl_auth_enable = yes
> smtpd_sasl_local_domain = $myhostname
> smtpd_sasl_path = private/auth
> smtpd_sasl_type = dovecot
> smtpd_tls_CAfile = $config_directory/ssl/DigiCertCA.crt
> smtpd_tls_CApath = $config_directory/ssl
> smtpd_tls_cert_file = $config_directory/ssl/star_mydomain1.com.crt
> smtpd_tls_key_file = $config_directory/ssl/star_mydomain1.com.key
> smtpd_tls_loglevel = 1
> smtpd_tls_received_header = yes
> smtpd_use_tls = yes
> smtp_fallback_relay = [mx.mydomain1.com]
> virtual_alias_domains = mydomain1.com pro-mydomain3.com pro-mydomain3.it
> virtual_alias_maps = regexp:$config_directory/domain_rewriting
> ldap:$config_directory/ldap-virtual-aliases.cf
> virtual_mailbox_domains = $config_directory/vdomains.txt
> virtual_mailbox_maps = regexp:$config_directory/domain_rewriting
> ldap:$config_directory/ldap-virtual-maps.cf
> virtual_transport = maildrop
>
> --master.cf--
> smtp  inet  n   -   n   -   -   smtpd
> 465inet  n   -   n   -   -   smtpd
>   -o syslog_name=postfix/smtps
>   -o smtpd_tls_wrappermode=yes
>   -o smtpd_sasl_auth_enable=yes
> pickupunix  n   -   n   60  1   pickup
> cleanup   unix  n   -   n   -   0   cleanup
> qmgr  unix  n   -   n   300 1   qmgr
> tlsmgrunix  -   -   n   1000?   1   tlsmgr
> rewrite   unix  -   -   n   -   -   trivial-rewrite
> bounceunix  -   -   n   -   0   bounce
> defer unix  -   -   n   -   0   bounce
> trace unix  -   -   n   -   0   bounce
> verifyunix  -   -   n   -   1   verify
> flush unix  n   -   n   1000?   0   flush
> proxymap  unix  -   -   n   -   -   proxymap
> proxywrite unix -   -   n   -   1   proxymap
> smtp  unix  -   -   n   -   -   smtp
> relay unix  -   -   n   -   -   smtp
> showq unix  n   -   n   -   -   showq
> error unix  -   -   n   -   -   error
> retry unix  -   -   n   -   -   error
> discard   unix  -   -   n   -   -   discard
> local unix  -   n   n   -   -   local
> virtual   unix  -   n   n   -   -   virtual
> lmtp  unix  -   -   n   -   -   lmtp
> anvil unix  -   -   n   -   1   anvil
> scacheunix  -   -   n   -   1   scache
> maildrop  unix  - n n - - pipe  flags=ODRhu user=vmail
> argv=/usr/local/bin/maildrop /etc/maildroprc -d ${user}@${domain}
> ${extension} ${recipient} ${user} ${nexthop} ${sender} ${mailbox}
> ///
>
>
>
>
>
>
> On Wed, Apr 18, 2018 at 5:11 PM, Alfredo De Luca  > wrote:
>
>> Thanks 

Re: undisclosed-recipients

[Doug Hardie]

> On 20 April 2018, at 01:30, Karel  wrote:
> 
> Hello,
> 
> is it legitimate to use "To: undisclosed-recipients", or is to only
> (mainly) used by spammers ?
> 
> Seems to me, if I get an email, I should also know who else was the
> email sent to.
> Its like having a conference call and you don't know who's participating.
> 
> If I wanted to block emails without any recipient, what would be the
> best way to do it ?

One good use of that is to prevent spam.  If you are sending an email to a 
number of people (announcements etc.) then the normal response is for people to 
reply-all.  That spams a lot of people who don't want to see those, or don't 
want their email address published.  By using BCCs, you avoid both issues.

-- Doug

undisclosed-recipients

[Karel]

Hello,

is it legitimate to use "To: undisclosed-recipients", or is to only
(mainly) used by spammers ?

Seems to me, if I get an email, I should also know who else was the
email sent to.
Its like having a conference call and you don't know who's participating.

If I wanted to block emails without any recipient, what would be the
best way to do it ?

thanks,
Karel

Re: prevent NDRs for sieve-forwarded emails

[mj]

Thanks for your reply.

On 04/19/2018 02:51 PM, Matus UHLAR - fantomas wrote:


rewriting sender of the forwarded mail in the SRS
(https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme) way and delivering
all the mail to rewritten sender to someone who is able to fix or remove
such forwarding.

I would consider delivering mail back to the mailbox of u...@ourdomain.com
or maybe postmas...@ourdomain.com



So, if I understand correctly, I would set something like:

sieve_redirect_envelope_from = ""
(dovecot.conf)

Thanks again!

MJ