Re: Upgraded to 3.4 today. All logging has Stopped?

[Bill Cole]

On 9 Jan 2019, at 22:01, Larry Stone wrote:

Has anyone managed to do this? I’d rather not have to compile on 
old Mac and transfer.



Not as far as any of us know. It’s been discussed here before and no 
solution has been found.



As I suggested back when this thread was young, this command will put 
all of Postfix's log messages into /var/log/mail.log:


   log stream --info --predicate 'senderImagePath CONTAINS "postfix"' 
--style syslog >> /var/log/mail.log


It's trivial to create a launchd .plist file to run that at boot time 
and let you start/stop it with launchctl.


That may be a bit heavy for a busy server because it pipes all of the 
log messages through a userspace process, but it will do what you want.


It remains true that macOS has been unfit for most server work since 
Sierra as a direct result of conscious choices by Apple to make it so. 
Even if the ugly hack above works for you, it's just a battle tactic in 
a war that cannot be won. Apple has clearly demonstrated that they want 
macOS to be not be used for server duty.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole

The AUTH parameter on MAIL commands

[Jacky]

Hi,

Want to set up a dovecot submission server and which will act as a proxy 
in front of a postfix server.


When users deliver email via the dovecot submission proxy, it will 
authenticate users and deliver mail to postfix submission service.


The dovecot submission service support AUTH parameter with the MAIL 
command (https://tools.ietf.org/html/rfc4954#section-5) and will send 
the parameter during the connection of the postfix submission server.


But it seems that postfix ignored the parameter and treats all users as 
unauthenticated.


Does anyone know how to configure postfix to accept the AUTH parameter?

Found doc from google search, but it is for exim only 
(https://www.exim.org/exim-html-current/doc/html/spec_html/ch-smtp_authentication.html#SECTauthparamail)


Regards,

Jacky

Re: Upgraded to 3.4 today. All logging has Stopped?

[James Brown]

> On 10 Jan 2019, at 2:01 pm, Larry Stone  > wrote:
> 
> On Jan 9, 2019, at 19:01, James Brown  > wrote:
>> 
>> Thanks Viktor. It would be great if Postfix would log to disk on newer 
>> versions of macOS X like it did before. My Mojave test mail server has 
>> Dovecot logging to /var/log/mail.log but Postfix doesn’t.
> 
> Is this a recent build of Dovecot or was it built on an older version of 
> MacOS before the logging changes? If the former, ask on the Dovecot list how 
> they did it. If the latter, it’s a meaningless data point until Dovecot is 
> rebuilt on a newer version of MacOS.
> 
>> Has anyone managed to do this? I’d rather not have to compile on old Mac and 
>> transfer. 
> 
> Not as far as any of us know. It’s been discussed here before and no solution 
> has been found.
> 
> — Larry Stone
>   lston...@stonejongleux.com 

Hi Larry. It’s a recent build of Dovecot, compiled on Mojave. Steps to compile 
were:

export CPPFLAGS="-I/usr/local/opt/mysql@5.7/include/mysql 
-I/usr/local/Cellar/openssl@1.1/1.1.1/include"
export LDFLAGS="-L/usr/local/opt/mysql@5.7/lib -L/usr/local/opt/openssl@1.1/lib"

./configure --with-mysql --with-ssl=openssl

make

sudo make install

The setting file for logging, “etc/dovecot/conf.d/10-logging.conf” does have 
this:

##
## Log destination.
##

# Log file to use for error messages. "syslog" logs to syslog,
# /dev/stderr logs to stderr.
#log_path = syslog
log_path = /var/log/mail.log

So I’ve had to change this so that it writes directly to the file, and not to 
syslog.

James.

Re: Upgraded to 3.4 today. All logging has Stopped?

[Larry Stone]

On Jan 9, 2019, at 19:01, James Brown  wrote:
> 
> Thanks Viktor. It would be great if Postfix would log to disk on newer 
> versions of macOS X like it did before. My Mojave test mail server has 
> Dovecot logging to /var/log/mail.log but Postfix doesn’t.

Is this a recent build of Dovecot or was it built on an older version of MacOS 
before the logging changes? If the former, ask on the Dovecot list how they did 
it. If the latter, it’s a meaningless data point until Dovecot is rebuilt on a 
newer version of MacOS.

> Has anyone managed to do this? I’d rather not have to compile on old Mac and 
> transfer. 

Not as far as any of us know. It’s been discussed here before and no solution 
has been found.

— Larry Stone
   lston...@stonejongleux.com

Re: Upgraded to 3.4 today. All logging has Stopped?

[Wietse Venema]

James Brown:
> 
> > On 30 Nov 2018, at 4:07 am, Viktor Dukhovni  
> > wrote:
> > 
> > 
> >> On Nov 29, 2018, at 9:15 AM, Robert Chalmers  wrote:
> >> 
> >> I upgraded to and installed 3.4 today. It appears to be running, but all 
> >> logging has stopped. The last entry in the log file was at the exact time 
> >> I started the new version.
> > 
> > Apple changed the syslog API to use the os_log(3) interface, but apparently
> > if you compile on a sufficiently old MacOS/X system, you get linked against
> > the traditional syslog API.
> > 
> > So some people do that, but the os_log(3) stuff is configurable, so you
> > should be able to configure the logs to be saved.  The in-memory logs
> > can be queried with log(1).  For example:
> > 
> >  # postfix reload
> >  postfix/postfix-script: refreshing the Postfix mail system
> > 
> >  # log show --info --style syslog --last 2m --predicate 'eventType == 
> > logEvent'  | grep libpostfix-util
> >  2018-11-29 12:02:26.765215-0500  localhost postlog[59348]: 
> > (libpostfix-util.dylib) refreshing the Postfix mail system
> >  2018-11-29 12:02:26.776443-0500  localhost master[33680]: 
> > (libpostfix-util.dylib) reload -- version 3.4-20181125, configuration 
> > /var/tmp/postfix/etc
> > 
> > Someone should figure out how create an asl.conf(5) configuration
> > that causes appropriate Postfix logs to land on disk, and share
> > the recipe.
> > 
> > I should note that the new API appears to strip everything up to
> > the last "/" in the log name, so we're losing the "postfix/"
> > prefixes.  Might need to change the code (on MacOS) to send
> > 
> > postfix:master
> > 
> > rather than
> > 
> > postfix/master
> > 
> > -- 
> > Viktor
> 
> Thanks Viktor. It would be great if Postfix would log to disk on newer 
> versions of macOS X like it did before. My Mojave test mail server has 
> Dovecot logging to /var/log/mail.log but Postfix doesn?t.
> 
> Has anyone managed to do this? I?d rather not have to compile on old Mac and 
> transfer. 
> 
> Really need logging to disk for fail2ban to work with Postfix.

For that, you will have to build Postfix on an older MacOS. Postfix
does not decide where the logging is sent. Postfix logs to the
syslog port as it has been doing for 20+ years.

Wietse

Wietse

Re: Upgraded to 3.4 today. All logging has Stopped?

[James Brown]

> On 30 Nov 2018, at 4:07 am, Viktor Dukhovni  
> wrote:
> 
> 
>> On Nov 29, 2018, at 9:15 AM, Robert Chalmers  wrote:
>> 
>> I upgraded to and installed 3.4 today. It appears to be running, but all 
>> logging has stopped. The last entry in the log file was at the exact time I 
>> started the new version.
> 
> Apple changed the syslog API to use the os_log(3) interface, but apparently
> if you compile on a sufficiently old MacOS/X system, you get linked against
> the traditional syslog API.
> 
> So some people do that, but the os_log(3) stuff is configurable, so you
> should be able to configure the logs to be saved.  The in-memory logs
> can be queried with log(1).  For example:
> 
>  # postfix reload
>  postfix/postfix-script: refreshing the Postfix mail system
> 
>  # log show --info --style syslog --last 2m --predicate 'eventType == 
> logEvent'  | grep libpostfix-util
>  2018-11-29 12:02:26.765215-0500  localhost postlog[59348]: 
> (libpostfix-util.dylib) refreshing the Postfix mail system
>  2018-11-29 12:02:26.776443-0500  localhost master[33680]: 
> (libpostfix-util.dylib) reload -- version 3.4-20181125, configuration 
> /var/tmp/postfix/etc
> 
> Someone should figure out how create an asl.conf(5) configuration
> that causes appropriate Postfix logs to land on disk, and share
> the recipe.
> 
> I should note that the new API appears to strip everything up to
> the last "/" in the log name, so we're losing the "postfix/"
> prefixes.  Might need to change the code (on MacOS) to send
> 
>   postfix:master
> 
> rather than
> 
>   postfix/master
> 
> -- 
>   Viktor

Thanks Viktor. It would be great if Postfix would log to disk on newer versions 
of macOS X like it did before. My Mojave test mail server has Dovecot logging 
to /var/log/mail.log but Postfix doesn’t.

Has anyone managed to do this? I’d rather not have to compile on old Mac and 
transfer. 

Really need logging to disk for fail2ban to work with Postfix.

James.

Re: It is possible for Postfix logging to bypass journald?

[Robert L Mathews]

On 1/9/19 4:05 PM, Curtis wrote:
> We recently switched our Postfix mail servers to Ubuntu Server 18, which
> uses journald for logging. Since we have monitoring systems that parse
> /var/log/maillog, we enabled rsyslog with imuxsock so we still can parse
> the log like we did before journald.  But, it's unreliable.
> 
> Our monitoring systems are reporting failed deliveries of messages
> because of missing log lines in /var/log/maillog.

We had this problem. It was fixed by putting this in
/etc/systemd/journald.conf:

# allow for busy mail logs; allows 1000 per second
RateLimitInterval=5s
RateLimitBurst=5000

And/or by putting this into /etc/rsyslog.conf:

$SystemLogRateLimitInterval 0

(The latter is supposedly no longer necessary, but it used to be, and
does not appear to be harmful.)

-- 
Robert L Mathews, Tiger Technologies, http://www.tigertech.net/

Re: It is possible for Postfix logging to bypass journald?

[Wietse Venema]

Curtis:
> We recently switched our Postfix mail servers to Ubuntu Server 18, which 
> uses journald for logging. Since we have monitoring systems that parse 
> /var/log/maillog, we enabled rsyslog with imuxsock so we still can parse 
> the log like we did before journald.  But, it's unreliable.

I recall that system-effing-d has a rare-limiting feature that very
helpfully drops Postfix logging.

Here's one search result with suggestions for systemd.
https://www.rootusers.com/how-to-change-log-rate-limiting-in-linux/

Another search result: systemd and rsyslog both have rate limits.
https://support.asperasoft.com/hc/en-us/articles/216128628-How-to-disable-rsyslog-rate-limiting

It is time to update the Postfix page on LINUX logging brain damage.

Wietse

It is possible for Postfix logging to bypass journald?

[Curtis]

We recently switched our Postfix mail servers to Ubuntu Server 18, which 
uses journald for logging. Since we have monitoring systems that parse 
/var/log/maillog, we enabled rsyslog with imuxsock so we still can parse 
the log like we did before journald.  But, it's unreliable.


Our monitoring systems are reporting failed deliveries of messages 
because of missing log lines in /var/log/maillog.  When using journalctl 
to query the journal, the missing lines can be found, but these queries 
are too CPU intensive.


We also see that journald is occasionally logging messages such as this:

Jan 08 20:55:16 host123 systemd-journald[11136]: Forwarding to syslog 
missed 2 messages.


Since this message doesn't provide any information as to why the 
messages were missed, I have to wonder if it's related to this warning 
message on the rsyslog site:


"Note: It must be noted, however, that the journal tends to drop 
messages when it becomes busy instead of forwarding them to the system 
log socket. This is because the journal uses an async log socket 
interface for forwarding instead of the traditional synchronous one."


See: 
https://www.rsyslog.com/doc/v8-stable/configuration/modules/imuxsock.html#imuxsock-systemd-details-label


I'm aware we could switch to using imjournal, which might solve the 
issue since it reads the journal directly (which does seem to contain 
the missing messages), but I have to imagine that it would come at a 
very high CPU cost.


See: 
https://www.rsyslog.com/doc/v8-stable/configuration/modules/imjournal.html


So, I'm trying to figure out if it would be possible to get Postfix to 
use an alternate logging mechanism that would completely bypass journald 
so that we can have reliable loggging in a manner that is less CPU 
intensive than journald/imjournal.


Ideas?

Thanks,

Curtis

Re: mailer-daemon bounce notifications with original message in clear text?

[Jim Rice]

As a followup, we found a workaround...

postconf -e bounce_size_limit=1
zmcontrol restart

(Yes, this is Zimbra.)

This had the effect of including the bounce notification and headers,
but without the original email content (and no .eml attachment).



--
Sent from: http://postfix.1071664.n5.nabble.com/Postfix-Users-f2.html