Re: Postfix "fatal: daemon initialization failure"
> If you would file a bug in the Debian BTS with the details, I would > appreciate it. I'd like to see what we can do in Debian to improve the > situation. > > Thanks, > > Scott K I'll try to remember to do so. The Postfix work I'm doing is a small part of a bigger migration project that'll be taking up most of my waking hours for at least another week. Once things have stabilised again I'll do my best to drop by Debian BTS.
Re: Postfix "fatal: daemon initialization failure"
On January 26, 2022 4:57:09 PM UTC, Laura Smith wrote: >Good news, I found the cause of of the problem. > >I was using interface aliases to permit different postfix instances. > >However these were configured in the traditional/legacy manner using >/etc/network/interfaces. > >It seems that although the Postfix systemd unit employs the >"network-online.target" dependency, this dependency does not recognise the >legacy manner of configuring aliases. > >Therefore Postifx was crashing becuase it was unable to bind to the IPs that >had not yet been bound to the interfaces. > >Once I re-configured networking in the modern style using systemd-networkd >(/etc/systemd/network/if.network) Postfix was happy because the >"network-online.target" dependency was now operating correctly. If you would file a bug in the Debian BTS with the details, I would appreciate it. I'd like to see what we can do in Debian to improve the situation. Thanks, Scott K
Re: Postfix "fatal: daemon initialization failure"
Good news, I found the cause of of the problem. I was using interface aliases to permit different postfix instances. However these were configured in the traditional/legacy manner using /etc/network/interfaces. It seems that although the Postfix systemd unit employs the "network-online.target" dependency, this dependency does not recognise the legacy manner of configuring aliases. Therefore Postifx was crashing becuase it was unable to bind to the IPs that had not yet been bound to the interfaces. Once I re-configured networking in the modern style using systemd-networkd (/etc/systemd/network/if.network) Postfix was happy because the "network-online.target" dependency was now operating correctly.
Re: Postfix "fatal: daemon initialization failure"
Viktor Dukhovni: > On Wed, Jan 26, 2022 at 11:22:53AM -0500, Wietse Venema wrote: > > > Sorry, master -w" DOES fork (it's re-implemented with clone() on LINUX). > > > >-w Wait in a dummy foreground process, while the real master daemon > > initializes in a background process. The dummy foreground > > process returns a zero exit status only if the master daemon > > initialization is successful, and if it completes in a reason- > > able amount of time. > > Indeed you're right, sorry about the disparaging remark... > > > I'll leave it to you to resolve this question. > > :-(. > > The OP can start by posting "postconf -nf" and "postconf -Mf" output, > as well as any other maillog entries just above any of: > > postfix-authrelay/master[1179]: fatal: daemon initialization failure > postfix-inetgen/master[1195]: fatal: daemon initialization failure > postfix-authrelay/postfix-script[1404]: fatal: mail system startup failed > postfix-inetgen/postfix-script[1406]: fatal: mail system startup failed > > As well as try to find relevant information in the trace. As for > "strace" complaining about "decode_nlattr()", I believe that's a > Linux-issue in strace itself (perhaps the kernel and userland are not > entirely matched up), and so ends up on stderr even with "strace -o > file". I'd like to add one thing: unfortunately the master babysitter process currently does not distinguish between: - The background daemon took too long to report that initialization is complete (the babysitter could log an ETIMEOUT error). - The background daetmon (was) terminated without reporting that initialization is complete (the babysitter could log the termination status). - The background daemon reported completion, and closed the channel before the babysitter had a chance to read the channel. Postfix has code in many places to wait-before-close, but the code in this particular place is relatively new and is less defensive. Every now and then someone improves kernel code and thinks that it is a good idea to make close-after-write destructive. All of the above can be found out from strace output. Wietse
Re: Postfix "fatal: daemon initialization failure"
On Wed, Jan 26, 2022 at 11:22:53AM -0500, Wietse Venema wrote: > Sorry, master -w" DOES fork (it's re-implemented with clone() on LINUX). > >-w Wait in a dummy foreground process, while the real master daemon > initializes in a background process. The dummy foreground > process returns a zero exit status only if the master daemon > initialization is successful, and if it completes in a reason- > able amount of time. Indeed you're right, sorry about the disparaging remark... > I'll leave it to you to resolve this question. :-(. The OP can start by posting "postconf -nf" and "postconf -Mf" output, as well as any other maillog entries just above any of: postfix-authrelay/master[1179]: fatal: daemon initialization failure postfix-inetgen/master[1195]: fatal: daemon initialization failure postfix-authrelay/postfix-script[1404]: fatal: mail system startup failed postfix-inetgen/postfix-script[1406]: fatal: mail system startup failed As well as try to find relevant information in the trace. As for "strace" complaining about "decode_nlattr()", I believe that's a Linux-issue in strace itself (perhaps the kernel and userland are not entirely matched up), and so ends up on stderr even with "strace -o file". -- Viktor.
Re: Postfix "fatal: daemon initialization failure"
Viktor Dukhovni: > > On 26 Jan 2022, at 10:43 am, Laura Smith > > wrote: > > > > On stdout I get a few lines of : > > strace: decode_nlattr: [xlat 0x, dflt "AF_???", decoders 0x] size > > is zero (going to pass nla_type as decoder argument), but opaque data > > (0x) is not - will be ignored > > > > "fgrep fork output-file" doesn't produce any results ? > > Typo on Wietse's part I think. "master -w" does not fork, > it runs in the foreground. You need to see what happens > shortly before it bails out, above any resource cleanup > as it is exiting. Sorry, master -w" DOES fork (it's re-implemented with clone() on LINUX). -w Wait in a dummy foreground process, while the real master daemon initializes in a background process. The dummy foreground process returns a zero exit status only if the master daemon initialization is successful, and if it completes in a reason- able amount of time. I'll leave it to you to resolve this question. Wietse
Re: Postfix "fatal: daemon initialization failure"
Laura Smith: > > > To debug, run as root: > > > > strace -f -o output-file /usr/libexec/postfix/master -w > > > > and look for the process that is created after fork(). > > On stdout I get a few lines of : > strace: decode_nlattr: [xlat 0x, dflt "AF_???", decoders 0x] size is > zero (going to pass nla_type as decoder argument), but opaque data (0x) > is not - will be ignored strace reports trace info (and warnings) on STDERR not STDOUT. I asked you to capture THAT output in a file, with "-o output-file". > "fgrep fork output-file" doesn't produce any results ? LINUX has reimplemented fork() as something that calls clone(). In the strace output, look for line point where it logs a new process id. Something like this: clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD strace: Process 275086 attached , child_tidptr=0x7fde036a9850) = 275086 >From here on, strace output is prefixed with the process ID. Here, the process 275086 is the daemon initializes in the background. The other process is waits for the background daemon to report that it is ready. Of particular interest are the last handful events of both processes. Wietse
Re: Postfix "fatal: daemon initialization failure"
> On 26 Jan 2022, at 10:43 am, Laura Smith > wrote: > > On stdout I get a few lines of : > strace: decode_nlattr: [xlat 0x, dflt "AF_???", decoders 0x] size is > zero (going to pass nla_type as decoder argument), but opaque data (0x) > is not - will be ignored > > "fgrep fork output-file" doesn't produce any results ? Typo on Wietse's part I think. "master -w" does not fork, it runs in the foreground. You need to see what happens shortly before it bails out, above any resource cleanup as it is exiting. -- Viktor.
Re: Postfix "fatal: daemon initialization failure"
> To debug, run as root: > > strace -f -o output-file /usr/libexec/postfix/master -w > > and look for the process that is created after fork(). On stdout I get a few lines of : strace: decode_nlattr: [xlat 0x, dflt "AF_???", decoders 0x] size is zero (going to pass nla_type as decoder argument), but opaque data (0x) is not - will be ignored "fgrep fork output-file" doesn't produce any results ?
Re: Postfix "fatal: daemon initialization failure"
Wietse Venema: > Laura Smith: > > Any ideas where to start troubleshooting the below ? The logs offer no > > further hints or information as to what might have happened ?!? > > > > postfix-authrelay/master[1179]: fatal: daemon initialization failure > > postfix-inetgen/master[1195]: fatal: daemon initialization failure > > postfix-authrelay/postfix-script[1404]: fatal: mail system startup failed > > postfix-inetgen/postfix-script[1406]: fatal: mail system startup failed > > > > I'm running (or trying to !) Postfix 3.5.6 on Debian Bullseye. > > It creates a master proces in the background, and that background > process disappears before it can log what is wrong. > > To debug, run as root: > > strace -f -o output-file /usr/libexec/postfix/master -w > > and look for the process that is created after fork(). Also, look for AppArmor error logging https://wiki.ubuntu.com/DebuggingApparmor Wietse
Re: Postfix "fatal: daemon initialization failure"
Laura Smith: > Any ideas where to start troubleshooting the below ? The logs offer no > further hints or information as to what might have happened ?!? > > postfix-authrelay/master[1179]: fatal: daemon initialization failure > postfix-inetgen/master[1195]: fatal: daemon initialization failure > postfix-authrelay/postfix-script[1404]: fatal: mail system startup failed > postfix-inetgen/postfix-script[1406]: fatal: mail system startup failed > > I'm running (or trying to !) Postfix 3.5.6 on Debian Bullseye. It creates a master proces in the background, and that background process disappears before it can log what is wrong. To debug, run as root: strace -f -o output-file /usr/libexec/postfix/master -w and look for the process that is created after fork(). Wietse
Postfix "fatal: daemon initialization failure"
Any ideas where to start troubleshooting the below ? The logs offer no further hints or information as to what might have happened ?!? postfix-authrelay/master[1179]: fatal: daemon initialization failure postfix-inetgen/master[1195]: fatal: daemon initialization failure postfix-authrelay/postfix-script[1404]: fatal: mail system startup failed postfix-inetgen/postfix-script[1406]: fatal: mail system startup failed I'm running (or trying to !) Postfix 3.5.6 on Debian Bullseye.
