At Sun, 7 Jun 2020 12:46:51 +0200, Johnny Billquist <b...@update.uu.se> wrote: Subject: Re: Postfix and local mail delivery - still relevant in 2020? > > On 2020-06-07 07:32, Greg A. Woods wrote: > > > > At Sun, 7 Jun 2020 01:53:34 +0200, Johnny Billquist <b...@update.uu.se> > > wrote: > > Is the slow startup you observe happening with just the default > > configuration (for local delivery)? > > It's happening with any kind of configuration. > > Basically, anything written in C++ is almost a lost cause even before > starting.
Do you mean you're under the impression Postifix is written in C++?
Perhaps you should look at the Postfix code -- it's entirely pure C.
> postfix alone takes close to a minute to get going at boot time on my
> VAXen...
> And that is just counting before rc continues to the next item. I
> haven't checked if postfix are then still being busy actually getting
> things sorted in the background...
That's very interesting.
The slowest machine I have running at the moment is a little old Soekris
box, with a FLASH disk as root:
# time /etc/rc.d/postfix restart
postfix/postfix-script: stopping the Postfix mail system
postfix/postfix-script: waiting for the Postfix mail system to terminate
postfix/postfix-script: starting the Postfix mail system
3.31s real 1.21s user 0.76s system
Now that's a little unfair for two reasons.
First off the programs were just running so the good sized parts of
their text segments were already in memory.
So I stopped Postfix, flushed it from RAM by reading a bunch of big
files from an attached cache disk and started it again:
# time /etc/rc.d/postfix start
postfix/postfix-script: starting the Postfix mail system
3.29s real 1.08s user 0.77s system
So, that's still pretty fast.
Now the second unfairness is this:
# file /usr/libexec/postfix/master
/usr/libexec/postfix/master: ELF 32-bit LSB executable, Intel 80386,
version 1 (SYSV), statically linked, for NetBSD 8.99.32, stripped
I.e. my whole build is static-linked.
This is perhaps the big problem since the two processes that have to
start are like this on stock builds:
$ ldd /usr/libexec/postfix/master
/usr/libexec/postfix/master:
-lsaslc.0 => /usr/lib/libsaslc.so.0
-lcrypto.14 => /usr/lib/libcrypto.so.14
-lcrypt.1 => /lib/libcrypt.so.1
-lc.12 => /usr/lib/libc.so.12
-lgcc_s.1 => /usr/lib/libgcc_s.so.1
-lssl.14 => /usr/lib/libssl.so.14
-lgssapi.11 => /usr/lib/libgssapi.so.11
-lkrb5.27 => /usr/lib/libkrb5.so.27
-lhx509.6 => /usr/lib/libhx509.so.6
-lasn1.10 => /usr/lib/libasn1.so.10
-lcom_err.8 => /usr/lib/libcom_err.so.8
-lroken.20 => /usr/lib/libroken.so.20
-lutil.7 => /usr/lib/libutil.so.7
-lwind.1 => /usr/lib/libwind.so.1
-lheimbase.2 => /usr/lib/libheimbase.so.2
-lsqlite3.1 => /usr/lib/libsqlite3.so.1
-lheimntlm.5 => /usr/lib/libheimntlm.so.5
-lldap.4 => /usr/lib/libldap.so.4
-llber.3 => /usr/lib/liblber.so.3
$ ldd /usr/libexec/postfix/qmgr
/usr/libexec/postfix/qmgr:
-lsaslc.0 => /usr/lib/libsaslc.so.0
-lcrypto.14 => /usr/lib/libcrypto.so.14
-lcrypt.1 => /lib/libcrypt.so.1
-lc.12 => /usr/lib/libc.so.12
-lgcc_s.1 => /usr/lib/libgcc_s.so.1
-lssl.14 => /usr/lib/libssl.so.14
-lgssapi.11 => /usr/lib/libgssapi.so.11
-lkrb5.27 => /usr/lib/libkrb5.so.27
-lhx509.6 => /usr/lib/libhx509.so.6
-lasn1.10 => /usr/lib/libasn1.so.10
-lcom_err.8 => /usr/lib/libcom_err.so.8
-lroken.20 => /usr/lib/libroken.so.20
-lutil.7 => /usr/lib/libutil.so.7
-lwind.1 => /usr/lib/libwind.so.1
-lheimbase.2 => /usr/lib/libheimbase.so.2
-lsqlite3.1 => /usr/lib/libsqlite3.so.1
-lheimntlm.5 => /usr/lib/libheimntlm.so.5
-lldap.4 => /usr/lib/libldap.so.4
-llber.3 => /usr/lib/liblber.so.3
That's an awful lotta libraries to rattle through! The runtime linker
isn't terribly fast as it has quite a bit of computation to do, and just
doing the I/O alone to page in enough of all those files on an older
machine is going to take quite a long time.
Now I don't know what your storage situation is on your VAXen, but if
you can possibly afford to static-link your build you'll find things
start so much faster you'll be VERY surprised.
Static linked those two programs are just:
# size /usr/libexec/postfix/master /usr/libexec/postfix/qmgr
text data bss dec hex filename
674331 4428 42840 721599 b02bf /usr/libexec/postfix/master
4219996 26140 53016 4299152 419990 /usr/libexec/postfix/qmgr
So, one is indeed quite large, but not huge by today's standards.
A complete static-build for i386 is 1.8G (that's everything but tests,
debug, and x11).
Now ideally what I want to do for embedded systems is static-link every
binary into one crunchgen binary. I've done this for 5.2 on i386, and
the whole base system (or most of it, no compiler, tests, or x11; and no
ntpd or named or postfix) is just 7.4MB compressed. Yes, just 7.4
megabytes:
$ ls -l -h netbsd-NET5501.EMBED.gz
-r--r--r-- 3 woods wheel 7.4M Jun 5 2016 netbsd-NET5501.EMBED.gz
The whole root filesystem fits in 12.5MB, and the uncompressed bootable
file is still well under 20MB:
$ ls -l netbsd-NET5501.EMBED
-rwxr-xr-x 1 woods wheel 18268428 Jun 5 2016 netbsd-NET5501.EMBED
That's a kernel with an embedded root filesystem containing all most all
the base programs (274 in total) all in one binary:
$ file ramdiskbin
ramdiskbin: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
statically linked, for NetBSD 5.2, stripped
$ size ramdiskbin
text data bss dec hex filename
9700273 270300 4222768 14193341 d892bd ramdiskbin
$ ls -l
ramdiskbin -rwxr-xr-x 1 woods wheel 9974568 Jun 5 2016 ramdiskbin
And let me tell you, running programs on that system is so stunningly
fast you might think they didn't even run even though the processor
isn't a whirlwind by any stretch. Most text pages for _all_ programs
will _always_ be in memory _all_ of the time, even on a rather tiny
machine.
To really make this useful for general NetBSD builds will take some more
hacking on the build system (basically it might go something like always
building a ".cro" file for every program possible, then generating a
crunchgen config to link them all together and generate an mtree file to
generate all the (sym)links; and possibly doing it on a per-set basis
(e.g. one binary for base, one for the toolchain, one for x11, or
something like that).
> 133,000 lines of code is, in my book, a lot. That is not a good sign.
It is a lot, but it's all very modular and very very little is used to
start the default configuration.
--
Greg A. Woods <gwo...@acm.org>
Kelowna, BC +1 250 762-7675 RoboHack <wo...@robohack.ca>
Planix, Inc. <wo...@planix.com> Avoncote Farms <wo...@avoncote.ca>
pgpOCOkbOvNEk.pgp
Description: OpenPGP Digital Signature
