Re: Mail server install questions

[Steven Smith]

> Both dns-server and logrotate are not proper parts of a mail-server, they are 
> potential runtime dependencies.

Whoops that’s right. I can include these as a variant. In the meantime just 
load each service individually. Please let me know if there are other requested 
mods or issues.

Re: Mail server install questions

[Gerben Wierda]

> On 8 Sep 2019, at 21:06, Steven Smith  wrote:
> 
> Again, this isn’t an issue. Just don’t load dns-server. Or delete the port. 
> It’s not even part of the mail-server launch daemon: 
> https://github.com/macports/macports-ports/blob/master/mail/mail-server/Portfile
>  
> 


mail-server.wrapper:

#
# Start
#
Start()
{
port load dns-server
port load clamav-server
port load apache-solr8
port load redis
port load dcc
port load postfix
port load dovecot2
port load rspamd
port load logrotate
}

Both dns-server and logrotate are not proper parts of a mail-server, they are 
potential runtime dependencies. mail-server may contain logrotate settings, but 
in terms of structure log rate has a wider remit.

G  

Re: Mail server install questions

[Steven Smith]

> I was wondering, btw, how the situation was regarding push notifications to 
> iOS devices.

I’m motivated to address that every time I have to wait two seconds for my 
email cache to refresh… I’ll get to that after I clear off some other items on 
the plate.

> I wish you had not put DNS in the mail-server, it’s really not part of it 
> (apple-server maybe, but not mail-server).

Again, this isn’t an issue. Just don’t load dns-server. Or delete the port. 
It’s not even part of the mail-server launch daemon: 
https://github.com/macports/macports-ports/blob/master/mail/mail-server/Portfile

> So, what is your setting for Host Name and Computer name in Server.app? Both 
> FQDN?

Always go FQDN.

Re: Mail server install questions

[Gerben Wierda]

On 8 Sep 2019, at 18:10, Steven Smith  wrote:
> 
>>> Also, I would like to influence the host, domain, and old for the 
>>> auto-configuration. Is there a way to do that? I would like to run the 
>>> mail-server configuration stage again with the correct names
>> 
>> Answering myself: in Server.app: set 'Computer Name’ to th eFQDN (just as 
>> Host Name, so something like host.domain.tld and not just ‘Host’). 
>> /bin/hostname reports the 'Computer Name' field, not the ‘Host Name’ field.
> 
> This out-of-scope for MacPorts, but here’s a few comments about what it 
> sounds like you’re trying to do.
> 
> Migration from old macOS Server.
> 
> I’ve done this myself, trying to follow 
> https://developer.apple.com/support/macos-server/macOS-Server-Service-Migration-Guide.pdf
>  
> .
>  This Apple migration guide is helpful, but deficient in several key aspects, 
> e.g. DNS, VPN, Calendar and Contacts, and Mail.  FWIW, here are my own notes 
> on migration:
> • https://github.com/essandess/macOS-Open-Source-Server 
> 
> • 
> https://github.com/essandess/macOS-Open-Source-Server/blob/master/macOS%20Server%20Migration%20Notes.md
>  
> 
Yes, I found most of these over the months.

> Also, I could be wrong, but it sounds like your trying to migrate your 
> services on the same server as your old, running Server.app version 5.7.

No. I have a brand new Mac mini late 2018 that is being setup up form a 
greenfield state. The old server is currently still running and I am building 
the new one until a cutover point when I will move mail to the new server 
(without the users noticing it, if I’m lucky). But much has to be done. The 
whole server and the remote  backup for instance.

> This would be a Very Bad Idea. Rather, buy a new box,configure it as a 
> sandbox,

What do you mean with 'configure it as a sandbox’? You mean, setit up 
independently? Yes, that is what I am doing.

> harden everything, migrate user data, then deploy. 

Exactly, that is the plan.

For the local macOS accounts I’m still running them as ‘Mobile Accounts’ with 
PHD-like synchronisation set up using ChronoSync. Not ideal, but it works. 

> Running a Mail Server.
> 
> There is no more Server.app Mail server. If you decide to run one yourself, 
> it means knowing what every line in the postfix and dovecot and rspamd 
> configuration does, and knowing and checking the user and group permissions 
> of all files and directories used for the mail server. You can’t assume that 
> the MacPorts mail-server example—or any other—configuration is appropriate 
> for your own network or users. You have to check it line-by-line and test it 
> before you adopt and deploy it. If you’re not willing to embrace these steps, 
> you should purchase a commercial mail server, or use a cloud service email 
> provider, for which there are many options. Aside from the basic rtfm’s on 
> the MTAs and MDAs, here’s a few helpful background links on configuring a 
> BSD/Linux mail server:
> 
> • https://www.c0ffee.net/blog/mail-server-guide/ 
> 
> • 
> https://arstechnica.com/information-technology/2014/02/how-to-run-your-own-e-mail-server-with-your-own-domain-part-1/
>  
> 
> 
> Whether or not you decide to run your own mail server, transitioning from the 
> old Server.app version 5.7 that’s running a full suite of services means 
> configuring a new box from bare metal up. You’ll need to do this 
> step-by-step. One thing that’s still useful useful with the latest Server.app 
> is TLS certificate management, whose cents can be dropped straight into the 
> postfix and dovecot configuration used in the mail-server port.

Yes, that is the plan. I’ve set up smooth use of Letsencrypt certificates which 
are not just automatically updated, but also fully removes the no longer used 
cert from Keychain and Server. I’m using these for Mail and www.rna.nl 
 and they are renewed smoothly without filling up my system 
with outdated ones  (sadly, there is a little issue left, that is that it isn’t 
able yet to detect that when there still is a different certificate for the 
same FQDN (intermediate, to be exact, used for Messages and Open Directory, I 
still haven’t been able to figure out how to use the Letsencrypt certs for 
that, but maybe because of my Computer Name / Host Name mismatch). Script is 
here on GitLab:

Gerben Wierda / macOS-Server-certbot-deployhook 


(Other thing there is a way to manage Virtualbox VMs for docker containers. 
Something I can get back to when I finally have 

Re: Mail server install questions

[Steven Smith]

>> Also, I would like to influence the host, domain, and old for the 
>> auto-configuration. Is there a way to do that? I would like to run the 
>> mail-server configuration stage again with the correct names
> 
> Answering myself: in Server.app: set 'Computer Name’ to th eFQDN (just as 
> Host Name, so something like host.domain.tld and not just ‘Host’). 
> /bin/hostname reports the 'Computer Name' field, not the ‘Host Name’ field.

This out-of-scope for MacPorts, but here’s a few comments about what it sounds 
like you’re trying to do.

Migration from old macOS Server.

I’ve done this myself, trying to follow 
https://developer.apple.com/support/macos-server/macOS-Server-Service-Migration-Guide.pdf.
 This Apple migration guide is helpful, but deficient in several key aspects, 
e.g. DNS, VPN, Calendar and Contacts, and Mail.  FWIW, here are my own notes on 
migration:
• https://github.com/essandess/macOS-Open-Source-Server
• 
https://github.com/essandess/macOS-Open-Source-Server/blob/master/macOS%20Server%20Migration%20Notes.md

Also, I could be wrong, but it sounds like your trying to migrate your services 
on the same server as your old, running Server.app version 5.7. This would be a 
Very Bad Idea. Rather, buy a new box, configure it as a sandbox, harden 
everything, migrate user data, then deploy. Or at least do this on a VM with 
its own independent DNS address, then be prepared to save the disk image and 
write over the old box. It’s easiest and best just to get a new box, and keep 
the old one around just in case. The new Minis are great for this, and used 
2012 Minis that can be upgraded to 10.14 are available quite inexpensively.

Running a Mail Server.

There is no more Server.app Mail server. If you decide to run one yourself, it 
means knowing what every line in the postfix and dovecot and rspamd 
configuration does, and knowing and checking the user and group permissions of 
all files and directories used for the mail server. You can’t assume that the 
MacPorts mail-server example—or any other—configuration is appropriate for your 
own network or users. You have to check it line-by-line and test it before you 
adopt and deploy it. If you’re not willing to embrace these steps, you should 
purchase a commercial mail server, or use a cloud service email provider, for 
which there are many options. Aside from the basic rtfm’s on the MTAs and MDAs, 
here’s a few helpful background links on configuring a BSD/Linux mail server:

• https://www.c0ffee.net/blog/mail-server-guide/
• 
https://arstechnica.com/information-technology/2014/02/how-to-run-your-own-e-mail-server-with-your-own-domain-part-1/

Whether or not you decide to run your own mail server, transitioning from the 
old Server.app version 5.7 that’s running a full suite of services means 
configuring a new box from bare metal up. You’ll need to do this step-by-step. 
One thing that’s still useful useful with the latest Server.app is TLS 
certificate management, whose cents can be dropped straight into the postfix 
and dovecot configuration used in the mail-server port.

Getting back to your specific MacPorts question above, yes, if you change your 
network settings the Portfile activation stage will detect this and change 
default settings appropriately. However, as mentioned, it’s on you to make sure 
the settings in this example configuration are the ones you actually want for 
your own network and mail server, and edit the actual configuration 
appropriately.

I’ve had my own mail server transition from Server.app for about six months 
now, and it’s much nicer than the old one, and, I believe, more secure: postfix 
run in chroot, up-to-date MTA and MDA services, a blazingly fast anti-spam 
capability with much-improved spam/ham training workflow, and DKIM configured 
on the box. After I got it configured and running, I haven’t had to touch it 
through multiple MacPorts upgrades of postfix and dovecot.

Re: Mail server install questions

[Bjarne D Mathiesen]

Bill Cole wrote:
> The dovecot2 port does have less active maintenance than it should, but
> the broader (predominantly Linux) community is huge, the official wiki
> (https://wiki.dovecot.org) is reasonably complete and up-to-date, and
> development is robustly led by a going commercial entity: Open-Xchange.
> The port needs a maintainer, but the software is far from dead.

see : https://github.com/macports/macports-ports/pull/5236

-- 
Bjarne D Mathiesen
Korsør ; Danmark ; Europa
--
denne besked er skrevet i et (næsten) M$-frit miljø
MacOS X 10.13.6 High Sierra :
   17" 2011 MacBook Pro ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3
   2012 Mac Pro ; 2 x 3.46GHz 6-Core Xeon ; 48GB
MacOS X 10.6.8 Snow Leopard :
   Mac Mini ; 2GHz Core 2 Duo (64 bit) ; 4GB (3GB actual) 667MHz
   Mac Mini ; 1.83GHz Core Duo (32 bit) ; 2GB 667Mhz

Re: Mail server install questions

[Gerben Wierda]

> On 8 Sep 2019, at 12:00, Gerben Wierda  wrote:
> 
> Also, I would like to influence the host, domain, and old for the 
> auto-configuration. Is there a way to do that? I would like to run the 
> mail-server configuration stage again with the correct names

Answering myself: in Server.app: set 'Computer Name’ to th eFQDN (just as Host 
Name, so something like host.domain.tld and not just ‘Host’). /bin/hostname 
reports the 'Computer Name' field, not the ‘Host Name’ field.

Re: Mail server install questions

[Gerben Wierda]

I’ve bitten the bullet, uninstalled everything mail-server related I already 
had (though apparently, while I did uninstall dovecot, mail-server still 
thought I had it installed without solr) and installed the mail-server port.

As I did not have Java/JavaSDK installed (and now I have to because of solr) 
the processing of port mail-server failed. 

Also, I would like to influence the host, domain, and old for the 
auto-configuration. Is there a way to do that? I would like to run the 
mail-server configuration stage again with the correct names

G

Re: Mail server install questions

[Bill Cole]

On 7 Sep 2019, at 7:48, Gerben Wierda wrote:

So, I’m back to my (slow) migration of an existing macOS High Sierra 
+ Server.app Apple-’supported’ mail server to one based on macOS 
Mojave + Server.app + macports. Server.app is running. DNS is running. 
Users are in OpenDirectory. Their backup home directories (synced with 
clients) are available. Now it’s time to migrate the mail server. 
That is (as on High Sierra): postfix + dovecot + spamassasin + clamav 
+ greylisting. But while I’m at it I’d like to enable DMARC at 
least. I’m used to managing the configuration by editing files (such 
as main.cf and master.cf, whitelists, etc.) at the unix level. A 
user-friendly way to manage sieve filtering by end-users would be nice 
(I had roundcube once, have been editing the sieve file by hand since 
then on the server).


I am unaware of any end-user-friendly Sieve management tool other than 
the feature in RoundCube. If you find one you like, please consider 
making a port for it.


I have a few questions that arose during preparation (mostly because I 
was unable to find documentation for the port):
I was looking at available documentation. There is a mail-server 
‘aggregate’, but it wants X11. Why?


Generic answer: because developers have a weak sense of what system 
administration is.


Specific answer: mail-server->gmime->vala->graphviz->X11 (and general 
dependency Hell)


I have no idea why mail-server needs gmime. Steven Smith should know, as 
it's a direct dependency.


Also, if you want to avoid pulling in the core Haskell "stack" as a 
result of how Open-Xchange has chosen to regenerate a handful of 
Markdown files in very uncommon circumstances that never include 
MacPorts builds, see the patch in 
https://trac.macports.org/ticket/58890.



How do I find out what variants I need.


A port should specify any required variants of its dependencies which 
are not defaults.



Definitely pure,


??? Is that a typo?

but for instance do I need a variant that can use the local Open 
Directory for authentication (postfix and dovecot) and if so, how do I 
find out?


As Steven has said, that's not an option because OD support in Dovecot 
(which provides auth service for Postfix) was an Apple customization. 
Their custom code should be in their OSS repository (if it still exists) 
and be backportable, if you're into doing such things.


It should be possible to use the LDAP server component of OD as a userdb 
and passdb backend for Dovecot. See 
https://wiki.dovecot.org/AuthDatabase/LDAP


Why would I add lucene etc. if all searching and indexing happens on 
the client side (Mail.app, spotlight)?


That's a big "if."

Some clients use server-side IMAP search, some use their own search 
facilities and/or those provided by their OS (e.g. Spotlight.) If you 
know that all IMAP clients used by your users only use client-side 
search, you do not get anything from any of the server-side search 
options of Dovecot. If clients DO use servber-siode search, it helps a 
great deal to have a server-side index (i.e. Solr.)


I looked at installs for postfix and I noticed in the last year it has 
been installed only once. Is that right? dovecot2 has 2. dovecot2 has 
no maintainer. I find wiki pages, but then they are often 
unfinished/incomplete. Makes me wonder: is there any volume in this 
community or will I be effectively be the only one?


Postfix is typically very stable, getting a major update annually and 
patches in-between only for major bugs & security issues. The MacPorts 
port has had 6 updates in 2019, skipping the 3.4.2-3 patchelevels which 
came unusually fast. See 
https://github.com/macports/macports-ports/commits/master/mail/postfix/Portfile 
for details.


The dovecot2 port does have less active maintenance than it should, but 
the broader (predominantly Linux) community is huge, the official wiki 
(https://wiki.dovecot.org) is reasonably complete and up-to-date, and 
development is robustly led by a going commercial entity: Open-Xchange. 
The port needs a maintainer, but the software is far from dead.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)

Re: Mail server install questions

[Steven Smith]

> The list that mail-server wants to install (given that I’ve already installed 
> postfix, dovecot, and some more because I tried to workaround it) is very 
> long.
> 
> And it seems to consists of a lot of stuff that is not needed at all for a 
> mail server. It raises all these questions I don’t know an answer to and that 
> stalls me. E.g.: Why is cyrus-sasl2 installed when cyrus isn’t there at all 
> (we use dovecot)? Why is cairo (a vector drawing program) installed? Why does 
> it install bind9 and dns-server?

Uh, because all those dependencies are either specified by other dependent 
ports, or are explicitly necessary within the main port.

sasl isn’t a dependency of mail-server, it’s a dependency of openldap, which is 
a dependency of gnupg2, which is a dependency of gpgme, which is a dependency 
of gmime, which is a dependency of rspamd, which mail-server uses. See:

port deps mail-server | grep sasl
port rdeps mail-server | less

And for any mail server you’re running you’ll want to split the horizon so your 
LAN clients don’t reach out to the internet to grab mail from the server 
sitting on the same LAN. You need a DNS server for that. If you don’t want to 
run the MacPorts DNS server, just don’t load it, as discussed in the Portfile 
notes.

> (Maybe some sort of library being reused, but I already have unbound and nsd 
> running, so I don’t need that software and it will only clutter my system and 
> possibly clash with the DNS stuff that is already there). Fontconfig? 
> Freetype? Why should I use logrotate (yet another tool) while newsyslog works 
> fine on macOS (and is already configured)? Etc. 

You should configure your system the way you think it should be configured. 
Mail server configuration involves a lot of interdependent, fragile parts, and 
the mail-server port provides a baseline working example. As it says in its 
Portfile description, it’s very easy to modify to the tools you want to use.

> I kind of dislike getting all this software on my machine when it is not 
> needed. I don’t know what it all does, it might open attack vectors I’m 
> unaware of. It makes me uncertain. It increases maintenance load and risk of 
> extra work.
> 
> When I follow the dependencies of mail-server on MacPorts.org, there is no 
> way I can find the dependencies (e.g. follow the ‘depends on’ and nowhere you 
> end up requiring X11. 

Please try these commands:

man port
port rdeps mail-server

Re: Mail server install questions

[Gerben Wierda]

> On 7 Sep 2019, at 19:10, Steven Thomas Smith  wrote:
> 
>>> I have a few questions that arose during preparation (mostly because I was 
>>> unable to find documentation for the port): I was looking at available 
>>> documentation. There is a mail-server ‘aggregate’, but it wants X11. Why? 
>>> How do I find out what variants I need.
>> I don’t know which dependency wants X11. The port dependencies are all 
>> independent MacPorts ports, so defer to their design.

The list that mail-server wants to install (given that I’ve already installed 
postfix, dovecot, and some more because I tried to workaround it) is very long.

And it seems to consists of a lot of stuff that is not needed at all for a mail 
server. It raises all these questions I don’t know an answer to and that stalls 
me. E.g.: Why is cyrus-sasl2 installed when cyrus isn’t there at all (we use 
dovecot)? Why is cairo (a vector drawing program) installed? Why does it 
install bind9 and dns-server? (Maybe some sort of library being reused, but I 
already have unbound and nsd running, so I don’t need that software and it will 
only clutter my system and possibly clash with the DNS stuff that is already 
there). Fontconfig? Freetype? Why should I use logrotate (yet another tool) 
while newsyslog works fine on macOS (and is already configured)? Etc. 

I kind of dislike getting all this software on my machine when it is not 
needed. I don’t know what it all does, it might open attack vectors I’m unaware 
of. It makes me uncertain. It increases maintenance load and risk of extra work.

When I follow the dependencies of mail-server on MacPorts.org, there is no way 
I can find the dependencies (e.g. follow the ‘depends on’ and nowhere you end 
up requiring X11. 

The following dependencies will be installed: 
 OpenBLAS
 Xft2
 apache-solr8
 bind9
 boost
 cairo
 clamav-server
 clucene
 colm
 cyrus-sasl2
 dcc
 dns-server
 fann
 fontconfig
 freetype
 fribidi
 fswatch
 gd2
 gdbm
 giflib
 gmime
 gnupg2
 gobject-introspection
 gpgme
 graphite2
 graphviz
 gts
 gzip
 harfbuzz
 hiredis
 hyperscan
 jpeg
 kerberos5
 libLASi
 libassuan
 libcomerr
 libgcc
 libgcc9
 libgcrypt
 libgpg-error
 libksba
 libmagic
 libnetpbm
 libpixman
 libpng
 libstemmer
 libunwind
 libunwind-headers
 libusb
 libusb-compat
 logrotate
 lua
 luajit
 lz4
 npth
 openldap
 ossp-uuid
 pango
 perl5
 perl5.28
 pinentry-mac
 pth
 py27-beaker
 py27-funcsigs
 py27-mako
 py27-markdown
 py27-markupsafe
 py27-setuptools
 py37-ply
 python37
 python3_select
 ragel
 readline
 redis
 rspamd
 sf-pwgen
 tcp_wrappers
 tiff
 urw-fonts
 vala
 webp
 xorg-libX11
 xorg-libXau
 xorg-libXaw
 xorg-libXdmcp
 xorg-libXext
 xorg-libXmu
 xorg-libXt
 xorg-libice
 xorg-libpthread-stubs
 xorg-libsm
 xorg-libxcb
 xorg-xcb-proto
 xorg-xcb-util
 xorg-xorgproto
 xpm
 xrender
 zstd

It is worrying me, all this stuff. It gives me the feeling my server will be 
not as clean as possible, but a bag of stuff.

G

Re: Mail server install questions

[Steven Smith]

Also, there’s a port submission of Apple’s Calendar and Contacts Server at 
https://github.com/macports/macports-ports/pull/4978.

This uses mail-server for iMIP along with other MacPorts ports and also works 
great.

Please chime in there with any feedback on the submission or Portfile design.

Re: Mail server install questions

[Steven Smith]

> So, I’m back to my (slow) migration of an existing macOS High Sierra + 
> Server.app Apple-’supported’ mail server to one based on macOS Mojave + 
> Server.app + macports. Server.app is running. DNS is running. Users are in 
> OpenDirectory. Their backup home directories (synced with clients) are 
> available. Now it’s time to migrate the mail server. That is (as on High 
> Sierra): postfix + dovecot + spamassasin + clamav + greylisting. But while 
> I’m at it I’d like to enable DMARC at least. I’m used to managing the 
> configuration by editing files (such as main.cf and master.cf, whitelists, 
> etc.) at the unix level.
Yes, that’s exactly the scenario the port mail-server was designed for. It 
provides configuration of MacPorts ports that performs integrated 
smtp+imap+av+DMARC+dkim+search and other stuff. Its configuration is based in 
part on the old macOS Server.app version 5.7 Mail server, but updates this with 
a lot of newer capabilities. Bottom line: rspamd in, spamassassin out. I’m 
using it right now and it performs much better than the old macOS Server.app 
Mail server. The one thing it’s missing is fine-grain, managed acl’s. You’ll 
need to evaluate the security model for your own situation yourself and make 
adjustments, and please provide feedback if you have any concerns.
> A user-friendly way to manage sieve filtering by end-users would be nice (I 
> had roundcube once, have been editing the sieve file by hand since then on 
> the server).
Dovecot-sieve with specific user space sieve directories is part of this 
configuration.
> I have a few questions that arose during preparation (mostly because I was 
> unable to find documentation for the port): I was looking at available 
> documentation. There is a mail-server ‘aggregate’, but it wants X11. Why? How 
> do I find out what variants I need.
I don’t know which dependency wants X11. The port dependencies are all 
independent MacPorts ports, so defer to their design.
> Definitely pure, but for instance do I need a variant that can use the local 
> Open Directory for authentication (postfix and dovecot) and if so, how do I 
> find out?
The old macOS Server.app had a modified dovecot that provides OD 
authentication. It has some nice features like using UUIDs for user 
directories. This isn’t available now. But the code is open source if anyone 
wanted to patch it in. My expectation is that this would be a major project and 
unnecessary.
> Why does the dovecot port add users/group that already exist on macOS 
> (_dovecot, _dovenull)?
You want to keep MacPorts-managed users/groups separate from OS-managed ones. 
Also, I’m not certain/doubt that those exist on a non-Server.app macOS. 
However, user postfix does, and you definitely don’t want to cross over into 
native postfix space.
> Why would I add lucene etc. if all searching and indexing happens on the 
> client side (Mail.app, spotlight)?
Because solr searches from mobile devices are lightning fast and awesome. Night 
and day difference from the old macOS Server.app configuration.
> I looked at installs for postfix and I noticed in the last year it has been 
> installed only once. Is that right? dovecot2 has 2. dovecot2 has no 
> maintainer. I find wiki pages, but then they are often unfinished/incomplete. 
> Makes me wonder: is there any volume in this community or will I be 
> effectively be the only one?
No. That mainly a function of macOS Server.app providing a Mail server. It was 
completely unnecessary to use open source mail server tools on macOS so long as 
Server.app supported it. But it doesn’t anymore, so we’re have a mail-server 
port that provides a configuration for this capability. Also, those numbers are 
from mpstats users that volunteer usage statistics—that’s not all users.