Re: Is LDAP+SSL supported?

2020-07-26 Thread gilles
July 25, 2020 2:26 PM, "Éloi Rivard"  wrote:

>> In my opinion, table-ldap from extras is doomed as it relies on a lib
>> that
>> is barely maintained and doing LDAP asynchronously is painful.
> 
> Are you saying the support for table-ldap may stop in a near future?
> 

Nope, the table API has been fairly stable for a long time so there is
no extra work for me to leave it as is, it won't stop in a near future
but I won't invest time in it as... I don't use it.


>> I doubt the
>> code will go much further than it currently does.
> 
> However, would you still accept patches for ldaps support?
> 

Yes, but I won't test these patches, I'll only review them and it will
be easier to get them in if you find a couple users willing to test.


>> If the table-procexec work I documented on my blog gets pushed to
>> OpenBSD,
>> then it will ease the writing of a table-ldap with a modern library.
> 
> I will keep an eye on this then.



Re: Is LDAP+SSL supported?

2020-07-20 Thread gilles
July 17, 2020 9:39 AM, "Éloi Rivard"  wrote:

> Hi,
> 
> I have a LDAP table that is working great, but now I would like to avoid clear
> connections and enable SSL. There is an old mail [1] stating that it is not
> possible, but I would like to check if it is still the case 7 years later.
> 
> So here is my configuration: smtpd.conf has a LDAP table.
> 
> table ldap ldap:/etc/mail/ldap.conf
> 
> And /etc/mail/ldap.conf has a very basic configuration:
> 
> url ldap://ldap.mydomain.tld
> username cn=admin,dc=mydomain,dc=tld
> password 
> basedn ou=Users,dc=mydomain,dc=tld
> 
> ...
> 
> Switching ldap:// to ldaps:// prevents OpenSMTPD to start. Am I missing
> something or is the feature not implemented yet?
> 

Hello,

Nothing has changed because I'm the author of the backend and I don't have
an incentive to continue working on it as I've never used it. I thought if
I wrote a working ldap backend, someone with actual interest would pick up
the work and improve it but it never happened.

In my opinion, table-ldap from extras is doomed as it relies on a lib that
is barely maintained and doing LDAP asynchronously is painful. I doubt the
code will go much further than it currently does.

If the table-procexec work I documented on my blog gets pushed to OpenBSD,
then it will ease the writing of a table-ldap with a modern library.


Gilles



Re: Sort to different maildir subdirs based on recipient address?

2020-06-30 Thread gilles
June 30, 2020 12:25 PM, "Unicorn"  wrote:

>> You have two methods to achieve what you want:
>> 
>> 1- use a smarter mda, such as `fdm`, which allows you to specify
>> where mails are supposed to be delivrered based on rules. in this
>> case, you would simply have a rule that recipient address blog@
>> should land in .Blog
> 
> I would like to try to stick to just smtpd, don't want to get in over
> my head with too many moving parts that I don't understand :)
> 
>> 2- alias blog to admin+blog instead of admin, this way when smtpd
>> extracts email extension, it will check if a .blog folder exists and
>> deliver there if exists but this is more limited that a real
>> classification.
> 
> I did not know that was possible, thank you! I tested it and it does
> exactly what I need it to, so I will go with this solution for now.
> 

Great :-)

I tend to use fdm because it lets you do really nice classification,
I encourage you to look at it for future uses !



> Have a great day,
> Unicorn
> 
> PS: I also wanted to say thanks for your awesome guide on setting up
> smtpd with rspamd and dovecot, it has helped me immensely and I
> actually massively appreciate the details that others may find
> trivial. It really helps as a beginner, I wish more people with your
> knowledge made such excellent guides to make these topics more
> accessible and understandable. :)
> 

Thanks !



Re: Sort to different maildir subdirs based on recipient address?

2020-06-30 Thread gilles
June 30, 2020 6:34 AM, "Unicorn"  wrote:

> Hello everyone,
> 

Hello,


> I am a newbie to mail in general and opensmtpd in particular and I am
> currently trying to figure out how to sort mail to various subdirs of
> the maildir based on the address that an email was sent to.
> 
> So basically, this is my intended setup:
> 
> 1. Somebody sends email to b...@mydomain.org
> 2. "blog" is an alias that forwards to a real "admin" account
> 3. In the maildir of "admin", the email goes to a subdir called "blog"
> 
> I looked through the the smtpd.conf(5) manpage and found the "match
> action" directive, my thought was something like this:
> 
> action "sort_to_folder" maildir "~/Maildir/.Blog" alias 
> match for rcpt-to "b...@mydomain.org" action "sort_to_folder"
> 
> But I found that with this setup, junk will not get removed, and if I
> add "junk" after the custom directory, I believe it will end up in
> "~/Maildir/.Blog/.Junk" instead of "~/Maildir/.Junk".
> 
> Is there a more elegant/smart solution that I am missing? I would be
> happy to learn from you. :)
> 

Yeah this won't work like that:

The maildir mda takes the maildir root directory as a parameter, it'll
expect .Junk to be relative.

You have two methods to achieve what you want:

1- use a smarter mda, such as `fdm`, which allows you to specify where mails are
   supposed to be delivrered based on rules. in this case, you would simply have
   a rule that recipient address blog@ should land in .Blog

2- alias blog to admin+blog instead of admin, this way when smtpd extracts email
   extension, it will check if a .blog folder exists and deliver there if exists
   but this is more limited that a real classification.

Gilles



Re: syslog logging changed ?

2020-06-30 Thread gilles
I'm going to investigate this, I don't recall anything change in there but 
there's been tons of portable specific cleanup so it might just have introduced 
a regression.

Gilles


June 26, 2020 8:33 PM, "Reio Remma"  wrote:

> On 26.06.2020 18:03, Harald Dunkel wrote:
> 
>> Hi folks,
>> 
>> before 6.7 the smtpd log file entries were easy to find: Just
>> look for "smtpd" in /var/log/mail.log.
>> 
>> With 6.7 this became "y express". On OpenBSD 6.7 its still "smtpd"
>> as expected, so I wonder wth?
>> 
>> Regards
>> Harri
> 
> Unfortunately something has broken since last release.
> 
> I was unable to track it down myself:
> 
> https://github.com/OpenSMTPD/OpenSMTPD/issues/1059
> 
> Good luck!
> Reio



Re: Newbie config question

2020-06-05 Thread gilles
On my phone but I'll show you tomorrow if no one answers before, this is trivialGillesOn Jun 5, 2020 18:28, David Favor  wrote:I've been wrestling with this for days with no progress.

Can someone drop me a v6.6.4 config to do something similar to the following.

    da...@davidfavor.com   - maildir
    i...@davidfavor.com    - forward to da...@davidfavor.com
    supp...@davidfavor.com - forward to f...@helpdesk.com using MailGun Relay Service

    supp...@radicalhealth.com - maildir
    i...@radicalhealth.com    - forward to supp...@radicalhealth.com
    da...@radicalhealth.com   - send natively to da...@davidfavor.com (no Smarthost or Relay Service)

Just a raw config file will be fine, I can remove
whatever I don't require right now, like DKIM signing,
which I'll add later.

I'm just trying to get basic OpenSMTPD delivery working.

Thanks.




Re: "bouncing messages from ..." (was: request (privately) for maillog)

2020-06-03 Thread gilles
I've done some maintenance on all mail servers this night,
it's going to be a bit shaky today


June 4, 2020 7:01 AM, ml+opensmtpd_m...@esmtp.org wrote:

> Happened again for me. Anyone else?



Re: Unable to remove mail from queue

2020-06-03 Thread Gilles Chehade
this is due to a short-coming with how inflight envelopes are handled:

when a mail is passed from scheduler to mta, it is marked as "inflight" and
can't be removed until it comes back to scheduler.

this is usually not a big deal because an envelope is marked inflight only
a few seconds usually...

... except that eric@ and I came with an optimization to avoid envelopes
going back and forth into the scheduler when they have multiple routes or
when there's a chance a route gets enabled soon, they are kept in the MTA
for a bit longer, but this means that they can't be removed either.

we had discussed a quick fix for this but since the MTA layer is supposedly
going to be simplified a lot, it was not worth the effort.

I don't know where eric@ stands wrt this as of today


On Sun, May 31, 2020 at 8:00 PM Chris Bennett 
wrote:

> On Sun, May 31, 2020 at 05:24:18PM +0200, Mischa Peters wrote:
> > Hi All,
> >
> > I just noticed something strange on one of my mailservers running
> OpenSMTPd 6.7.0p1  (OpenBSD 6.7).
> > The mailserver was trying to deliver a spam mailbounce to fedex, it kept
> failing so I removed it from the queue.
> > The logs kept showing it was being delivered, eventhough nothing was
> showing in the queue.
> > After a restart of smtpd the message did show up in the queue again.
> >
> > root@smtp1:~ # smtpctl show queue
> > cd9b0933db878954|local|mta|auth|@|prvs=1417a4ec2a=bou...@nds.fedex.com
> |prvs=1417a4ec2a=bou...@nds.fedex.com
> |1590676002|1590676002|1590937323|0|inflight|99|
> >
> > root@smtp1:/var/spool/smtpd/queue/cd/cd9b0933 # smtpctl remove
> cd9b0933db878954
> > 1 envelope removed
> > root@smtp1:/var/spool/smtpd/queue/cd/cd9b0933 # smtpctl remove
> cd9b0933db878954
> > 0 envelope removed
> > root@smtp1:/var/spool/smtpd/queue/cd/cd9b0933 # ls -la
> > total 52
> > drwx--  2 _smtpq  wheel512 May 28 16:26 .
> > drwx--  3 _smtpq  wheel512 May 30 20:49 ..
> > -rw---  1 _smtpq  wheel316 May 28 16:26 cd9b0933db878954
> > -rw---  1 _smtpq  wheel  19296 May 28 16:26 message
> > root@smtp1:/var/spool/smtpd/queue/cd/cd9b0933 # smtpctl show queue
> > root@smtp1:/var/spool/smtpd/queue/cd/cd9b0933 # rcctl restart smtpd
> > smtpd(ok)
> > smtpd(ok)
> > root@smtp1:/var/spool/smtpd/queue/cd/cd9b0933 # smtpctl show queue
> > cd9b0933db878954|local|mta|auth|@|prvs=1417a4ec2a=bou...@nds.fedex.com
> |prvs=1417a4ec2a=bou...@nds.fedex.com
> |1590676002|1590676002|1590937456|0|inflight|1|
> > root@smtp1:/var/spool/smtpd/queue/cd/cd9b0933 # ls -la
> > total 52
> > drwx--  2 _smtpq  wheel512 May 28 16:26 .
> > drwx--  3 _smtpq  wheel512 May 30 20:49 ..
> > -rw---  1 _smtpq  wheel316 May 28 16:26 cd9b0933db878954
> > -rw---  1 _smtpq  wheel  19296 May 28 16:26 message
> >
> > I assume this is not the expected result. :)
> > What else can I collect to pinpoint what is going on, before I rm the
> files?
> >
> > Mischa
> >
> >
>
> I also had this same problem. I rm'd the files.
> However, what is the right solution?
> (I was in a big rush and had to quickly solve the problem.)
>
> Chris Bennett
>
>
>
>


Re: Hello@All

2020-05-29 Thread gilles
May 28, 2020 10:51 AM, drav...@dravionsoftware.com wrote:

> Hi,
> 

Hi,


> I want to introduce myself to the list ;d
> 

Welcome


> By the way, is there anybody out there, tried to make OpenSMTPD work on
> Cygwin/Windows?
> 
> I was able to built OpenSMTPD under Windows Subsystem for Windows (WSL),
> but i am curious and eager to know if someone had some progress, making
> it work on Cygwin,
> MinGW of MSYS2 as well.
> 

No one has ever discussed this with me so I'm fairly confident no one tried :-)



Re: new table backends

2020-05-27 Thread gilles
May 27, 2020 2:27 AM, "Edgar Pettijohn"  wrote:

>
> [...]
> 
> Sweet. Looking at 
> https://github.com/poolpOrg/go-opensmtpd/blob/master/table/table.go
> seems like it will be pretty simple to write some nice tables.
> 

yes, I have also written a py-opensmtpd interface to table API ...

... and wrote a working table-consul in < 20 minutes which is < 50 lines of 
python



new table backends

2020-05-26 Thread gilles
Hellow,

I have been working on a new table backend: table-procexec.

What it does is translate the imsg table API to a line-based protocol that is
very similar to what we did for filters. A table backend can become a program
consuming table requests from stdin and responding to stdout:

stdin : 
table|0.1|1590532670.0123456|check|1234567890ABCDEF|mailaddr|gil...@poolp.org
stdout: table|1234567890ABCDEF|found


The idea behind this is to unlock table backend development to people who are
not necessarily C developers and who could still write useful implementations
in other languages. Like with filters, this makes it possible to use anything
from awk to shell, Go, Python and what not without OpenSMTPD caring about how
these backends are implemented.

I have written a Golang package to abstract the protocol details and that let
you write a backend by implementing the four basic operations of table API. A
sample table-example.go is available here to see how a backend looks like:

https://gist.github.com/poolpOrg/b3b97a65791a11a49f5e76ea51331ae4


If people are willing to help implement some backends, hit me up

Cheers,
Gilles



Re: Including remote addresses in smtpd syslog output

2020-05-26 Thread gilles
May 26, 2020 4:36 AM, "Kevin Zheng"  wrote:

> Hi folks,

Hellow,

> [...]
> 
> Chasing changes in syslog output is a part of maintaining software like
> SSHGuard. Unfortunately, my parser (which recently learned how to
> pledge!) is a bit dull and would require some re-education to remember
> SMTP sessions and their associated IP addresses. So, my questions are:
> 
> Why did OpenSMTPD stop reporting IP addresses on every line?
> 
> Is there any chance that OpenSMTPD can put IP addresses back on every line?

Sorry, I'll write a detailed answer for future reference:

A long time ago, people started relying on logs to write all kinds of scripts
and processors, which led to requests to add some info here and another there
until the logs became so dense that they scared users. People writing scripts
were happy, humans reading logs were overwhelmed with redundant informations.

Around this time, Theo asked me to make logs more readable, ideally so they'd
fit terminal lines for most logs. I did some compaction (i.e.: source -> src)
but this didn't help much because, as you mention, scripts used parsers which
didn't keep state and logs grew dense to keep state on their behalf. What did
not help further is that slight changes in format broke all existing scripts.


So it became obvious we were doing something wrong:

We WANT to make it easy for tools to extend OpenSMTPD, but this can't come at
the cost of freezing our maillog format or at the cost of users having harder
time going through maillog. The consumers, humans and scripts, have different
expectations so they need to consume different logs.


We now provide a reporting API which is basically a stream of events that can
be consumed by tools. It is a line-based format which is not meant to be read
by humans but meant to be easily parsed by tools and that provides all of the
information necessary to replicate the session states. Using this stream, one
can write a tiny filter which aggregates info and outputs logs tailored for a
specific third-party application with a guarantee that it won't break when we
make a subtle change to the maillog format. If I were working on SSHGuard for
example, I'd write an sshguard-exporter script that reads the stream and that
outputs to syslog a format SSHguard recognizes. This way, an smtpd user would
simply:

filter sshguard proc-exec "sshguard-exporter"
listen on all filter sshguard
action "foobar" relay filter sshguard

SSHguard itself would never need to be altered to follow changes in logs.


We decided to turn back maillog format into a human-oriented format, one that
guarantees you will find your info with two grep, but that does not guarantee
it won't have subtle changes at each release if they make logs more readable.
The maillog format logs events and their associated data, removing redundancy
and not keeping context across logs.

This allows me, as a human to easily grep for info and not be overflowed with
a lot of unrelated log lines that happen to carry the same information:

$ grep 'smtp connected' /var/log/maillog|tail -3
May 26 06:22:33 in smtpd[75500]: 0192b8a91f05254a smtp connected 
address=199.185.178.25
host=mail.openbsd.org
May 26 06:24:49 in smtpd[75500]: 0192b8ac864dd3e1 smtp connected 
address=199.185.178.25
host=mail.openbsd.org
May 26 06:24:56 in smtpd[75500]: 0192b8b1f1c2f60c smtp connected 
address=199.185.178.25
host=mail.openbsd.org

$ grep 'address=199.185.178.25' /var/log/maillog|tail -3
May 26 06:13:38 in smtpd[75500]: 0192b895dd7ab75f smtp connected 
address=199.185.178.25
host=mail.openbsd.org
May 26 06:22:12 in smtpd[75500]: 0192b8a43a226ada smtp connected 
address=199.185.178.25
host=mail.openbsd.org
May 26 06:22:33 in smtpd[75500]: 0192b8a91f05254a smtp connected 
address=199.185.178.25
host=mail.openbsd.org

And this also allows me to get a very compact format for an entire session:

$ grep '0192b8b1f1c2f60c' /var/log/maillog
May 26 06:24:56 in smtpd[75500]: 0192b8b1f1c2f60c smtp connected 
address=199.185.178.25
host=mail.openbsd.org
May 26 06:24:57 in smtpd[75500]: 0192b8b1f1c2f60c smtp tls
ciphers=TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256
May 26 06:25:04 in smtpd[75500]: 0192b8b1f1c2f60c smtp message msgid=4baeb9ac 
size=3110 nrcpt=1
proto=ESMTP
May 26 06:25:04 in smtpd[75500]: 0192b8b1f1c2f60c smtp envelope 
evpid=4baeb9ac8c6799b0
from= to=
May 26 06:25:14 in smtpd[75500]: 0192b8b1f1c2f60c smtp disconnected reason=quit


With this split, human log vs reporting api for tools, tomorrow we can decide
in the sample above to rename "message" into "msg" knowing that human readers
will not be affected and that scripts will not break as long as they use that
reporting API.

Hope it clarifies a bit,
Gilles



OpenSMTPD 6.7.1p1 released

2020-05-21 Thread gilles
Hello,

Two bugs were spotted by package maintainers right after the 6.7.0p1 release:

a- a packaging issue causing asr.h to be installed on the host system
b- a possible crash when the MTA establishes an IPv6 connection


I have rolled a minor release with the two bug fixes applied.

https://www.opensmtpd.org/archives/opensmtpd-6.7.1p1.tar.gz
https://github.com/OpenSMTPD/OpenSMTPD/releases/tag/6.7.1p1

Cheers,



WIP: filter-prometheus

2020-05-20 Thread gilles
Hello misc@,

I have started working on a new filter yesterday which implements a prometheus 
exporter.

This allows OpenSMTPD to expose metrics which can be scraped by prometheus to 
generate dashboards:

https://twitter.com/PoolpOrg/status/1263089397387464704

It is a work in progress but I don't have much experience with prometheus,
so if some of you are prometheus users and want to contribute let me know :-)

Gilles



Announce: OpenSMTPD 6.7.0p1 released

2020-05-19 Thread gilles
OpenSMTPD 6.7.0p1 has just been released.

OpenSMTPD is a FREE implementation of the SMTP protocol with some common
extensions. It allows ordinary machines to exchange e-mails with systems
speaking the SMTP protocol. It implements a fairly large part of RFC5321
and can already cover a large range of use-cases.

It runs on OpenBSD, NetBSD, FreeBSD, DragonFlyBSD, Linux and OSX.

The archives are now available from the main site at www.OpenSMTPD.org

We would like to thank the OpenSMTPD community for their help in testing
the snapshots, reporting bugs, contributing code and packaging for other
systems.

This is a major release with multiple bug fixes and new features.


Dependencies note:
==

This release builds with LibreSSL > 3.0.2 or OpenSSL > 1.1.0.

It's preferable to depend on LibreSSL as OpenSMTPD is written and tested
with that dependency. In addition, the features parity is not respected,
some features will not be available with OpenSSL, like ECDSA server-side
certificates support in this release. OpenSSL library is considered as a
best effort target TLS library and provided as a commodity, LibreSSL has
become our target TLS library.


Changes in this release:


New Features:

- Allowed use of the smtpd(8) session username in built-in filters when 
available.
- Introduced a bypass keyword to smtpd(8) so that built-in filters can bypass 
processing when a condition is met.
- Allowed use of 'auth' as an origin in smtpd.conf(5).
- Allowed use of mail-from and rctp-to as for and from parameters in 
smtpd.conf(5).


Bug fixes:

- Ensured legacy ssl(8) session ID is persistent during a client TLS session, 
fixing an issue using TLSv1.3 with smtp.mail.yahoo.com.
- Fixed security vulnerabilities in smtpd(8). Corrected an out-of-bounds read 
in smtpd allowing an attacker to inject arbitrary commands into the envelope 
file to be executed as root, and ensured privilege revocation in smtpctl(8) to 
prevent arbitrary commands from being run with the _smtpq group.
- Allowed mail.local(8) to be run as non-root, opening a pipe to lockspool(1) 
for file locking.
- Fixed a security vulnerability in smtpd(8) which could lead to a privilege 
escalation on mbox deliveries and unprivileged code execution on lmtp 
deliveries.
- Added support for CIDR in a: spf atoms in smtpd(8).
- Fixed a possible crash in smtpd(8) when combining "from rdns" with nested 
virtual aliases under a particular configuration.


Experimental Features: 

- Introduced smtp-out event reporting.
- Improved filtering protocol.


Checksums:
==

  SHA256 (opensmtpd-6.7.0p1.tar.gz) =
  c13f5dd7b9cb9421eabb62068537d90b0441cdb3ca2e5c1e753d7aee01b90bb9


Verify:
===

Starting with version 5.7.1, releases are signed with signify(1).

You can obtain the public key from our website, check with our community
that it has not been altered on its way to your machine.

   $ wget https://www.opensmtpd.org/archives/opensmtpd-20181026.pub

Once you are confident the key is correct, you can verify the release as
described below:

1- download both release tarball and matching signature file to same directory:

   $ wget https://www.opensmtpd.org/archives/opensmtpd-6.7.0p1.sum.sig
   $ wget https://www.opensmtpd.org/archives/opensmtpd-6.7.0p1.tar.gz


2- use `signify` to verify that signature file is properly signed and that the
   checksum matches the release tarball you downloaded:

   for portable version:
   $ signify -C -e -p opensmtpd-20181026.pub -x opensmtpd-6.7.0p1.sum.sig
   Signature Verified
   opensmtpd-6.7.0p1.tar.gz: OK


If you don't get an OK message, then something is not right and you should not
install without first understanding why it failed.


Support:


You are encouraged to register to our general purpose mailing-list:
http://www.opensmtpd.org/list.html

The "Official" IRC channel for the project is at:
#OpenSMTPD @ irc.freenode.net


Support us:


The project is maintained by volunteers, you can support us by:

- donating time to help test development branch during development cycle
- donating money to either one of the OpenBSD or OpenSMTPD project
- sponsoring developers through direct donations or patreon
- sponsoring developers through contracts to write features

Get in touch with us by e-mail or on IRC for more informations.


Reporting Bugs:
===

Please read http://www.opensmtpd.org/report.html
Security bugs should be reported directly to secur...@opensmtpd.org
Other bugs may be reported to b...@opensmtpd.org



Re: Questions about the proc-exec filter API

2020-05-19 Thread gilles
Here is fine yes

May 19, 2020 3:46 AM, po...@protonmail.com (mailto:po...@protonmail.com) wrote:
OpenSMTPd 6.7.0 
Based upon Filters(7) I have written a proof-of-concept filter which is 
functioning properly 
I have several questions about the details of the API. 
Is this the most appropriate forum for such questions?


Re: plain text authentication

2020-05-12 Thread gilles
May 12, 2020 1:47 AM, "Thomas Bohl"  wrote:

> Hi,
> 
>> I need to use plain text authentication. I have to migrate an old > postfix 
>> server that uses this
>> authentication mode. I have a lot of > devices configured in this way. I 
>> have to plan the migration
>> to TLS, > while I want to use OpenSMTPD with plain text authentication. It's 
>> possible?
> 
> It's not possible to use plain text authentication with OpenSMTPD.
> https://www.mail-archive.com/misc@opensmtpd.org/msg04397.html
> 
> I guess your migration plan has to give OpenSMTPD a different hostname or 
> port and let postfix run
> till ever device is moved to a TLS config.

The question is confusing:

PLAIN AUTH is an authentication method which uses plaintext credentials and 
encodes them in base64
in opposition to other authentication methods relying on challenges.

OpenSMTPD supports PLAIN AUTH, what it doesn't support is authentication 
outside of a TLS channel.

So yes, you can use OpenSMTPD with plain authentication BUT you can't use 
authentication if you do
not setup TLS first.

Gilles



Re: .forward format and usage

2020-04-05 Thread gilles
April 5, 2020 2:47 AM, "grmat"  wrote:

> Hi there,
> 

Hello,


> I'd like to setup GNU Mailman with OpenSMTPD. In #1040[1], poolpOrg told
> me command processing is to be done with a .forward file instead of
> aliases:
> 
>> I think aliases should not support command processing at all as handling 
>> these from a .forward file
>> is equivalent
> 
> So I looked up the man page forward(5) and it also says
> 
>> A .forward file contains a list of expansion values, as described in 
>> aliases(5).
> 
> Therefore, I assumed I could just set up a .forward file for mailman
> (in addition to aliasing the addresses to user "mailman").
> Example content for running mailing list gene...@example.org:
> 
> # general mailing list
> general: "|/path/to/mailman/mail/mailman post general"
> general-admin: "|/path/to/mailman/mail/mailman admin general"
> general-bounces: "|/path/to/mailman/mail/mailman bounces general"
> [...]
> 
> But apparently, that's not how .forward files work. aliases are
> key-value, but .forward rules are only values, and a mail would always
> be forwarded to *each* of the lines. At least that's what I understand
> from the errors I get. I don't think it's possible to set different
> forwarding rules for different addresses which are all aliased to the
> same user, hence I still have to go for aliases in this case?
> 
> Could anyone please confirm that or tell me where I'm wrong. Thanks in
> advance!
> 

It is possible to use some variables in .forward files, so you can write
a small script (this is just an untested example):

#! /bin/sh

if test "$1" = "general"; then
/path/to/mailman/mail/mailman post general
elif test "$1" = "general-admin"; then
/path/to/mailman/mail/mailman admin general
elif test "$1" = "general-bounces"; then
/path/to/mailman/mail/mailman bounces general
fi

Then in ~mailman/.forward, use:

|/path/to/that/wrapper "%{dest.user}"


This is superior to adding the commands in /etc/mail/aliases as you are
guaranteed that the mailman utility is privileges separated from daemon
and ran as the user who owns the .forward file.


For other software, like mlmmj it is even simpler as the command itself
uses the recipient email address to determine what command it should be
running and no script is needed.



Re: "bouncing messages from ..." (was: request (privately) for maillog)

2020-02-25 Thread gilles
February 26, 2020 8:30 AM, gil...@poolp.org wrote:

> February 24, 2020 9:08 PM, gil...@poolp.org wrote:
> 
>> February 24, 2020 6:54 PM, ml+opensmtpd_m...@esmtp.org wrote:
>> 
>>> On Mon, Feb 24, 2020, Peter J. Philipp wrote:
>> 
>> I got another "bouncing messages from misc@opensmtpd.org" message. The
>>> Me too... and it's the second time I cannot find any error log
>>> about such a failed delivery (I asked for more info the previous
>>> time it happened but so far nobody replied) so together with this
>>> report it seems the problem might be on the server side?
>> 
>> I'm investigating but I suspect it is an issue with mlmmj,
>> I get the same bounces ... and I'm local.
> 
> I can confirm it is an issue with mlmmj, there's no failed SMTP attempt
> for any of you who reported getting this message.
> 

found out.

upon restart of the MX @ opensmtpd.org, the list goes through greylisting
and the first session gets bounced once. I sometimes restart several time
to test diffs which explains why at times people get more of these, and I
obviously did a restart when the errata was published.

I changed the config so it doesn't go through greylisting again.



Re: "bouncing messages from ..." (was: request (privately) for maillog)

2020-02-25 Thread gilles
February 24, 2020 9:08 PM, gil...@poolp.org wrote:

> February 24, 2020 6:54 PM, ml+opensmtpd_m...@esmtp.org wrote:
> 
>> On Mon, Feb 24, 2020, Peter J. Philipp wrote:
>> 
>>> I got another "bouncing messages from misc@opensmtpd.org" message. The
>> 
>> Me too... and it's the second time I cannot find any error log
>> about such a failed delivery (I asked for more info the previous
>> time it happened but so far nobody replied) so together with this
>> report it seems the problem might be on the server side?
> 
> I'm investigating but I suspect it is an issue with mlmmj,
> I get the same bounces ... and I'm local.
> 

I can confirm it is an issue with mlmmj, there's no failed SMTP attempt
for any of you who reported getting this message.

Gilles



Re: "bouncing messages from ..." (was: request (privately) for maillog)

2020-02-24 Thread gilles
February 24, 2020 6:54 PM, ml+opensmtpd_m...@esmtp.org wrote:

> On Mon, Feb 24, 2020, Peter J. Philipp wrote:
> 
>> I got another "bouncing messages from misc@opensmtpd.org" message. The
> 
> Me too... and it's the second time I cannot find any error log
> about such a failed delivery (I asked for more info the previous
> time it happened but so far nobody replied) so together with this
> report it seems the problem might be on the server side?


I'm investigating but I suspect it is an issue with mlmmj,
I get the same bounces ... and I'm local.

Gilles



OpenSMTPD 6.6.4p1 released: addresses CRITICAL vulnerability

2020-02-24 Thread gilles
Hello misc@,

Qualys has found another critical vulnerability in OpenSMTPD.

It is very important that you upgrade your setups AS SOON AS POSSIBLE.

I can't comment yet as I was not involved in the bug fixing this time,
and didn't see the advisory, just the resulting bug fix diff.

I'll comment and do an analysis of the issue in a few days.


On OpenBSD:
---

Binary patches are available through syspatch.

Just run the syspatch command and make sure that your OpenSMTPD was restarted:

$ doas syspatch

On other systems
---

I have released version 6.6.4p1 of OpenSMTPD which addresses the vulnerability.

It is available from our website:

https://www.opensmtpd.org/archives/opensmtpd-6.6.4p1.tar.gz
https://www.opensmtpd.org/archives/opensmtpd-6.6.4p1.sum.sig

It is also available from Github:

https://github.com/OpenSMTPD/OpenSMTPD/releases/download/6.6.4p1/opensmtpd-6.6.4p1.tar.gz
https://github.com/OpenSMTPD/OpenSMTPD/releases/download/6.6.4p1/opensmtpd-6.6.4p1.sum.sig

Or using the `6.6.4p1` tag if you're building from source.



OpenSMTPD 6.6.3p1 released

2020-02-10 Thread gilles
Hello,

I have just released the minor version 6.6.3p1 of OpenSMTPD.


Following the advisory from Qualys late January, I have discussed various 
mitigation on my blog:

https://poolp.org/posts/2020-01-30/opensmtpd-advisory-dissected/


Several were implemented in OpenBSD -current and this new releases back-ports 
them to the portable version.

With this release:

- OpenSMTPD now declares maildir instead of mbox in its default configuration
- the mbox delivery method now uses a specific code path for execution with 
fixed parameters
- the mbox delivery method no longer requires privileges in the daemon
- the lmtp delivery method no longer receives sender/recipient on the command 
line

Other mitigation will be back-ported as they become available and new releases 
will be issued to include them.

In the mean time, I highly recommend that you:

- upgrade to this version to reduce the attack surface.
- stop using mbox is possible.
- stop delivering mail to root but create an alias to an unprivileged user 
instead.


The release can be downloaded from our website:

https://www.opensmtpd.org/archives/opensmtpd-6.6.3p1.tar.gz

or from Github:

https://github.com/OpenSMTPD/OpenSMTPD/releases/tag/6.6.3p1



Re: 553 ORCPT address syntax errors on OpenBSD-6.6-current

2020-02-03 Thread gilles
mail.local needs to be updated too

February 3, 2020 6:11 PM, "Scott Vanderbilt"  wrote:

> On 2/3/2020 8:11 AM, Gilles Chehade wrote:
> 
>> On Mon, Feb 03, 2020 at 06:37:38AM -0800, Scott Vanderbilt wrote:
>>> I'm starting to get several log entries for several errors of type:
>>> 
>>> 553ORCPT address syntax error
>>> 
>>> The error is intermittent since the server is able to process other incoming
>>> mails without error. For instance, I just sent myself an email from GMail,
>>> and it came through successfully.
>>> 
>>> Typical log entry will look like:
>>> 
>>> Feb?? 3 06:02:26 callistus smtpd[21460]: cb9690ea8af2a8ec smtp connected
>>> address=198.2.185.67 host=mail67.suw111.mcdlv.net
>>> Feb?? 3 06:02:26 callistus smtpd[21460]: cb9690ea8af2a8ec smtp tls
>>> ciphers=TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256
>>> Feb?? 3 06:02:27 callistus smtpd[21460]: cb9690ea8af2a8ec smtp
>>> failed-command command="RCPT TO:
>>> ORCPT=rfc822;li...@datagenic.com" result="553 ORCPT address syntax error"
>>> Feb?? 3 06:02:27 callistus smtpd[21460]: cb9690ea8af2a8ec smtp disconnected
>>> reason=quit
>>> 
>>> Mail logs prior to latest update to 6.6-current are free of these errors, so
>>> presumably the regression has been introduced in the latest snapshot
>>> (OpenBSD 6.6-current (GENERIC.MP) #628: Sat Feb?? 1 23:32:22 MST 2020). In
>>> fact, it looks as though it is related to this recent commit:
>>> https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.sbin/smtpd/smtp_session.c.diff?r1=1.422=1.423
>>> 
>>> In which case, it may be suggested that the change is being perhaps a little
>>> too strict.
>> 
>> indeed addresses in ORCPT are prefixed with a character that's not allowed in
>> the mailaddr character set. the fix has been committed, thanks
> 
> Thank you for your prompt response.
> 
> Now I am getting different errors after patching smtpd:
> 
> Feb 3 09:02:31 callistus smtpd[89610]: d5b5d676a4b8123d smtp connected 
> address=207.244.88.153
> host=hermes.apache.org
> Feb 3 09:02:32 callistus smtpd[89610]: d5b5d676a4b8123d smtp message 
> msgid=d9c025c9 size=6821
> nrcpt=1 proto=SMTP
> Feb 3 09:02:32 callistus smtpd[89610]: d5b5d676a4b8123d smtp envelope 
> evpid=d9c025c9d5c66d45
> from= 
> to=
> Feb 3 09:02:32 callistus mail.local: may only be run by the superuser
> Feb 3 09:02:32 callistus smtpd[89610]: d5b5d67c430c9962 mda delivery 
> evpid=d9c025c9d5c66d45
> from= 
> to= rcp
> t= user=lists delay=1s result=PermFail stat=Error 
> ("mail.local: may only be
> run by the superuser")
> Feb 3 09:02:32 callistus smtpd[89610]: d5b5d676a4b8123d smtp disconnected 
> reason=quit
> 
> No doubt I've done something wrong.



Re: 553 ORCPT address syntax errors on OpenBSD-6.6-current

2020-02-03 Thread Gilles Chehade
On Mon, Feb 03, 2020 at 06:37:38AM -0800, Scott Vanderbilt wrote:
> I'm starting to get several log entries for several errors of type:
> 
> 553ORCPT address syntax error
> 
> The error is intermittent since the server is able to process other incoming
> mails without error. For instance, I just sent myself an email from GMail,
> and it came through successfully.
> 
> Typical log entry will look like:
> 
> Feb?? 3 06:02:26 callistus smtpd[21460]: cb9690ea8af2a8ec smtp connected
> address=198.2.185.67 host=mail67.suw111.mcdlv.net
> Feb?? 3 06:02:26 callistus smtpd[21460]: cb9690ea8af2a8ec smtp tls
> ciphers=TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256
> Feb?? 3 06:02:27 callistus smtpd[21460]: cb9690ea8af2a8ec smtp
> failed-command command="RCPT TO:
> ORCPT=rfc822;li...@datagenic.com" result="553 ORCPT address syntax error"
> Feb?? 3 06:02:27 callistus smtpd[21460]: cb9690ea8af2a8ec smtp disconnected
> reason=quit
> 
> Mail logs prior to latest update to 6.6-current are free of these errors, so
> presumably the regression has been introduced in the latest snapshot
> (OpenBSD 6.6-current (GENERIC.MP) #628: Sat Feb?? 1 23:32:22 MST 2020). In
> fact, it looks as though it is related to this recent commit: 
> https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.sbin/smtpd/smtp_session.c.diff?r1=1.422=1.423
> 
> In which case, it may be suggested that the change is being perhaps a little
> too strict.
> 

indeed addresses in ORCPT are prefixed with a character that's not allowed in
the mailaddr character set. the fix has been committed, thanks




OpenSMTPD advisory dissected

2020-01-31 Thread gilles
Hello,

I have written a detailed write-up about the recent event:

https://poolp.org/posts/2020-01-30/opensmtpd-advisory-dissected/

Hope it clarifies what happened and what we intend to do to avoid it in the 
future.

Gilles



Re: Interim mitigation for CVE-2020-7247

2020-01-29 Thread gilles
January 29, 2020 12:19 PM, "Andreas Broecking"  wrote:

> Hi all,
> 
> first of all, thanks Gilles for the heads-up and a fix on short notice.
> 
> For people like me who relay on the portable version and for systems which 
> relay on built packages
> as they lack the local development tools, a filter should help to mitigate 
> the problem until a
> package could be built on another system.
> 
> Shouldn’t something like 
> 
> filter exploit_check phase mail-from match mail-from regex { '<*\;*' } \
> disconnect "550 no exploiting, kthx”
> 
> listen on $interface filter exploit_check
> 
> sufficiently mitigate the problem?
> I am not fluent in regex’ing so maybe I am missing an edge case. It does 
> prevent the example
> exploit and any others I’ve seen in the last hours.
> 

filter exploit_check phase mail-from match mail-from regex ".*;.*" \
disconnect "550 no exploiting, kthx”

listen on $interface filter exploit_check

This should work yes but I don't have an unpatched system to test it.
You also need it on "listen on socket"

The best mitigation would be to use maildir instead of mbox as it's not 
impacted.



OpenSMTPD 6.6.2p1 released: addresses CRITICAL vulnerability

2020-01-28 Thread gilles
Hello misc@,

Qualys has found a critical vulnerability leading to a possible privilege 
escalation.

It is very important that you upgrade your setups AS SOON AS POSSIBLE.

We'll provide more details when the advisory will be out and I'll take time to 
write
about how this bug was made possible, but in the meantime get your setups fixed 
!


On OpenBSD:
---

Binary patches are available through syspatch.

Just run the syspatch command and make sure that your OpenSMTPD was restarted:

$ doas syspatch



On other systems
---

I have released version 6.6.2p1 of OpenSMTPD which addresses the vulnerability.


It is available from our website:

https://www.opensmtpd.org/archives/opensmtpd-6.6.2p1.tar.gz
https://www.opensmtpd.org/archives/opensmtpd-6.6.2p1.sum.sig


It is also available from Github:

https://github.com/OpenSMTPD/OpenSMTPD/releases/download/6.6.2p1/opensmtpd-6.6.2p1.tar.gz
https://github.com/OpenSMTPD/OpenSMTPD/releases/download/6.6.2p1/opensmtpd-6.6.2p1.sum.sig


Or using the `6.6.2p1` tag if you're building from source.



Re: often (but not always) two envelopes per mail in queue

2020-01-27 Thread gilles
January 27, 2020 8:00 PM, "Tassilo Philipp"  wrote:

> Ok, I have a better idea now...
> 
>>> a- an envelope is created for each RCPT TO in a transaction...
>>> b- ... and additional envelopes may be created by aliases _during_ that 
>>> RCPT TO
>> 
>> Thanks a ton for those two pointers, I'll investigate and write what I > can 
>> figure out.
> 
> You correctly guessed a), turns out that my MUA (msmtp, but only when 
> triggered by mutt) somehow
> doubles all RCPT TO addresses (but not always, as said). I'm not sure at the 
> moment why, but I can
> exclude OpenSMTPd from being the culprit.
> 
> I guess this was already going on for months, but previously used MSAs 
> weren't under my control, so
> I never noticed it. Oops.
> 
> Thanks again for your precise answer which pointed me into the right 
> direction, and sorry that I
> didn't check it more thoroughly from the get go.

no problem, glad you figured



Re: often (but not always) two envelopes per mail in queue

2020-01-27 Thread gilles
January 27, 2020 11:49 AM, "Tassilo Philipp"  
wrote:

> Hello,
> 
> I noticed that for most emails I submit through my instance of OpenSMTP, 
> there are most of the time
> (but interestingly not always) two envelopes in the queue. However, mail 
> delivery works fine, and
> only one copy arrives at the recipient.
> 
> [...]
> 
> Looks like they are exact duplicates, but with their distinct envelope IDs. 
> Here's the relay log:
> 
> Jan 27 10:20:43 schacht01 smtpd[50381]: 11490b9d19a2b420 mta delivery 
> evpid=10da45aca8ae47a5
> from= to= rcpt=<-> 
> source=""
> relay="" delay=8s result="Ok" stat="250 2.6.0 
> <2020015810049931962...@snd.com>
> [InternalId=3259974] Queued mail for delivery"
> Jan 27 10:20:43 schacht01 smtpd[50381]: 11490b9d19a2b420 mta delivery 
> evpid=10da45acc22f61bd
> from= to= rcpt=<-> 
> source=""
> relay="" delay=8s result="Ok" stat="250 2.6.0 
> <2020015810049931962...@snd.com>
> [InternalId=3259974] Queued mail for delivery"
> 
> So, both are relayed, it seems, however, on the receiving side only one copy 
> arrives, and no errors
> reported. I guess that if there is a double relay, the receiver might still 
> only accept one copy,
> b/c the Message-Id being the same? Or isn't there a double relay to begin 
> with?
> 

Both were relayed, the recipient probably deduped on the Message-Id



> I'm also a bit puzzled why in rare cases, only one envelope is in the queue - 
> what I would expect -
> but I cannot find why or what the difference is.
> 
> I have only one thing listening on the submission port:
> 
> # netstat -4an | grep -F .587
> tcp4 0 0 .587 *.* LISTEN
> 
> The corresponding smtp.conf's config file line is, as well as the used 
> action/match pair:
> 
> listen on $listen_ext port submission tls-require pki mx_pki auth hostname 
> $mx_domain senders
>  mask-src tag "MSA"
> 
> action "relay" relay helo $mx_domain
> match auth from any for any action "relay"
> 
> So I guess my question is very simple in nature: Is this normal behaviour, or 
> is there something
> going on that needs to be investigated?
> 

There is something that needs to be investigated, this is not normal no.

I won't rule out a bug in OpenSMTPD but given how envelopes are created, it is 
very unlikely:

a- an envelope is created for each RCPT TO in a transaction...
b- ... and additional envelopes may be created by aliases _during_ that RCPT TO

You provided only a partial smtpd.conf so it's hard to know if b- is possible 
here,
as relaying doesn't go through aliasing which would hint at a- being the 
explanation.
smtpd.conf has to be read as a whole otherwise troubleshooting is not possible.

You can run smtpd with `-T smtp` to inspect SMTP transactions as they are 
submitted,
you can also use `-T rules` to check which rule was matched and make sure it is 
that
match auth and not another one that got hit first.



Re: filter oddities

2020-01-25 Thread gilles
January 25, 2020 9:21 PM, "Edgar Pettijohn"  wrote:

> On 01/25/20 14:20, gil...@poolp.org wrote:
> 
>> January 25, 2020 8:50 PM, "Edgar Pettijohn"  wrote:
>> 
>>> I haven't seen any mention of this, but for some reason in my limited 
>>> "testing" of filters I have
>>> to use \r\n in my responses to smtpd. Is this normal? Doesn't seem to be 
>>> documented and what
>>> filters I've looked at don't appear to be using \r\n.
>> 
>> Indeed, you must certainly NOT use "\r\n" in filters.
>> 
>>> For example without ORS = "\r\n" the following script will cause smtpd to 
>>> basically just hang.
>>> There is no errors reported, but when I attempt to telnet localhost 25 the 
>>> daemon doesn't greet me.
>>> After adding ORS = "\r\n" everything works as expected.
>> 
>> I'm not familiar with awk beyond very basic uses, could this be due to some
>> flushing not happening by default and which gets triggere with "\r\n" ?
>> 
>> This is another awk filter which doesn't use ORS:
>> 
>> https://github.com/jirutka/opensmtpd-filter-rewrite-from/blob/master/filter-rewrite-from
>> 
>> so I'm not sure why yours block but the solution is not with "\r\n" for sure
> 
> I get the same with filter-rewrite-from.

Can you run filter traces while you reproduce ?



Re: filter oddities

2020-01-25 Thread gilles
January 25, 2020 8:50 PM, "Edgar Pettijohn"  wrote:

> I haven't seen any mention of this, but for some reason in my limited 
> "testing" of filters I have
> to use \r\n in my responses to smtpd. Is this normal? Doesn't seem to be 
> documented and what
> filters I've looked at don't appear to be using \r\n.
> 

Indeed, you must certainly NOT use "\r\n" in filters.


> For example without ORS = "\r\n" the following script will cause smtpd to 
> basically just hang.
> There is no errors reported, but when I attempt to telnet localhost 25 the 
> daemon doesn't greet me.
> After adding ORS = "\r\n" everything works as expected.
> 

I'm not familiar with awk beyond very basic uses, could this be due to some
flushing not happening by default and which gets triggere with "\r\n" ?

This is another awk filter which doesn't use ORS:

https://github.com/jirutka/opensmtpd-filter-rewrite-from/blob/master/filter-rewrite-from

so I'm not sure why yours block but the solution is not with "\r\n" for sure



Re: smtpd-filters.7 patch

2020-01-25 Thread gilles
The diff reads ok but I wonder why you removed this sentence:

-No decision is ever taken by the report stream.

I think it made it a bit more clear that reporting is informative only.



Re: Skip recipient verification and forward everything to a LMTP socket

2020-01-22 Thread gilles
January 22, 2020 9:53 AM, "Éloi Rivard"  wrote:

>> What about RFC 1891? Is there an option to disable use of additional
>> parameters such as ORCPT [1] to ensure compatibility with smtp tools that 
>> does
>> not support this standard?
> 
> Actually I was misunderstanding this. There is no issue with ORCPT.
> 
>> It is inaccurate that no system user is involved here, all recipients do
>> resolve into a username because some user has to do the LMTP session. In
>> virtual setups, like yours seems to be, the proper way is to create some
>> dedicated user and map all recipients to that:
>> 
>> action sourcehut lmtp "/tmp/lists.forge.mydomain-tls-lmtp.sock" \
>> virtual { "@" = _sourcehut }
>> 
>> In cases where you have a full list of recipients and do not need to get
>> virtual mappings involved, you can do:
>> 
>> action sourcehut lmtp "/tmp/lists.forge.mydomain-tls-lmtp.sock" \
>> user _sourcehut
>> 
>> But no matter what, any action in smtpd.conf is a command that is going
>> to get executed and a process has to have a owner, so there is going to
>> be a system user involved.
> 
> Indeed, this solution seems to work:
> 
> action srht lmtp "/tmp/lists.forge.mydomain-tls-lmtp.sock" rcpt-to virtual {
> "@" = listssrht }
> match from any for any action srht
> 
> Now I encounter another issue: sourcehut mailing lists have the form "
> ~user/listn...@lists.forge.mydomain.tld" [1]. There is also a backup formatm. 
> "
> u.user.listn...@lists.forge.mydomain.tld". The backup format works fine, but 
> the
> tilde character does not seem to be handled correctly in the main format. 
> Those
> are the commands received by the lmtp client when I send a mail to
> ~user/listn...@lists.forge.mydomain.tld:
> 
> LHLO localhost
> MAIL FROM:
> RCPT TO:<:user/listn...@lists.forge.mydomain.tld>
> 
> In the "RCPT TO" command, the user has no tilde. The sourcehut developpers 
> argue
> that it is a valid character for an email adress. Would you consider 
> supporting
> tildes in OpenSMTPD?
> 

The '~' tilde character is already allowed but escaped, which is why you see 
':'.

It seems like the lmtp MDA gets the escaped version when it should get the raw 
version,
please fill a but report on Github and I'll have a look shortly.



Re: Skip recipient verification and forward everything to a LMTP socket

2020-01-20 Thread gilles
ORCPT is only emitted if peer advertises support for it:


if (s->ext & MTA_EXT_DSN) { 

  
mta_send(s, "RCPT TO:<%s>%s%s%s%s", 

  
e->dest,

  
e->dsn_notify ? " NOTIFY=" : "",

  
e->dsn_notify ? dsn_strnotify(e->dsn_notify) : "",  

  
e->dsn_orcpt ? " ORCPT=" : "",  

  
e->dsn_orcpt ? e->dsn_orcpt : "");  

  
} else  

  
mta_send(s, "RCPT TO:<%s>", e->dest);



January 20, 2020 11:33 AM, "Éloi Rivard"  wrote:

>> But no matter what, any action in smtpd.conf is a command that is going
>> to get executed and a process has to have a owner, so there is going to
>> be a system user involved.
> 
> Thank you for the explanations, this is clearer.
> 
> What about RFC 1891? Is there an option to disable use of additional 
> parameters
> such as ORCPT [1] to ensure compatibility with smtp tools that does not 
> support
> this standard?
> 
> [1] https://tools.ietf.org/html/rfc1891#section-5.2



Re: Skip recipient verification and forward everything to a LMTP socket

2020-01-18 Thread gilles
January 15, 2020 6:03 PM, "Éloi Rivard"  wrote:

> Hi,
> 
> I would like to put a OpenSMTPD server in front of a sourcehut lists
> installation [1] (that is, a mailing list system for sourcehut).
> OpenSMTPD and sourcehut communicate through a lmtp unix socket. Here is
> my configuration (without the filter and pki parts):
> 
> listen on eth0 tls pki lists.forge.mydomain.tld
> action sourcehut lmtp /tmp/lists.forge.mydomain-tld-lmtp.sock
> 
> match from any for domain "lists.forge.yaal.fr" action "sourcehut"
> 
> Now with this configuration I only get "550 Invalid recipient" errors,
> which is expected because OpenSMTPD has no way to know what is a valid
> sourcehut list recipient.
> 
> How can I make OpenSMTPD just skip the recipient verification, and just
> forward everything to the lmtp socket?
> 

There are two ways:

1- synchronize the list of recipients in a recipient table in smtpd, that
   may be less convenient because you need to have the list of recipients
   on the SMTP side AND the lmtp side, but... that's the clean way.

2- you can have a virtual mapping with a catch-all so that all recipients
   are accepted and passed to the LMTP socket, this works but is not very
   clean because it will backscatter if LMTP rejects the recipient.


> I read about userbase catchall, but my understanding is that userbases
> maps recipients to a system user, and that seems irrelevant for me as
> no system user is involved here.
> 

The userbase feature is to provide an alternate mechanism to resolve the
usernames to uid, gid and home directory. I don't think it's useful here
but your comment has hinted me at the issue:

It is inaccurate that no system user is involved here, all recipients do
resolve into a username because some user has to do the LMTP session. In
virtual setups, like yours seems to be, the proper way is to create some
dedicated user and map all recipients to that:

action sourcehut lmtp "/tmp/lists.forge.mydomain-tls-lmtp.sock" \
virtual { "@" = _sourcehut }

In cases where you have a full list of recipients and do not need to get
virtual mappings involved, you can do:

action sourcehut lmtp "/tmp/lists.forge.mydomain-tls-lmtp.sock" \
user _sourcehut

But no matter what, any action in smtpd.conf is a command that is going
to get executed and a process has to have a owner, so there is going to
be a system user involved.



Re: catch all aliases per users/aliases

2020-01-08 Thread gilles
January 8, 2020 4:11 PM, "Mathieu Roy"  
wrote:

> Hi there,
> 

Hi,


> I'd be interested to replace my current exim setup (DKIM, SPF, greylisting, 
> bogofilter,
> spamassassin, pyzor, etc) by opensmtpd.
> 
> I am using exim since decades (and am satisfied) now but opensmtpd setup 
> seems more straighforward.
> 
> Reading the manual, it seems to me that opensmptd can manage all of this, 
> except that I have
> trouble to find a way to setup custom anti-spam aliases, similar to the 
> address used to send this
> mail, composed by arbitrary strings surrounding the relevant user name, sent 
> to a dedicated
> subdomain.
> 
> With exim, it is configured as such :
> 
> shopping_catchall_p3:
> debug_print = "R: shopping_catchall for PL0$local_partPL0...@$domain"
> driver = redirect
> local_part_prefix = PL0
> local_part_suffix = PL0*
> domains = spm.rien.pl
> condition = ${if exists{/etc/aliases.d/$domain}}
> data = ${lookup{$local_part}lsearch{/etc/aliases.d/$domain}}
> file_transport = address_file
> 
> Basically, it looks for the said arbitrary strings as local_part_prefix and 
> local_part_suffix and
> that is enough.
> 
> Is there any way to do something like this with opensnmtpd ?
> 

Not with the base daemon as it does little more than SMTP.

Implementing the feature in a filter is easy but no such filter exists as of 
today.



Re: Sendmail reporting 421 4.3.0 Temporary Error

2020-01-06 Thread gilles
for the record, solved off-list on IRC:

permission on directories within the queue had been altered.


January 6, 2020 2:19 AM, "jrmu"  wrote:

> Greetings,
> 
> I am running OpenBSD 6.6 GENERIC#3 amd64 and getting an
> inexplicable 421 4.3.0 Temporary Error when using 
> sendmail with opensmtpd.
> 
> Here is my current mail setup:
> 
> Inside bnc3.ircnow.org, I have /etc/mail/smtpd.conf:
> 
> table aliases file:/etc/mail/aliases
> table secrets file:/etc/mail/secrets
> 
> listen on lo0
> 
> action "local_mail" mbox alias 
> action "outbound" relay host smtp+tls://supp...@ircnow.org:587 \
> auth 
> 
> match for local action "local_mail"
> match for any action "outbound"
> 
> Inside /etc/mail/secrets:
> 
> support supp...@ircnow.org:PASSWORD
> 
> I am using ircnow.org as a mail relay. Here is its 
> /etc/mail/smtpd.conf:
> 
> pki mail.ircnow.org cert "/etc/ssl/ircnow.org.fullchain.pem"
> pki mail.ircnow.org key "/etc/ssl/private/ircnow.org.key"
> 
> # tables setup
> table aliases file:/etc/mail/aliases
> table domains file:/etc/mail/domains
> table passwd passwd:/etc/mail/passwd
> table virtuals file:/etc/mail/virtuals
> table hosts file:/etc/mail/hosts
> 
> listen on lo0 mask-src
> listen on lo0 port 10028 tag DKIM mask-src
> listen on egress port 25 tls pki mail.ircnow.org mask-src
> listen on egress port 587 tls-require pki mail.ircnow.org auth  
> mask-src 
> action "lmtp" lmtp "/var/dovecot/lmtp" rcpt-to virtual 
> action "relay" relay
> action "relay_dkim" relay host smtp://127.0.0.1:10027
> 
> match from any for domain  action "lmtp"
> match tag DKIM for any action "relay"
> match from src  for any action "relay_dkim"
> match auth from any for any action "relay_dkim"
> 
> On bnc3.ircnow.org, I run this command:
> 
> $ /usr/sbin/sendmail -tv -F support -f support < samplemail
> 
> samplemail contains this message:
> 
> From: support  
> To: u...@example.com 
> Subject: Welcome to IRCNow! 
> MIME-Version: 1.0 
> Content-Type: text/plain; charset=us-ascii
> Content-Disposition: inline 
> 
> Welcome to IRCNow!
> 
> When running the above command, I get this output:
> 
> <<< 220 bnc3.ircnow.org ESMTP OpenSMTPD
> 
>> EHLO localhost
> 
> <<< 250-bnc3.ircnow.org Hello localhost [local], pleased to meet you
> <<< 250-8BITMIME
> <<< 250-ENHANCEDSTATUSCODES
> <<< 250-SIZE 36700160
> <<< 250 HELP
>> MAIL FROM:
> 
> <<< 421 4.3.0 Temporary Error
> sendmail: command failed: 421 4.3.0 Temporary Error
> 
> On bnc3.ircnow.org, I see this in /var/log/maillog:
> 
> Jan 3 08:26:27 bnc3 smtpd[]: smtp disconnected reason=quit
> Jan 3 08:38:36 bnc3 smtpd[]: smtp connected address=local host=bnc3.ircnow.org
> Jan 3 08:38:36 bnc3 smtpd[]: smtp failed-command command="MAIL 
> FROM: "
> result="421 4.3.0 Temporary Error"
> Jan 3 08:38:36 bnc3 smtpd[]: smtp disconnected reason=quit
> 
> The mail relay (ircnow.org) never even sees the email, since
> bnc3.ircnow.org is not sending it. Instead, the mail is discarded.
> 
> I then tried running $ nc localhost 25:
> 
> 220 bnc3.ircnow.org ESMTP OpenSMTPD
> ehlo bnc3
> 250-bnc3.ircnow.org Hello bnc3 [127.0.0.1], pleased to meet you
> 250-8BITMIME
> 250-ENHANCEDSTATUSCODES
> 250-SIZE 36700160
> 250-DSN
> 250 HELP
> mail from: 
> 421 4.3.0 Temporary Error
> 
> About 50-70% of the time, I get the above message. Could opensmtpd
> be greylisting sendmail from localhost?
> 
> I tried enabling debugging:
> 
> # smtpd -dv
> 
> Then, I attempt to use sendmail:
> 
> $ /usr/sbin/sendmail -tv -F support -f support < samplemail
> 
> mta delivery evpid= from= to= 
> rcpt=<->
> source="209.141.50.204" relay="209.141.46.110 (mail.ircnow.org)" delay=0s 
> result="Ok" stat="250
> 2.0.0 Message accepted for delivery"
> debug: mta: : no task for relay 
> [relay:ircnow.org,port=587,smtp+tls,auth=secrets:support,mx]
> mta: debug: last connection: hanging on for 10s
> debug: mta: flush for (-> u...@example.com)
> debug: mta: ... timeout for 
> [relay:ircnow.org,port=587,smtp+tls,auth=secrets:support,mx]
> debug: mta: draining 
> [relay:ircnow.org,port=587,smtp+tls,auth=secrets:support,mx] refcount=2,
> ntask=0, nconnector=1, nconn=1
> debug: mta: all done for 
> [relay:ircnow.org,port=587,smtp+tls,auth=secrets:support,mx]
> smtp connected address=local host=bnc3.ircnow.org smtp failed-command 
> command="MAIL
> FROM: " result="421 4.3.0 Temporary Error"
> smtp disconnected reason=quit
> debug: control -> client: pipe closed
> debug: clearing p=client, fd=11, pid=0
> mta: timeout for session hangon
> 
> I tried adding tracing:
> 
> # smtpd -dv -T all
> 
> Then calling:
> 
> $ /usr/sbin/sendmail -tv -F support -f support < samplemail
> 
> mproc: lka -> pony: enabled
> debug: smtpd: scanning offline queue...
> debug: smtpd: offline scanning done
> mproc: control -> client-proc: enabled
> ramstat: increment: control.session
> ramstat: control.session (0x): 0 -> 1
> mproc: control -> pony : 4 IMSG_CTL_SMTP_SESSION
> imsg: pony <- control: IMSG_CTL_SMTP_SESSION (len=4)
> smtp: 0x: connected to listener 0x 

Re: Questions About Filters

2020-01-03 Thread gilles
January 4, 2020 12:25 AM, "Antonino Sidoti"  wrote:

> Hello,
> 

Hello,


> I have some basic questions about filters?
> 
> What do we need to negate the rdns for the following command?
> 
> filter f01 phase connect match !rdns disconnect "550 missing rDNS”
> 

I'm unsure I understand this question, the example you show negates rdns,
this is what I use myself to junk incoming sessions without rdns.


> Can someone please explain the difference between reject and disconnect when 
> used in a filter?
> 

Very simple.

When you use `reject` the command is rejected but the session isn't 
disconnected.
If a client had multiple mails for you, rejecting a mail can allow it to submit 
a
different mail before it gets disconnected.

When you use `disconnect` the client gets disconnected after the rejection, so 
it
has to connect again.


> Many thanks
> 
> Nino



Re: fix build on netbsd

2019-12-04 Thread Gilles Chehade
On Wed, Dec 04, 2019 at 07:27:07PM -0600, Edgar Pettijohn wrote:
> diff --git a/openbsd-compat/openbsd-compat.h
> b/openbsd-compat/openbsd-compat.h
> index 6c73e5b5..c7af0135 100644
> --- a/openbsd-compat/openbsd-compat.h
> +++ b/openbsd-compat/openbsd-compat.h
> @@ -122,7 +122,7 @@ int getpeereid(int , uid_t *, gid_t *);
> ??unsigned int arc4random(void);
> ??#endif
> 
> -#if defined(HAVE_ARC4RANDOM_STIR)
> +#if !defined(HAVE_ARC4RANDOM_STIR)
> ??void arc4random_stir(void);
> ??#elif defined(HAVE_ARC4RANDOM) || defined(LIBRESSL_VERSION_NUMBER)
> ??/* Recent system/libressl implementation; no need for explicit stir */
> 

with this, your build is fixed ?

I've been fixing the .c part of openbsd-compat but haven't worked on the
.h part yet, will have  look at it



-- 
Gilles Chehade @poolpOrg

https://www.poolp.org    patreon: https://www.patreon.com/gilles



CVE-2019-19521 what about OpenSMTPD ?

2019-12-04 Thread Gilles Chehade
Hello,

In case you haven't seen, multiple CVE were released by Qualys:

https://www.openwall.com/lists/oss-security/2019/12/04/5

CVE-2019-19521 refers to an Authentication bypass allowing remote people
to authenticate to an OpenSMTPD without credentials.

A few people were wondering why we didn't publish a patch so here is the
explanation to clarify a bit.

TL;DR:
- if you're not on OpenBSD, you can disregard, you're not affected
- if you're on OpenBSD, run `syspatch` and, once done, restart smtpd, it
  is _normal_ that you don't see an smtpd patch


Details:

The CVE show-cases a vulnerability using smtpd, ldapd, radiusd, sshd and
su but the issue is really in a libc API they use: bsd_auth(3). There is
an incorrect code pattern which is coupled with an insufficient check to
the username, and this allows the authentication bypass that is shown on
multiple consumers.

So should you worry ?

If you're not using OpenBSD you can disregard this advisory, bsd_auth(3)
doesn't exist elsewhere.

If you're using OpenBSD, RUN `syspatch` RIGHT AWAY, then restart daemons
which perform user authentication. The issue being in the libc, you will
not see a patch for smtpd, it is normal, you still have to restart it so
it catches up the libc update.

If you're using an OpenBSD that's no longer supported (<=6.4) you're now
at risk and need to upgrade or disable network daemons that do auth.


Could your OpenSMTPD be used to send spam ?

If you're not using OpenBSD, nope.

If you're using OpenBSD, it's technically possible but unlikely. You can
check by going through your logs and looking for user "-schallenge". The
bypass only makes sense for setups that expose auth and provide rules to
match auth users.


If you have questions, you can follow up to this mail,
Cheers,


-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: Exploit CVE-2019-19521?

2019-12-04 Thread Gilles Chehade
On Wed, Dec 04, 2019 at 11:08:44PM +0100, Henry Jensen wrote:
> Hi,
> 

Hi,


> from https://seclists.org/oss-sec/2019/q4/120
> 
> ==
> 1.2. Case study: smtpd
> ==
> 
> To demonstrate how smtpd's authentication can be bypassed, we follow the
> instructions from the manual page of smtpd.conf:
> 
> [...]
>
> I did verify, that this attack worked on my unpatched OpenBSD 6.6 Box.
> But I didn't get much further. After the authentication succeeded
> I continued with MAIL FROM: and RCPT TO: After the RCPT TO: the
> connection was aborted. After I patched my system I could no longer get
> a 235 2.0.0 Authentication succeeded message
> 
> Question is: would it have been possible in the "real world" to exploit
> this to relay arbitrary messages (e.g. spam)?
> 

Yes it would have been most definitely possible now if you have yourself
relayed spam, I'll tell you that it's very unlikely this was used.

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: Virtual domains & Virtual Users...

2019-11-23 Thread gilles
November 23, 2019 5:08 PM, "Implausibility"  wrote:

> Hi again.
> 
> My mail server has been running fine since last weekend, and I'm trying to 
> expand its functionality
> by including the ability to send and receive mail for my list of domains, and 
> for eMail addresses
> which forward to locally defined users -- but I can't seem to get it working, 
> and I think the issue
> is my (mis-)understanding of how the match parameter works...
> 
> In order to get virtual users working, I've added three lines to the config:
> 
>> table vusers file:/etc/mail/vusers
>> action "vusers" maildir junk virtual 
>> match from any for domain  rcpt-to virtual  action "vusers"

your match rule is not correct, I'm not sure what you want to do:

- rcpt-to lacks a table parameter, but I'm unsure if it's even needed here
- virtual  can't be in the match rule, it must be in your action, which 
is already the case


> I was able to get mail delivered for local users to my virtual domains 
> previously without issue.
> But I can't get virtual addresses working...
> 
> I've tried a half a dozen varieties of the match command, and I keep getting 
> 'syntax error', and it
> doesn't give me any hint as to what exactly the problem is. I want to accept 
> eMail from any
> destination, to the domains defined in the domains table, that are sending to 
> recipients listed in
> the vusers table, to deliver them to the maildir for access via Dovecot IMAP.
>



portable layer rework

2019-11-19 Thread gilles
Hello,

OpenSMTPD portable was first released many many years ago thanks to the work of 
Charles Longeau.
IIRC this was his first OpenBSD portable project and he bootstrapped it from 
OpenSSH.
It served us well and opened us to the Linux world but it also came with its 
share of errors inherent to a new project.

Unless some of you are interested, I won't enter the details, but the basic 
idea is that the portable version has a library which contains the bits that 
were detected as missing/broken on a target system and, because of it's done, 
it may contain a bit more than we think it does. In most cases, this isn't a 
problem because if OpenSMTPD uses the portable version of strwhatever() instead 
of libc's, it isn't really a big deal.


There are two cases however that bit us:

The first one was the "OpenSMTPD portable on OpenBSD" case. For years, I kept 
repeating that running OpenSMTPD portable on OpenBSD was not supported and that 
this would lead to unexpected issues. This became a reality at some point when 
building portable on OpenBSD resulted in an executable that would crash at 
startup despite being fundamentally the same code as the master branch which 
would work just fine. It turned out that this was because, despite being on 
OpenBSD, some bits from the compat layer were still built and these bits not 
being the system ones, they didn't comply with our pledge().

The second one came shortly before 6.6.0p1 as people started reporting crashes 
on some systems and the issue was traced back to an incorrect inclusion of 
arc4random() related functions. If a target system was using LibreSSL, which 
brings arc4random(), we would somehow end up using the LibreSSL one but use our 
own entropy stirring function and blow up in pieces.


There are other downsides to the way the portable layer works and while we had 
work-arounds so people wouldn't notice on systems we were actively supporting 
(Linux, FreeBSD, ...), it is less than perfect for developers willing to bring 
new targets. Freddy Dissaux who worked on bringing OpenSMTPD to Solaris hit 
several of these downsides.


Anyways, I have started reworking the layer so that it conditionally builds 
what it needs.
The compat layer on OpenBSD is now empty as it should and the resulting 
executable runs reliably as it should too.
There's still work required because code set aside, we also have headers issues 
(for which we have a workaround).


I REALLY need feedback from portable users running the development branch.

Ihor Antonov has setup a CI that lets us spot failures to build on Linux 
glibc/musl, however:

- we don't have CI for FreeBSD, NetBSD, DFlyBSD, OSX, Solaris
- CI won't spot either one of the two issues I solved above as they occur at 
startup / runtime


You can help us make the portable branch really good in a very short time and 
on the long term by testing and reporting regularly that the portable branch 
still works for you.


Gilles



Re: Primary Domains vs. Virtual Domains - what's the difference?

2019-11-18 Thread gilles
November 18, 2019 4:14 PM, "Charles Collicutt"  wrote:

> On 18 Nov 2019, at 13:07, gil...@poolp.org wrote:
> 
>> With a virtual domain, OpenSMTPD assumes that domain.org == the content of 
>> the virtual table.
>> The virtual mechanism is not optional, the recipient MUST exist in the table 
>> to be valid.
> 
> Can virtual users be used with sub-addresses, e.g user+...@virtualdomain.org ?
> 
> When I tried that (some time ago) it failed saying that the user did not 
> exist.
> 

It has worked for years so without more information I can't help with why it 
failed

Gilles



Re: Primary Domains vs. Virtual Domains - what's the difference?

2019-11-18 Thread gilles
November 17, 2019 5:19 PM, "Implausibility"  wrote:

> I'm reading the man pages for makemap, and there are two types of database 
> maps described, as per
> $subject.
> 
> What are the functional / operational differences between Primary & Virtual 
> Domains? When when I
> choose one over the other? Can I get examples of when I'd choose a table of 
> primary domains, and
> when I'd choose a table of virtual domains?
> 
> Which one should I choose if I want to send *and* receive mail from domains 
> that are not the same
> as my mail server's name? (e.g... my business has many websites... 
> example.com, example.us,
> example.io - and I'd like to send and receive mail for each of them 
> separately... say, to route the
> eMail to the sales rep for a specific territory)
> 
> Thanks.


I'll try to summarise it as simple as possible:

During the SMTP transaction, an e-mail address is provided for the recipient:

u...@domain.org

With a primary domain, OpenSMTPD assumes that domain.org == the local machine.
It will look for a system account matching the user and if one is found, then 
mail is delivered.
The aliases mechanism is optional, if an alias exists a transform happens, 
otherwise the username is looked up as is.

With a virtual domain, OpenSMTPD assumes that domain.org == the content of the 
virtual table.
The virtual mechanism is not optional, the recipient MUST exist in the table to 
be valid.


If you use a primary domain, when you create a new system user you can mail 
them right away.
If you use a virtual domain, you'll have to declare that user in the table.



Re: opensmtpd setresgid ubuntu crash

2019-11-16 Thread Gilles Chehade
On Fri, Nov 15, 2019 at 12:03:01PM +0100, Martijn van Duren wrote:
> That seems to do the trick. Thanks.
> Sorry for the noise.
> 

I have traced back the issue to a pasto in configure.ac which caused the
setresuid.c file to be included on systems with setresuid() and this has
bad side-effects because the openbsd-compat setresuid() function will do
some funny things.

The portable branch should work again for all.

Writing this from a Debian/arm64 with a working smtpd

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: opensmtpd setresgid ubuntu crash

2019-11-15 Thread Gilles Chehade
Try using the 6.6.1p1 tag, I'm currently reworking the dev branch to completely revamp compat layer, things will be shaky for the next few daysOn Nov 15, 2019 11:22, Martijn van Duren  wrote:EHLO,

I'm currently trying to port filter-dnsbl to ubuntu, but I'm stuck at
not being able to startup smtpd. Is there anyone who has seen this
before and who has a (possible) solution?

This all is freshly installed.

OS: Ubuntu 18.04.3 LTS
OpenSMTPD: git portable (latest)
Installed packages:
- build-essential
- autoconf
- libtool
- libssl-dev
- libz-dev
- bison
- libasr-dev
- gdb
configure parameters: none
backtrace:
#0  setresgid (rgid=rgid@entry=1001, egid=1001, egid@entry=, sgid=1001, sgid@entry=) at setresgid.c:29
#1  0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#2  0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#3  0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#4  0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#5  0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#6  0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#7  0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#8  0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#9  0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#10 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#11 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#12 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#13 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#14 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#15 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#16 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#17 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#18 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#19 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#20 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#21 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#22 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#23 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#24 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#25 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#26 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#27 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#28 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#29 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#30 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#31 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#32 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#33 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#34 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29
#35 0x55c9b388d0c8 in setresgid (rgid=rgid@entry=1001, egid=, sgid=) at setresgid.c:29


martijn@




Re: 6.6.1p1 fails to build on Void Linux

2019-11-11 Thread Gilles Chehade
On Mon, Nov 11, 2019 at 08:10:50PM -0600, epektasis wrote:
> Thank you for your reply.  Libevent-2.1.11_1 is installed.  So is
> autoconf-2.69_7, automake-1.16.1_2, bison-3.4.2_1, libtool-2.4.6_4, and
> libasr-1.0.3_1.  There are several fatal errors for some missing header
> files; I guess I'll try to track them down and see if I can get this
> going again.  I'll let you know.
> 

In some distributions, packages are split between two, so you have for
example libevent and libevent-dev, the former for runtime dependencies
and the second for build time dependencies with headers and such. This
may be the case here ?

I'm on my openbsd laptop right now, as soon as I boot on a Linux one I
will try to build on void linux and get back to you, cheers.

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: 6.6.1p1 fails to build on Void Linux

2019-11-11 Thread Gilles Chehade
Very likely libevent is missing...

I fixed a configure bug where missing libevent caused a SSL test to fail and 
thus lead to a confusing error.



> On 12 Nov 2019, at 00:28, epektasis  wrote:
> 
> I'm trying to build opensmtpd-6.6.1p1 on an updated Void Linux.  Already
> overcame a couple of missing development packages by installing them.
> But I can't seem to get by this error, which occurs in the configure
> process just after opensslv.h is found (configure exit status 1):
> 
>configure error: error: *** Can't find recent LibreSSL crypto
>(see config.log for details)
> 
> LibreSSL 3.0.2_1 is installed.  So is libcrypto44 and 45 (for the
> latter, both 64 and 32 bit versions).  I read the README file and made
> sure the stated dependencies are installed.  Nothing in the config.log
> jumps out at me; I'll attach it nevertheless.
> 
> -- 
> 




New portable release 6.6.1p1 released yesterday

2019-11-06 Thread gilles
Hello misc@,

I have released OpenSMTPD 6.6.1p1 yesterday.

The release is very close to 6.6.0p1, it however fixes all known portable 
issues,
ranging from OpenSSL vs LibreSSL build or runtime errors to Glibc vs Musl 
errors.

If you were using an older version of OpenSMTPD because your disto did not ship 
a
recent version due to portability issues, this should no longer be the case.

It should be technically possible to build 6.6.1p1 no matter if you're on BSD, 
on
OSX, on Linux, with an FreeBSD libc, a Glibc, a musl, etc...

If you are a package maintainer and need help, get in touch.

The following page lists package versions:

https://repology.org/project/opensmtpd/versions


We hope to see 6.6.1p1 everywhere :-p



Re: builtin filter regex question

2019-10-31 Thread gilles
October 24, 2019 8:35 PM, "Joerg Jung"  wrote:

> Hi,
> 
> I used some regex filters in the past which I'm trying to convert to the
> latest builtin filters. In particular, I stumbled over a HELO filter,
> which rejects non-FQDN HELO forcing SMTP protocol, aka: 
> Sendmail FEATURE(block_bad_helo) or Postfix reject_non_fqdn_helo_hostname
> 
> I had significant success rate with this kind of blocking, since a good
> portions of spammers seem to be too lazy to configure HELO correctly.
> 
> Here is what I came up with:
> 
> # reject HELO/EHLO with leading or trailing dot, and without dots (non-FQDN)
> filter helo phase helo connect match helo regex { "^\.", "\.$", "^[^\.]*$" } 
> disconnect "554 5.7.1
> HELO rejected" 
> filter ehlo phase ehlo connect match helo regex { "^\.", "\.$", "^[^\.]*$" } 
> disconnect "554 5.7.1
> EHLO rejected
> 
> Now, I just need a way to skip/allow IPv6 address literals, e.g. there
> are no dots in EHLO [::1], but still a valid/allowed value.
> With old filter-regex I just did a negotiation: ! regex "^\[" to
> not apply filter to v6 literals
> 
> Any ideas/hints how to add/implement this with the new builtin regex
> filter syntax?
> 

Sadly there would have been a very easy way if I had that use-case in mind 
pre-release,
which would be to make the "proceed" action explicit, you could have had a 
filter
match the inet6 address and proceed to shortcut the matching of non fqdn.

As of today, there will be no option but to craft your regex to contain both 
the pattern
you want to match AND exclude [ as far as I see it.



Re: Mailing list expansion problem.

2019-10-31 Thread gilles
October 26, 2019 1:23 PM, "Reio Remma"  wrote:

> On 26/10/2019 14:18, Reio Remma wrote:
> 
>> On the subject of catch all aliases, I tried adding one to my setup > with 
>> odd results.
>> 
>> My usual setup with virtual users:
>> 
>> action deliver_lmtp lmtp "/var/run/dovecot/lmtp" rcpt-to virtual > 
>>  userbase 
>> 
>> match from any for domain  rcpt-to  action > 
>> deliver_lmtp
>> 
>> To get catch all working, I had to remove rcpt-to  from > the 
>> match:
>> 
>> match from any for domain  action deliver_lmtp
>> 
>> Otherwise the response was: 550 Invalid recipient: 
>> 
>> One I removed the actual catch all alias and sent mail to a > non-existent 
>> account, the usual:
>> 
>> 550 Invalid recipient: 
>> 
>> turned into:
>> 
>> 524 5.2.4 Mailing list expansion problem: 
>> 
>> Any ideas how I could use a catch all alias _and_ get a proper 550 > Invalid 
>> recipient if I don't?
> 
> This is all with the current (v6.6) portable from a week ago or so.


This error occurs when aliases expansion encounters an error during its 
processing,
there's not enough info here to understand what happened in the expansion loop:

- it is likely a table content issue either in virtuals or in userinfo table or 
both
- using `smtpd -dv -T expand` will help you understand what went wrong during 
expansion

Gilles



Re: Accept mail for all recipients

2019-10-31 Thread gilles
catch all are a virtual thing, they don't work for aliases

October 26, 2019 12:51 PM, "Reio Remma" mailto:r...@mrstuudio.ee?to=%22Reio%20Remma%22%20)> wrote:
On 26/10/2019 13:35, Sergey Seacher wrote: Hello!

How can I make, opensmtpd accept mail for all recipients: that are present in 
the file /etc/opensmtpd/aliases and that are not present?
I had the rule in my /etc/opensmtpd/smtpd.conf file:accept 
from any 
for domain domain.ltd alias  
deliver to lmtp "/run/dovecot/lmtp" rcpt-to I have changed this rule to:accept 
from any 
for domain domain.ltd 
deliver to lmtp "/run/dovecot/lmtp" rcpt-to Now, if I send to any recipient in 
my domain, for example rggg...@domain.ltd (mailto:rggg...@domain.ltd), I 
receive error 550, but I need mail to be deliver to i...@domain.ltd 
(mailto:i...@domain.ltd)  
Do you mean a catch all alias? Try adding to your aliases file:

@domain.ltd i...@domain.ltd (mailto:i...@domain.ltd)

And re-add alias  to your accept rule.

Good luck,
Reio


Announce: OpenSMTPD 6.6.0 released

2019-10-26 Thread Gilles Chehade
OpenSMTPD 6.6.0 has just been released.

OpenSMTPD is a FREE implementation of the SMTP protocol with some common
extensions. It allows ordinary machines to exchange e-mails with systems
speaking the SMTP protocol. It implements a fairly large part of RFC5321
and can already cover a large range of use-cases.

It runs on OpenBSD, NetBSD, FreeBSD, DragonFlyBSD and glibc-based Linux.

The archives are now available from the main site at www.OpenSMTPD.org

We would like to thank the OpenSMTPD community for their help in testing
the snapshots, reporting bugs, contributing code and packaging for other
systems.

This is a major release with multiple bug fixes and new features.


Dependencies note:
==

This release builds with LibreSSL > 3.0.2 or OpenSSL > 1.1.0.

It's preferable to depend on LibreSSL as OpenSMTPD is written and tested
with that dependency. In addition, the features parity is not respected,
some features will not be available with OpenSSL, like ECDSA server-side
certificates support in this release. OpenSSL library is considered as a
best effort target TLS library and provided as a commodity, LibreSSL has
become our target TLS library.


Changes in this release (since 6.4.0):
==

- various improvements to documentation and code
- reverse dns session matching criteria added to smtpd.conf(5)
- regex table lookup support added to smtpd.conf(5)
- introduced support for ECDSA certificates with an ECDSA privsep engine
- introduced builtin filters for basic filtering of incoming sessions
- introduced option to deliver junk to a Junk folder in mail.maildir(8)
- fixed the smtp(1) client so it uses correct default port for SMTPS
- fixed an smtpd(8) crash on excessively large input
- ensured mail rejected by an LMTP server stay queued


Experimental features:

- introduced a filters API to allow writing standalone filters for smtpd
- introduced proxy-v2 support allowing smtpd to operate behind a proxy


Checksums:
==

  SHA256 (opensmtpd-6.6.0.tar.gz) =
  fcf4496493d211c7024798b8107194ff6f2469b143b232f8559d36ce98d5d728

  SHA256 (opensmtpd-6.6.0p1.tar.gz) =
  75a420941963a672b21fe6c820c51de07f1ac94a0d6d4aa4f7364124d85efce9


Verify:
===

Starting with version 5.7.1, releases are signed with signify(1).

You can obtain the public key from our website, check with our community
that it has not been altered on its way to your machine.

   $ wget https://www.opensmtpd.org/archives/opensmtpd-20181026.pub

Once you are confident the key is correct, you can verify the release as
described below:

1- download both release tarball and matching signature file to same directory:

   for OpenBSD version:
   $ wget https://www.opensmtpd.org/archives/opensmtpd-6.6.0.sum.sig
   $ wget https://www.opensmtpd.org/archives/opensmtpd-6.6.0.tar.gz

   for portable version:
   $ wget https://www.opensmtpd.org/archives/opensmtpd-6.6.0p1.sum.sig
   $ wget https://www.opensmtpd.org/archives/opensmtpd-6.6.0p1.tar.gz


2- use `signify` to verify that signature file is properly signed and that the
   checksum matches the release tarball you downloaded:

   for OpenBSD version:
   $ signify -C -e -p opensmtpd-20181026.pub -x opensmtpd-6.6.0.sum.sig
   Signature Verified
   opensmtpd-6.6.0.tar.gz: OK

   for portable version:
   $ signify -C -e -p opensmtpd-20181026.pub -x opensmtpd-6.6.0p1.sum.sig
   Signature Verified
   opensmtpd-6.6.0p1.tar.gz: OK


If you don't get an OK message, then something is not right and you should not
install without first understanding why it failed.


Support:


You are encouraged to register to our general purpose mailing-list:
http://www.opensmtpd.org/list.html

The "Official" IRC channel for the project is at:
#OpenSMTPD @ irc.freenode.net


Support us:


The project is maintained by volunteers, you can support us by:

- donating time to help test development branch during development cycle
- donating money to either one of the OpenBSD or OpenSMTPD project
- sponsoring developers through direct donations or patreon
- sponsoring developers through contracts to write features

Get in touch with us by e-mail or on IRC for more informations.


Reporting Bugs:
===

Please read http://www.opensmtpd.org/report.html
Security bugs should be reported directly to secur...@opensmtpd.org
Other bugs may be reported to b...@opensmtpd.org


-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: Portable buildung issues

2019-10-22 Thread gilles
For what it's worth, it turns out LibreSSL 3.0.2 was released three days ago.

We have made it possible to build with OpenSSL but our target is still LibreSSL,
which means that code is written assuming LibreSSL and we test mostly with 
LibreSSL.

Unsurprisingly we recommend that package maintainers have OpenSMTPD depend upon 
LibreSSL,
and only fallback to OpenSSL if not possible.

Some features may be lacking when doing that fallback,
for instance ECDSA server certificates will only work if linked against 
LibreSSL.



October 22, 2019 7:01 PM, "Ede Wolf"  wrote:

> Thanks for the heads up. Last time I have been using libressl 2.9.2, I'll 
> give 3.0.1 a go, it
> happens to be in the repos as well, just not marked as stable.
> 
> Ede
> 
> Am 22.10.19 um 16:28 schrieb Gilles Chehade:
> 
>> Sorry, will expand:
>> We're in between two LibreSSL releases which is why the LibreSSL you're > 
>> using is not compatible.
>> When LibreSSL 3.0.2 is released, it will automagically build with it. > 
>> LibreSSL 3.0.1 development
>> version is already working.
>> LibreSSL remains our target for both OpenBSD and portable but we're kind > 
>> of in a time warp right
>> now in between two versions.
>> On Tue, Oct 22, 2019, 16:23 Gilles Chehade  
>> >
>> wrote:
>> LibreSSL is supported and recommended, this really needs to be fixed
>> before the 6.6.0p1 portable release.
>> On Tue, Oct 22, 2019, 14:44 John Smith > > wrote:
>> Hello,
>> thanks very much for all your replies. In deed, I rebuild world
>> replacing openssl with libressl, basically only for opensmtpd.
>> So it is the github issue. I knew smtpd portable supports
>> openssl, but it did not come to my mind, that libressl is not
>> supported at all for the portable version, openssl being just
>> being an extra service, so I thought I'll do it something good.
>> I'll revert to openssl and report back. May take a day or two.
>> Thanks again
>> Ede



Re: Portable buildung issues

2019-10-22 Thread Gilles Chehade
Sorry, will expand:

We're in between two LibreSSL releases which is why the LibreSSL you're
using is not compatible.

When LibreSSL 3.0.2 is released, it will automagically build with it.
LibreSSL 3.0.1 development version is already working.

LibreSSL remains our target for both OpenBSD and portable but we're kind of
in a time warp right now in between two versions.

On Tue, Oct 22, 2019, 16:23 Gilles Chehade  wrote:

> LibreSSL is supported and recommended, this really needs to be fixed
> before the 6.6.0p1 portable release.
>
> On Tue, Oct 22, 2019, 14:44 John Smith  wrote:
>
>> Hello,
>>
>> thanks very much for all your replies. In deed, I rebuild world replacing
>> openssl with libressl, basically only for opensmtpd. So it is the github
>> issue. I knew smtpd portable supports openssl, but it did not come to my
>> mind, that libressl is not supported at all for the portable version,
>> openssl being just being an extra service, so I thought I'll do it
>> something good.
>>
>> I'll revert to openssl and report back. May take a day or two.
>>
>> Thanks again
>>
>> Ede
>>
>>
>>


Re: Portable buildung issues

2019-10-22 Thread Gilles Chehade
LibreSSL is supported and recommended, this really needs to be fixed before
the 6.6.0p1 portable release.

On Tue, Oct 22, 2019, 14:44 John Smith  wrote:

> Hello,
>
> thanks very much for all your replies. In deed, I rebuild world replacing
> openssl with libressl, basically only for opensmtpd. So it is the github
> issue. I knew smtpd portable supports openssl, but it did not come to my
> mind, that libressl is not supported at all for the portable version,
> openssl being just being an extra service, so I thought I'll do it
> something good.
>
> I'll revert to openssl and report back. May take a day or two.
>
> Thanks again
>
> Ede
>
>
>


Re: Portable buildung issues

2019-10-22 Thread gilles
we really really really need more details, I have no idea what system that is 
:-)

October 22, 2019 1:38 PM, "John Smith"  wrote:

> Hello,
> 
> cloned today, I am having problems building smtpd. After configure:
> 
> /data/git/opensmtp # make
> make all-recursive
> make[1]: Entering directory '/data/git/opensmtp'
> Making all in openbsd-compat
> make[2]: Entering directory '/data/git/opensmtp/openbsd-compat'
> gcc -DHAVE_CONFIG_H -I. -I.. -I../smtpd -I../openbsd-compat 
> -I../openbsd-compat/err_h
> -I/usr/include -march=skylake -fomit-frame-pointer -O2 -pipe -fPIC -DPIC 
> -Wall -Wpointer-arith
> -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess 
> -Wno-pointer-sign
> -Wno-unused-result -fno-strict-aliasing -fno-builtin-memset -fPIE 
> -D_BSD_SOURCE -D_DEFAULT_SOURCE
> -c -o arc4random.o arc4random.c
> arc4random.c:167:21: error: macro "arc4random_stir" passed 1 arguments, but 
> takes just 0
> arc4random_stir(void)
> ^
> arc4random.c:168:1: error: expected '=', ',', ';', 'asm' or '__attribute__' 
> before '{' token
> {
> ^
> make[2]: *** [Makefile:445: arc4random.o] Error 1
> make[2]: Leaving directory '/data/git/opensmtp/openbsd-compat'
> make[1]: *** [Makefile:418: all-recursive] Error 1
> make[1]: Leaving directory '/data/git/opensmtp'
> make: *** [Makefile:350: all] Error 2
> 
> Any idea what I might be missing? As I have a rather minimal system, some 
> package my be lacking.
> Any further details that are needed?
> 
> Thanks
> 
> Ede



Re: upcoming 6.6.0 release

2019-10-17 Thread Gilles Chehade
On Thu, Oct 17, 2019 at 03:07:45PM +0300, Reio Remma wrote:
> On 17/10/2019 15:06, Gilles Chehade wrote:
> > On Thu, Oct 17, 2019 at 02:54:26PM +0300, Reio Remma wrote:
> > > On 17/10/2019 14:20, gil...@poolp.org wrote:
> > > > Hello misc@,
> > > > 
> > > > As some have noticed, the 6.6.0 tag was created on Github to match the 
> > > > code from smtpd in OpenBSD 6.6.
> > > > 
> > > > A portable branch, branch-6.6.0p1, has been forked from there and can 
> > > > be used to test the matching portable version:
> > > > 
> > > >   https://github.com/OpenSMTPD/OpenSMTPD/tree/branch-6.6.0p1
> > > > 
> > > > 
> > > > I have not tagged 6.6.0p1 yet because there's still some time and I 
> > > > want to gain confidence that it works for most systems and 
> > > > distributions we have supported so far.
> > > > 
> > > > Note that:
> > > > 
> > > > - this release will depend on either LibreSSL 3.0.x or OpenSSL 1.1.x
> > > > - musl-based distros may have issues at this point, they are being 
> > > > tracked down, musl is not a showstopper for me as we have had issues in 
> > > > previous releases too but if we can track down the problem I'd be happy 
> > > > (unsuccessful so far)
> > > > 
> > > > I _really_ need help on testing this as I had unexpected hand surgery 
> > > > and doing the tests myself is extremely long.
> > > > 
> > > > Please report as a follow up to this mail what you did test,
> > > > 
> > > > 
> > > Hello! Do you have any pointers as to what an SRS key should look like?
> > > 
> > just make it something hard to guess :-)
> > 
> 
> So it's just a few random letters? Reading "key" I always think of some kind
> of a hash. :)
> 

the key you set is rehashed so make it whatever you want


-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: upcoming 6.6.0 release

2019-10-17 Thread Gilles Chehade
On Thu, Oct 17, 2019 at 02:54:26PM +0300, Reio Remma wrote:
> On 17/10/2019 14:20, gil...@poolp.org wrote:
> > Hello misc@,
> > 
> > As some have noticed, the 6.6.0 tag was created on Github to match the code 
> > from smtpd in OpenBSD 6.6.
> > 
> > A portable branch, branch-6.6.0p1, has been forked from there and can be 
> > used to test the matching portable version:
> > 
> >  https://github.com/OpenSMTPD/OpenSMTPD/tree/branch-6.6.0p1
> > 
> > 
> > I have not tagged 6.6.0p1 yet because there's still some time and I want to 
> > gain confidence that it works for most systems and distributions we have 
> > supported so far.
> > 
> > Note that:
> > 
> > - this release will depend on either LibreSSL 3.0.x or OpenSSL 1.1.x
> > - musl-based distros may have issues at this point, they are being tracked 
> > down, musl is not a showstopper for me as we have had issues in previous 
> > releases too but if we can track down the problem I'd be happy 
> > (unsuccessful so far)
> > 
> > I _really_ need help on testing this as I had unexpected hand surgery and 
> > doing the tests myself is extremely long.
> > 
> > Please report as a follow up to this mail what you did test,
> > 
> > 
> 
> Hello! Do you have any pointers as to what an SRS key should look like?
> 

just make it something hard to guess :-)


-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



upcoming 6.6.0 release

2019-10-17 Thread gilles
Hello misc@,

As some have noticed, the 6.6.0 tag was created on Github to match the code 
from smtpd in OpenBSD 6.6.

A portable branch, branch-6.6.0p1, has been forked from there and can be used 
to test the matching portable version:

https://github.com/OpenSMTPD/OpenSMTPD/tree/branch-6.6.0p1


I have not tagged 6.6.0p1 yet because there's still some time and I want to 
gain confidence that it works for most systems and distributions we have 
supported so far.

Note that:

- this release will depend on either LibreSSL 3.0.x or OpenSSL 1.1.x
- musl-based distros may have issues at this point, they are being tracked 
down, musl is not a showstopper for me as we have had issues in previous 
releases too but if we can track down the problem I'd be happy (unsuccessful so 
far)

I _really_ need help on testing this as I had unexpected hand surgery and doing 
the tests myself is extremely long.

Please report as a follow up to this mail what you did test,

Thanks,
Gilles



Re: filter-rspamd DKIM checks failing intermittently.

2019-10-16 Thread Gilles Chehade
On Wed, Oct 16, 2019 at 10:36:32PM +0300, Reio Remma wrote:
> So it's wasn't line breaks afterall.
> 
> It turned out that OpenSMTPD passes raw SMTP data lines to filters and raw
> SMTP lines have leading dot characters escaped by another dot, so .text
> became ..text. Feeding it to Rspamd like that made DKIM alignment tests
> fail, because body hash came out wrong.
> 
> A pull request has been submitted with a fix.
> 

nice catch :-)

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: filter-rspamd DKIM checks failing intermittently.

2019-10-13 Thread Gilles Chehade
Very likely yes, can you give it a try ?

On Sun, Oct 13, 2019, 15:15 Reio Remma  wrote:

> On 13.10.2019 16:09, Reio Remma wrote:
>
> On 13.10.2019 16:05, Gilles Chehade wrote:
>
> I don't think that is the issue, it is probably the filter-rspamd
> reconstruction of the message that is incorrect.
>
>
> I was thinking along the same lines, but I'm not sure how OpenSMTPD splits
> strings before passing them to the filter. Can the filter then extract
> "leftover" line endings for incoming strings and make decision based on
> that when joining the strings before Rspamd?
>
> Do you experience the same yourself?
>
>
> strings.NewReader(strings.Join(s.tx.message, "\n"))
>
> Wonder if we should use \r\n here?
>
>
>
> Reio
>
>
>
> On Sun, Oct 13, 2019, 15:00 Martijn van Duren <
> opensm...@list.imperialat.at> wrote:
>
>> On 10/13/19 1:59 PM, Reio Remma wrote:
>> > Hello!
>> >
>> > I finally moved to Rspamd (2.0) on my production server and I'm seeing
>> > lots of failed DKIM checks, specifically dkim=fail (body hash did not
>> > verify).
>> >
>> >
>> > Authentication-Results: host.domain.com;
>> >  dkim=fail (body hash did not verify) header.d=facebookmail.com
>> > header.s=s1024-2013-q3 header.b=pNWbKJUd;
>> >  dmarc=pass (policy=reject) header.from=facebookmail.com;
>> >  spf=pass (host.domain.com: domain of notificat...@facebookmail.com
>> > designates 66.220.144.215 as permitted sender)
>> > smtp.mailfrom=notificat...@facebookmail.com
>> >
>> > My current stab-in-the-dark theory is that there might be something
>> > going on with line endings when mails are fed to Rspamd.
>> >
>> > Any better theories? :)
>>
>> It's a known issue that mails that don't end on \r\n (both \r\r\n and
>> \n) cause issues. There's efforts going on to see how we can remedy
>> this, but in the mean time tell your senders that they should fix their
>> mails (RFC5321):
>>In addition, the appearance of "bare" "CR" or "LF" characters in text
>>(i.e., either without the other) has a long history of causing
>>problems in mail implementations and applications that use the mail
>>system as a tool.  SMTP client implementations MUST NOT transmit
>>these characters except when they are intended as line terminators
>>and then MUST, as indicated above, transmit them only as a 
>>sequence.
>> >
>> > Thanks,
>> > Reio
>> >
>> >
>>
>>
>
>


Re: filter-rspamd DKIM checks failing intermittently.

2019-10-13 Thread Gilles Chehade
I don't think that is the issue, it is probably the filter-rspamd
reconstruction of the message that is incorrect.

On Sun, Oct 13, 2019, 15:00 Martijn van Duren 
wrote:

> On 10/13/19 1:59 PM, Reio Remma wrote:
> > Hello!
> >
> > I finally moved to Rspamd (2.0) on my production server and I'm seeing
> > lots of failed DKIM checks, specifically dkim=fail (body hash did not
> > verify).
> >
> >
> > Authentication-Results: host.domain.com;
> >  dkim=fail (body hash did not verify) header.d=facebookmail.com
> > header.s=s1024-2013-q3 header.b=pNWbKJUd;
> >  dmarc=pass (policy=reject) header.from=facebookmail.com;
> >  spf=pass (host.domain.com: domain of notificat...@facebookmail.com
> > designates 66.220.144.215 as permitted sender)
> > smtp.mailfrom=notificat...@facebookmail.com
> >
> > My current stab-in-the-dark theory is that there might be something
> > going on with line endings when mails are fed to Rspamd.
> >
> > Any better theories? :)
>
> It's a known issue that mails that don't end on \r\n (both \r\r\n and
> \n) cause issues. There's efforts going on to see how we can remedy
> this, but in the mean time tell your senders that they should fix their
> mails (RFC5321):
>In addition, the appearance of "bare" "CR" or "LF" characters in text
>(i.e., either without the other) has a long history of causing
>problems in mail implementations and applications that use the mail
>system as a tool.  SMTP client implementations MUST NOT transmit
>these characters except when they are intended as line terminators
>and then MUST, as indicated above, transmit them only as a 
>sequence.
> >
> > Thanks,
> > Reio
> >
> >
>
>


Re: Repeated 421 try again later erros

2019-10-10 Thread gilles
definitely rspamd given the message

October 9, 2019 10:41 PM, "Reio Remma" mailto:r...@mrstuudio.ee?to=%22Reio%20Remma%22%20)> wrote:
On 09.10.2019 23:13, Matt Schwartz wrote: Hello List,
I am getting a lot of repeated 421 try again later errors from various lists 
that I am a member of. There is one in particular that is coming from 
outbound.foodtecsolutions.com (http://outbound.foodtecsolutions.com). Here is 
an excerpt from my /var/log/maillog. I am running OpenBSD 6.6-current #344. 
Oct 9 16:07:53 meow smtpd[19379]: a52386b4311e607e smtp connected 
address=52.201.148.113 host=outbound.foodtecsolutions.com 
(http://outbound.foodtecsolutions.com)
Oct 9 16:07:53 meow smtpd[19379]: a52386b4311e607e smtp failed-command 
command="DATA" result="421 try again later"
Oct 9 16:07:53 meow smtpd[19379]: a52386b4311e607e smtp disconnected 
reason=quit 
Below is my smtpd.conf file: 
pki "mail" cert "/etc/ssl/mail.crt" 
pki "mail" key "/etc/ssl/private/mail.key"

table aliases file:/etc/mail/aliases (javascript:false)
table credentials passwd:/etc/mail/credentials
table extras file:/etc/mail/extras (javascript:false)
table relays file:/etc/mail/relays (javascript:false)
table rejects file:/etc/mail/rejects (javascript:false)
table virtuals file:/etc/mail/virtuals (javascript:false)

filter check_rejects phase connect match rdns regex  
disconnect "554 Forbidden"
filter check_rdns phase connect match !rdns 
disconnect "554 No Reverse DNS Configured"
filter rspamd proc-exec "filter-rspamd"

listen on lo filter rspamd
listen on egress tls pki "mail" hostname "mail.goblackcat.com 
(http://mail.goblackcat.com)" 
filter {check_rejects, check_rdns, rspamd}
listen on egress port submission tls-require pki "mail" hostname 
"mail.goblackcat.com (http://mail.goblackcat.com)" 
auth  filter {check_rejects, check_rdns, rspamd}

action "local_mail" mbox alias 
action "virtual_mail" maildir "/var/vmail/%{dest.domain}/%{dest.user}" junk 
virtual 
action "outbound" relay

match for local action "local_mail"
match !from src  mail-from "@goblackcat.com (http://goblackcat.com)" 
reject
match from any for domain "goblackcat.com (http://goblackcat.com)" action 
"virtual_mail"
match auth from any for any action "outbound"
match for any action "outbound" 
I am out of ideas with which to troubleshoot. I am already running smtpd with 
-v switch for more verbosity. 
Thanks, 
Matt  
Greylisting at work?

Good luck,
Reio


Re: Repeated 421 try again later erros

2019-10-10 Thread gilles
October 9, 2019 10:13 PM, "Matt Schwartz"  wrote:

> Hello List,
> 
> I am getting a lot of repeated 421 try again later errors from various lists 
> that I am a member of.
> There is one in particular that is coming from outbound.foodtecsolutions.com. 
> Here is an excerpt
> from my /var/log/maillog. I am running OpenBSD 6.6-current #344.
> 
> [...]
>
> filter rspamd proc-exec "filter-rspamd"
> 

filter rspamd does greylisting



Re: need help

2019-09-30 Thread gilles
September 30, 2019 4:25 PM, "Denis Fondras"  wrote:

> On Mon, Sep 30, 2019 at 01:55:28PM +, gil...@poolp.org wrote:
> 
>> Hello,
>> 
>> I'd like to bring native support for SPF in OpenSMTPD in a future release,
>> but for this I need a bit of help to make sure my SPF resolver works fine.
>> 
>> I have created a repository with a standalone executable that performs the
>> SPF lookup and checks if an IP address is allowed to send on behalf of the
>> sending domain:
>> 
>> https://github.com/poolpOrg/spf
>> 
>> https://github.com/poolpOrg/spf/blob/master/README.md
>> 
>> If you could test and report issues, it would be nice,
> 
> It seems IPv6 check is broken :
> 
> $ dig ledeuns.net TXT +short
> "v=spf1 ip4:185.22.129.11 ip6:2a00:6060:1::1 ip6:2a00:6060:::1005:ff02 
> -all"
> 
> $ ./spf ledeuns.net 185.22.129.1
> checking if 185.22.129.1 can send for ledeuns.net: fail
> $ ./spf ledeuns.net 185.22.129.11
> checking if 185.22.129.11 can send for ledeuns.net: pass
> $ ./spf ledeuns.net 2a00:6060:1::1
> checking if 2a00:6060:1::1 can send for ledeuns.net: fail


will fix that, thanks



Re: need help

2019-09-30 Thread gilles
September 30, 2019 4:51 PM, "Joel Carnat"  wrote:

> Le 30/09/2019 15:55, gil...@poolp.org a écrit :
> 
>> Hello,
>> I'd like to bring native support for SPF in OpenSMTPD in a future > release,
>> but for this I need a bit of help to make sure my SPF resolver works > fine.
>> I have created a repository with a standalone executable that performs > the
>> SPF lookup and checks if an IP address is allowed to send on behalf of > the
>> sending domain:
>> https://github.com/poolpOrg/spf
>> https://github.com/poolpOrg/spf/blob/master/README.md
>>> If you could test and report issues, it would be nice,
> 
> As much as I can understand it, recursion seem to not work.
> 
> Working example:
> # dig -t TXT carnat.net
> carnat.net. 14314 IN TXT "v=spf1 mx -all"
> # ./spf carnat.net 108.61.176.54
> checking if 108.61.176.54 can send for carnat.net: pass
> # ./spf carnat.net 157.55.9.128
> checking if 157.55.9.128 can send for carnat.net: fail
> 
> Not fully working example:
> # dig -t TXT outlook.com
> outlook.com. 600 IN TXT "v=spf1 include:spf-a.outlook.com 
> include:spf-b.outlook.com
> ip4:157.55.9.128/25 include:spf.protection.outlook.com 
> include:spf-a.hotmail.com
> include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsoft.com ~all"
> # ./spf outlook.com 157.55.9.128
> checking if 157.55.9.128 can send for outlook.com: EXISTS: 0
> EXISTS: 0
> pass
> 
> # dig -t TXT spf-a.hotmail.com
> spf-a.hotmail.com. 3600 IN TXT "v=spf1 ip4:157.55.0.192/26 
> ip4:157.55.1.128/26 ip4:157.55.2.0/25
> ip4:65.54.190.0/24 ip4:65.54.51.64/26 ip4:65.54.61.64/26 ip4:65.55.111.0/24 
> ip4:65.55.116.0/25
> ip4:65.55.34.0/24 ip4:65.55.90.0/24 ip4:65.54.241.0/24 ip4:207.46.117.0/24 
> ~all"
> # ./spf outlook.com 65.54.190.5
> checking if 65.54.190.5 can send for outlook.com: EXISTS: 0
> EXISTS: 0
> EXISTS: 0
> EXISTS: 0
> EXISTS: 0
> EXISTS: 0
> soft-fail

I'll look into that, I thought I had handled this case already but I may have 
missed something



Re: need help

2019-09-30 Thread gilles
I'll investigate that, but spfwalk isn't a real SPF resolver and may
yield incorrect results, it just helps a bit.


September 30, 2019 4:27 PM, "Nick Ryan"  wrote:

> Seems to work fine for some hosts but not gmail.com or outlook.com
> 
> mail3$ smtpctl spf walk < 1 (this is gmail.com)
> 35.190.247.0/24
> 64.233.160.0/19
> 
> mail3$ ./spf gmail.com 35.190.247.3 <- in the output of spfwalk
> checking if 35.190.247.3 can send for gmail.com: EXISTS: 0
> EXISTS: 0
> EXISTS: 0
> soft-fail
> 
> mail3$ ./spf gmail.com 185.185.185.185 <- made up address
> checking if 185.185.185.185 can send for gmail.com: EXISTS: 0
> EXISTS: 0
> EXISTS: 0
> soft-fail
> 
> mail3$ ./spf poolp.org 45.76.46.201
> checking if 45.76.46.201 can send for poolp.org: pass
> mail3$ ./spf poolp.org 45.76.46.202
> checking if 45.76.46.202 can send for poolp.org: fail
> 
> Regards - Nick
> 
> On 30/09/2019 14:55, gil...@poolp.org wrote:
> 
>> Hello,
>> I'd like to bring native support for SPF in OpenSMTPD in a future > release,
>> but for this I need a bit of help to make sure my SPF resolver works > fine.
>> I have created a repository with a standalone executable that performs > the
>> SPF lookup and checks if an IP address is allowed to send on behalf of > the
>> sending domain:
>> https://github.com/poolpOrg/spf
>> https://github.com/poolpOrg/spf/blob/master/README.md
>>> If you could test and report issues, it would be nice,



Re: need help

2019-09-30 Thread gilles
yup

September 30, 2019 4:23 PM, "Chris Bennett"  
wrote:

> ./spf no-seas-necio.ninja 162.255.139.10: pass
> ./spf no-seas-necio.ninja 162.255.139.11: soft-fail
> 
> Which matches my spf entry. v=spf1 mx ~all.
> Is that the correct response?
> 
> Chris Bennett



Re: need help

2019-09-30 Thread gilles
yes, this is debug code which i don't  want to spend time making portable ;-)


September 30, 2019 4:10 PM, "Reio Remma"  wrote:

> On 30/09/2019 16:55, gil...@poolp.org wrote:
> 
>> Hello,
>> 
>> I'd like to bring native support for SPF in OpenSMTPD in a future release,
>> but for this I need a bit of help to make sure my SPF resolver works fine.
>> 
>> I have created a repository with a standalone executable that performs the
>> SPF lookup and checks if an IP address is allowed to send on behalf of the
>> sending domain:
>> 
>> https://github.com/poolpOrg/spf
>> 
>> https://github.com/poolpOrg/spf/blob/master/README.md
>> 
>> If you could test and report issues, it would be nice,
> 
> Is it OpenBSD only atm?
> 
> On CentOS 7:
> 
> $ make
> Makefile:26: *** missing separator.  Stop.
> 
> Reio



need help

2019-09-30 Thread gilles
Hello,

I'd like to bring native support for SPF in OpenSMTPD in a future release,
but for this I need a bit of help to make sure my SPF resolver works fine.

I have created a repository with a standalone executable that performs the
SPF lookup and checks if an IP address is allowed to send on behalf of the
sending domain:

https://github.com/poolpOrg/spf

https://github.com/poolpOrg/spf/blob/master/README.md


If you could test and report issues, it would be nice,



Re: table-passwd

2019-09-18 Thread gilles
September 18, 2019 9:38 AM, gil...@poolp.org wrote:

> On my setup, file /etc/mail/accounts is a simple two columns 
> username/password table:
> 
> # head -1 /etc/mail/accounts.txt
> gil...@poolp.org:$2b$09$0ek9ozmo1u0mSsiRo/z2AumROLK.70T9A6bP3mFDqb38L0sC5RvT6
> #

obviously I replaced my real password with `encrypt test` ;-)



Re: table-passwd

2019-09-18 Thread gilles
September 17, 2019 11:41 PM, "Edgar Pettijohn"  wrote:

> On Sep 17, 2019 9:05 AM, Gilles Chehade  wrote:
> 
>> Hello,
>> 
>> Is there anyone using table-passwd for _any_ other purposes than sharing
>> with Dovecot ?
>> 
>> I have built a fully virtual setup which shares credentials with Dovecot
>> and since I managed to do it _without_ table-passwd I'm wondering if the
>> table backend is really useful and if it was not created because soneone
>> had overlooked the first few lines of the Dovecot documentation stating:
>> 
>> "For a password database, it's enough to have only the user and password
>> fields."
>> 
> 
> Not actually using it, but for dovecot to use it as a userdb as well as a 
> passdb it needs the
> additional fields.
> 

ok so I'm misunderstanding the use-case, let me explain why I'm curious:

I wrote table-passwd because I was told that if you wanted to create a virtual 
setup,
backed by one single user, you needed to have a passwd(5)-format file for 
Dovecot and
share that with OpenSMTPD.

But then I did a fully virtual setup for myself and I didn't use table-passwd, 
so the
rationale behind it falls a bit apart for me, unless there's other use-cases.

On my setup, file /etc/mail/accounts is a simple two columns username/password 
table:

# head -1 /etc/mail/accounts.txt 
gil...@poolp.org:$2b$09$0ek9ozmo1u0mSsiRo/z2AumROLK.70T9A6bP3mFDqb38L0sC5RvT6
#


I have the following OpenSMTPD config (three relevant lines):

table accounts "/etc/mail/accounts"

listen on egress port submission [...] auth 

action "deliver_local" maildir junk user _vusers


And I have the following Dovecot config:

# cat /etc/dovecot/conf.d/auth-mailbrix.conf.ext
passdb {
  driver = passwd-file
  args = scheme=CRYPT /etc/mail/accounts
}

userdb {
  driver = static
  args = uid=_vusers gid=_vusers home=/var/maildir/%d/%u
}

# grep auth-mailbrix.conf.ext
10-auth.conf:!include auth-mailbrix.conf.ext


This allows both OpenSMTPD and Dovecot to authenticate accounts that are not 
system
users, allows OpenSMTPD to drop mail to a maildir owned by system account in 
charge
of virtual accounts and allows Dovecot to properly serve these accounts.

Am I missing your use-cases here ?



table-passwd

2019-09-17 Thread Gilles Chehade
Hello,

Is there anyone using table-passwd for _any_ other purposes than sharing
with Dovecot ?

I have built a fully virtual setup which shares credentials with Dovecot
and since I managed to do it _without_ table-passwd I'm wondering if the
table backend is really useful and if it was not created because soneone
had overlooked the first few lines of the Dovecot documentation stating:

"For a password database, it's enough to have only the user and password
 fields."

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: Failed logins hammer/filter.

2019-09-17 Thread Gilles Chehade
On Mon, Sep 16, 2019 at 10:20:42AM +0300, Reio Remma wrote:
> Hello!
> 

Hello,


> Until upgrading to OpenSMTPD 6.6 I used fail2ban to ban excessive login
> failures from IPs, but that doesn't work any more with the log format
> changed from:
> 
> smtp event=failed-command address=185.13.39.7 host=vps-33288.fhnet.fr
> command="AUTH LOGIN (password)" result="535 Authentication failed"
> |
> smtp failed-command command="AUTH LOGIN (password)" result="535
> Authentication failed"
> 

using the human logs for this kind of programmatic stuff is no longer
supported, the proper way is to write a filter that registers for all
register events and parses that output instead.

we assume programs to read reports so the format is versionned and is
going to be easily parsed, we assume humans to read the logs so we're
going to adapt the logs without caring too much about scripts.


> Surprisingly SMTP isn't brute forced that much, but as I registered 472
> failed authentications from a single IP yesterday, I'm going to have a Go at
> a filter too. :)
> 

I do get a lot of brute-force but it mostly comes from compromised hosts
so filtering on !rdns, !fcrdns and matching some common dynamic patterns
kills the bulk of them.

-- 
Gilles Chehade     @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: Filters and rctp-to rewrite.

2019-09-09 Thread gilles
On Mon, Sep 09, 2019 at 07:48:16PM +0300, Reio Remma wrote:

> On 09.09.2019 18:13, Martijn van Duren wrote:
> On 9/9/19 3:37 PM, Reio Remma wrote:
>> Hello!
>> 
>> Slowly digging into filters.
>> 
>> Now I'm curious if it's possible to modify the recipient after say spam
>> check in data-line? I get the impression that rewriting rcpt-to at that
>> stage is impossible, but my goal would be to redirect/quarantine high
>> scoring spam to a special e-mail address.
>> 
>> Would it be doable somehow?
>> 
>> Thanks!
>> Reio
>> 
> It is not.
> 
> What you might be able to do is add an additional header and somehow let
> an lmtp server make the decision based on the header.
> 
> I haven't used lmtp myself, no clue if this actually works, but it's
> worth investigating :-)
> 
> Please reply to the threat if you managed to make it work.
> 
> Thanks Martijn and Gilles for the confirmation!
> 
> I'm currently using amavisd-new with the quarantine feature, but I'm itching
> to switch to Rspamd (greylisting here I come!).

Rspamd can let you interface Amavis.

I did for someone I'm managing mail for so that you don't have to plug a
ton of different tools directly to OpenSMTPD but let Rspamd broker these
different filters and produce a result.

> I'm using the quarantine to keep an eye on mails with a medium spam score so
> we won't lose the occasional legit mail with a higher than normal spam
> score. Additionally I can train these borderline mails correctly as
> ham/spam.

I will soon extend the filter-rspamd to include symbols it matched.

With this, you'll be able to do delivery-time classification and move to
a quarantine folder if you find the proper symbol.

> I now see Rspamd has a metadata exporter feature I could probably use to
> copy spammy mails to the quarantine mail address.

Dunno about that

> There are also Dovecot's sieve scripts. I'll have to see which work better.

I'm not a big fan of Sieve, it is far more complex than it should be.

I use it for Inbox -> SPAM, Spam -> Inbox training but quite franly if a
classification can happen at delivery time through MDA, I'd rather let a
fdm ruleset classify.

--
Gilles Chehade @poolpOrg

https://www.poolp.org patreon: https://www.patreon.com/gilles



Re: Filters and rctp-to rewrite.

2019-09-09 Thread gilles
September 9, 2019 3:37 PM, "Reio Remma"  wrote:

> Hello!
> 

Hello,


> Slowly digging into filters.
> 
> Now I'm curious if it's possible to modify the recipient after say spam check 
> in data-line? I get
> the impression that rewriting rcpt-to at that stage is impossible, but my 
> goal would be to
> redirect/quarantine high scoring spam to a special e-mail address.
> 

By the time you start receiving DATA, the RCPT TO decision has already been 
taken
in the SMTP transaction, so that would basically be a jump back in time.


> Would it be doable somehow?
> 

If you want to rewrite the RCPT TO, then not doable without your filter issuing 
a
SMTP transaction itself by connecting and playing a session.

If you want to junk (add X-Spam header), then you can simply have you filter do 
a
buffering of DATA and prepend header on commit (what filter-rspamd does).



Re: OpenSMTPD-Logwatch script.

2019-09-05 Thread gilles
Yes, see the smtpd.conf(5) man page:

filter myreporter proc-exec "/tmp/reporting.sh"

listen on [...] filter myreporter



September 5, 2019 10:30 AM, "Reio Remma"  wrote:

> On 03/09/2019 21:32, gil...@poolp.org wrote:
> 
>> September 3, 2019 8:29 PM, "Reio Remma"  wrote:
>> 
>>> On 27.04.2018 12:26, Reio Remma wrote:
>> 
>> Hello all,
>> 
>> I've whipped together a Logwatch script for OpenSMTPD. I've anyone is > 
>> interested on giving it a
>> try, it's now at:
>> 
>> https://github.com/whataboutpereira/OpenSMTPD-Logwatch
>>> Hello! I've now updated the Logwatch script to work with OpenSMTPD 6.6.0 
>>> (current portable).
>>> 
>>> Good luck,
>>> Reio
>> 
>> Nice
>> 
>> I wonder why you didn't parse the reporting output though, it would have 
>> been much much simpler :-)
> 
> About that:
> 
> proc reporting "/tmp/reporting.sh"
> report smtp on reporting
> 
> The 2nd line gives smtpd -n a syntax error. Has the syntax changed between 
> your post and now? :)
> 
> Reio



Re: New syntax and virtual aliases to remote addresses.

2019-09-05 Thread gilles
could be interesting to implement a tls builtin filter, so you could:

filter check-tls phase mail-from match !tls junk

and flag non tls options as spam, without discarding them completely


September 5, 2019 8:38 AM, "Reio Remma"  wrote:

> On 02/09/2019 18:37, Reio Remma wrote:
> 
>> On 02/09/2019 18:36, Reio Remma wrote:
>> 
>>> Now I ended up switcing to tls-require on port 25. I wonder how much >> 
>>> spam that will take down!
>>> :)
> 
> Well, that's depressing.
> 
> On the spam front - requiring TLS apparently cuts off about 99% of spam 
> (SpamAssassin is
> practically out of work), but we do get the occasional legit non-TLS 
> connection and I'm having to
> switch back to optional TLS. :/
> 
> So TLS is 20 years old but they're (banks etc) still sending somewhat private 
> info in plain text.
> 
> 'twas fun while it lasted. :)
> 
> I'm going to start schooling them one by one.
> 
> Reio



Re: Building 6.4.2p1 without ssl?

2019-09-02 Thread gilles
September 2, 2019 9:48 AM, "Ede Wolf"  wrote:

> Hello,
> 
> trying to compile opensmtp it fails with openssl errors, so I've tried to 
> specify --without-libssl
> at configure time, as at least for testing and learning the basics it is not 
> really that important,
> but it does not seem to get honored.
> 
> Any idea, what I may have to change?
> 
> Thanks
> 
> Ede
> 
> In case anybody has an idea for building with openssl, here are the final 
> words of the compiler:
> 

OpenSMTPD no longer supports OpenSSL but I made it build again, so the
next release (6.6.0) due in a few weeks will build fine for you on any
supported system that ships with OpenSSL 1.1.x.

There is no way to disable TLS support, this is a mandatory dependency
just like libevent.

Gilles



Re: New syntax and virtual aliases to remote addresses.

2019-09-02 Thread gilles
September 2, 2019 3:35 PM, "Reio Remma"  wrote:
> Hello!
> 
> I was able to have virtual aliases pointing to external addresses with the 
> old syntax, but it
> doesn't seem to work like that with new rules:
> 

Not awake enough to process what follows but the new syntax certainly allows 
this
as my whole infrastructure depends on it ;-)

I suspect that there's a problem with the ruleset that prevents external 
addresses
from matching a rule themselves.


> action filter_incoming relay host smtp://127.0.0.1:10024
> action sign_outgoing   relay host smtp://127.0.0.1:10026
> action relay_outgoing  relay
> action deliver_lmtplmtp "/var/run/dovecot/lmtp" rcpt-to virtual 
>  userbase 
> 
> match tag FILTERED for domain  action deliver_lmtp
> match tag SIGNED   for any action relay_outgoing
> match auth from anyfor domain  action deliver_lmtp
> match from any for domain  rcpt-to  action 
> filter_incoming
> match auth from anyfor any action sign_outgoing
> 
> Old rules were:
> 
> accept tagged Filtered for domain  virtual  userbase 
>  deliver to lmtp
> "/var/run/dovecot/lmtp" rcpt-to
> accept from local  for domain  virtual  userbase 
>  deliver to lmtp
> "/var/run/dovecot/lmtp" rcpt-to
> accept from !local for domain  recipient  relay via 
> smtp://127.0.0.1:10024
> accept tagged Signed   for any relay
> accept from local  for any relay via smtp://127.0.0.1:10026
> 
> recipi...@mydomain.com used to be nicely redirected to a remote 
> recipi...@someotherdomain.com, but
> with the new syntax it doesn't hit any rules.
> 
> I found that adding "match tag FILTERED for any action relay_outgoing" after 
> "match tag FILTERED
> for domain  action deliver_lmtp" solves the issue.
> 
> I suspect that the old "accept tagged Filtered" went straight to MTA after 
> expanding the virtual
> alias?
> 
> Does it sound right to "match tag FILTERED for any" after dealing with 
> FILTERED for our domains?
> 
> Can I do anything about DKIM breaking for forwarded mails?
> 
> Thanks,
> Reio



Re: OpenSMTP as a library

2019-09-02 Thread gilles
September 2, 2019 3:59 PM, "Manfred Rebentisch"  wrote:

> Hello,
> I am new to OpenSMTP and in this mailinglist.
> 

Hello and welcome,


> Is it possible to use OpenSMTP as a library to use mail send
> functionality from my C / C++ software?
> 
> I want to replace the old and unsupported esmtp library.
> 

Nope, OpenSMTPD is a standalone daemon and doesn't ship or build any kind of 
library.

However, the code is ISC licensed and the client code is isolated in smtpc.c, 
so feel
free to reuse.

> Thank you for an answer in advance
> 

Np,



Re: OpenSMTPD build on OpenSSL 1.1.x

2019-08-28 Thread Gilles Chehade
On Wed, Aug 28, 2019 at 10:55:05AM +0300, Reio Remma wrote:
> On 28/08/2019 10:44, gil...@poolp.org wrote:
> > 28 ao??t 2019 00:00 "Reio Remma"  a ??crit:
> > 
> > > On 27.08.2019 21:25, Richard Narron wrote:
> > > 
> > > > The OpenSMTPD portable version from 
> > > > https://github.com/OpenSMTPD/OpenSMTPD
> > > > works fine on Slackware64 current with OpenSSL 1.1.1c and gcc 9.2
> > > > It took me a while to get it to work though.
> > > > I first downloaded the "current" portable version from
> > > > https://opensmtpd.org/archives/opensmtpd-6.4.2p1.tar.gz
> > > > And I got errors very similar to those of Denis Fateyev on Fedora 30.
> > > > Next I downloaded the portable version from github.com
> > > > and found that autoconf had not been run and this was no good.
> > > > Finally I discovered the post on the mailing list which mentioned the
> > > > "bootstrap" script and then I was able to download and build the 
> > > > portable
> > > > version from git.
> > > > The code shows version "6.6.0-portable".
> > > > It runs fine on Slackware64 current and I'm happy that it now works with
> > > > OpenSSL 1.1
> > > > Regards,
> > > > Richard Narron
> > > Your success pushed me to try 6.6.0 on CentOS 7 with OpenSSL 1.1.1c.
> > > 
> > > Can anyone tell me if changing to -lcrypto -lssl to -l:libssl.a 
> > > -l:libcrypto.a is the correct way
> > > to get OpenSSL 1.1.1c statically compiled into OpenSMTPD? I ended up 
> > > using these (and -pthreads
> > > -ldl) and managed to build an RPM based on 6.0.3 RPM from CentOS 7.
> > > 
> > I don't know about the -l:lib notation sorry
> > 
> > Out of curiosity, why would you want ssl statically compiled into OpenSMTPD 
> > ?
> > This means that when an issue hits OpenSSL, updating OpenSSL and restarting 
> > the daemon will not be
> > enough to be back on track.
> > 
> > In addition, I'm not sure why you need -pthreads because OpenSMTPD is not 
> > multi-threaded.
> 
> Hello!
> 
> CentOS 7 has OpenSSL 1.0.2k as the max version and with OpenSSL 1.1.1c
> compiled into OpenSMTPD I can run the new OpenSMTPD version on a machine
> with CentOS 7's old OpenSSL version.
> 
> I had to add -pthreads and -ldl to pass 'make' with the static OpenSSL
> libraries. Without these I ran into errors hinting at threads and dl.
> 
> I'm a little wary of just forcibly replacing the whole OpenSSL 1.0.2k on a
> production machine. :)
> 

Understood !

OpenSSL 1.0.x is going to be supported until 2019-12-31 so this will get
solved by itself soon ;-)

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: OpenSMTPD build on OpenSSL 1.1.x

2019-08-28 Thread gilles
28 août 2019 00:00 "Reio Remma"  a écrit:

> On 27.08.2019 21:25, Richard Narron wrote:
> 
>> The OpenSMTPD portable version from https://github.com/OpenSMTPD/OpenSMTPD
>> works fine on Slackware64 current with OpenSSL 1.1.1c and gcc 9.2
>> It took me a while to get it to work though.
>> I first downloaded the "current" portable version from
>> https://opensmtpd.org/archives/opensmtpd-6.4.2p1.tar.gz
>> And I got errors very similar to those of Denis Fateyev on Fedora 30.
>> Next I downloaded the portable version from github.com
>> and found that autoconf had not been run and this was no good.
>> Finally I discovered the post on the mailing list which mentioned the
>> "bootstrap" script and then I was able to download and build the portable
>> version from git.
>> The code shows version "6.6.0-portable".
>> It runs fine on Slackware64 current and I'm happy that it now works with
>> OpenSSL 1.1
>> Regards,
>> Richard Narron
> 
> Your success pushed me to try 6.6.0 on CentOS 7 with OpenSSL 1.1.1c.
> 
> Can anyone tell me if changing to -lcrypto -lssl to -l:libssl.a 
> -l:libcrypto.a is the correct way
> to get OpenSSL 1.1.1c statically compiled into OpenSMTPD? I ended up using 
> these (and -pthreads
> -ldl) and managed to build an RPM based on 6.0.3 RPM from CentOS 7.
> 

I don't know about the -l:lib notation sorry

Out of curiosity, why would you want ssl statically compiled into OpenSMTPD ?
This means that when an issue hits OpenSSL, updating OpenSSL and restarting the 
daemon will not be
enough to be back on track.

In addition, I'm not sure why you need -pthreads because OpenSMTPD is not 
multi-threaded.



> The fresh RPM installed nicely on a clean CentOS 7 with their OpenSSH 1.0.2k 
> and OpenSMTPD started
> too:Aug 28 00:54:54 localhost smtpd[25943]: info: OpenSMTPD 6.6.0-portable 
> starting
> Aug 28 00:50:07 localhost smtpd[9338]: cfa3e1042696f77a mta connecting
> address=smtp://108.177.14.27:25 host=lt-in-f27.1e100.net
> Aug 28 00:50:07 localhost smtpd[9338]: cfa3e1042696f77a mta connected
> Aug 28 00:50:07 localhost smtpd[9338]: cfa3e1042696f77a mta tls
> ciphers=TLSv1.3:TLS_AES_256_GCM_SHA384:256
> Aug 28 00:50:07 localhost smtpd[9338]: cfa3e1042696f77a mta server-cert-check 
> result="success"
> Aug 28 00:50:07 localhost smtpd[9338]: cfa3e1042696f77a mta delivery 
> evpid=953ab16d13e43b2f
> from= to= rcpt=<-> 
> source="192.168.1.142"
> relay="108.177.14.27 (lt-in-f27.1e100.net)" delay=3m12
> s result="Ok" stat="250 2.0.0 OK 1566942607 w6si428635lfk.121 - gsmtp" More 
> testing will have to
> wait until tomorrow. :)
> 
> Good luck,
> Reio



Re: OpenSMTPD build on OpenSSL 1.1.x

2019-08-28 Thread gilles
Hello,

27 août 2019 20:25 "Richard Narron"  a écrit:

> The OpenSMTPD portable version from https://github.com/OpenSMTPD/OpenSMTPD
> works fine on Slackware64 current with OpenSSL 1.1.1c and gcc 9.2
> 

Yay !


> It took me a while to get it to work though.
> 
> I first downloaded the "current" portable version from
> https://opensmtpd.org/archives/opensmtpd-6.4.2p1.tar.gz
> 
> And I got errors very similar to those of Denis Fateyev on Fedora 30.
> 

Yes, our latest release was not OpenSSL 1.1 compatible, the next one which
will happen in October will be.


> Next I downloaded the portable version from github.com
> and found that autoconf had not been run and this was no good.
> 
> Finally I discovered the post on the mailing list which mentioned the
> "bootstrap" script and then I was able to download and build the portable
> version from git.
> 

That's because this is the development branch, we run the boostrap script
when we perform a release.

In October, all you'll have to do is download the tarball from the website,
like you did with 6.4.2p1, and it will work for OpenSSL 1.1


> The code shows version "6.6.0-portable".
> 
> It runs fine on Slackware64 current and I'm happy that it now works with
> OpenSSL 1.1
> 

Neat



Re: tags on the portable branch?

2019-08-26 Thread Gilles Chehade
On Sun, Aug 25, 2019 at 07:16:23AM +0200, Harald Dunkel wrote:
> Hi Gilles,
> 
> On 8/24/19 9:14 PM, Gilles Chehade wrote:
> > 
> > This is expected.
> > 
> > Version 6.4.x only builds with LibreSSL or OpenSSL 1.0.x
> > 
> 
> do you think it would be possible to set a tag matching support
> for openssl 1.1.1c as well? The version I am using right now now
> is based on 772da22936c8d80f7ad3284ea7e5bdbfdbee2efb, but this
> might be too experimental for production use.
> 

I'm unsure I understand what you want :-/

OpenSSL 1.1.x is only supported in the development branch so you need to
track latest commit in branch 'portable': the commit you're using is one
of the development branch from two weeks ago, if you are happy with that
keep it, it's no less or more experimental than any commit in the branch
since you're already running development code.

I wouldn't know what to tag honestly


-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: stumped on virtual aliases

2019-08-24 Thread Gilles Chehade
On Sat, Aug 24, 2019 at 04:14:21PM +0200, Joseph A Borg wrote:
> I???m trying to match an email: nos...@domain.tld to expand to 
> webmas...@domain.tld and deliver to local mailbox.
> 
> My setup is pretty simple and works well for virtual mailboxes linked to a 
> couple of virtual domains. now I???m trying t add aliases to some accounts.
> 
> From the error, it seems like smtpd is not transforming the alias address 
> into the final address for delivery.
> Seems like I???m not understanding how smtpd transforms an recipient address 
> in an  into the actual recipient for delivery.
> 
> I must be conceptually stumped on this one.
>
> [...]
> 
> the configuration files for the domain and tables follows.
> 
>
> # file:/etc/mail/domain.tld/accounts/valiases:
> =
> > nospam webmas...@domain.tld
> ## EOF
> 
> # file:/etc/mail/domain.tld/accounts/emails
> 
> > webmas...@domain.tld::/home/domain/mail/master
> > webdus...@domain.tld::/home/domain/mail/duster
> > webbus...@domain.tld::/home/domain/mail/buster
> ## EOF
> 

Your problem lies here.

expansion resolves an e-mail address into a user-part, so ultimately
what's looked up in a userbase is the part before '@'

if you're already using a virtual account, why not do something like
this ?

valiases:
==
nospam webmaster

emails:
==
webmaster   ::/hoome/domain/mail/master


if you really want the indirection, you can even:

nospam webmas...@domain.tld
webmas...@domain.tld   webmaster



> # file:/etc/mail/domain.sub.smtpd.conf
> =
> 
> > ## =
> > ## DOMAIN CONFIGURATION:
> > ## =
> > # TABLE DECLATATIONS:
> > # ---
> > table domains   \
> > file:/etc/mail/domain.tld/domains
> > table valiases  \
> > file:/etc/mail/domain.tld/accounts/valiases
> > table e-boxes   \
> > file:/etc/mail/domain.tld/accounts/emails
> > 
> > # ACTIONS
> > # ---
> > action valiases_set \
> > expand-only \
> > virtual 
> > action deliver_virtual_set  \
> > maildir \
> > userbase 
> > 
> > # MATCHES
> > # ---
> > match from any  \
> >   for domain   \
> >   action valiases_set
> > match from any  \
> >   for domain   \
> >   action deliver_virtual_set
> > 
> > 
> 

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: Service names in listen on directives

2019-08-24 Thread Gilles Chehade
On Sat, Aug 24, 2019 at 10:16:26PM +0200, Martijn van Duren wrote:
> On 8/24/19 10:06 PM, Gilles Chehade wrote:
> > On Sat, Aug 24, 2019 at 12:32:05PM -0700, Darren S. wrote:
> >> OpenBSD 6.5 amd64
> >> OpenSMTPD 6.5.0
> >>
> >> port [port]
> >> Listen on the given port instead of the default port 25.
> >>
> >> I wanted to confirm if service names are intended to be supported for
> >> `listen on` option in smtpd.conf.
> >>
> >> These result in syntax failure:
> >>
> >> listen on lo port smtp
> >> listen on lo port smtps
> >>
> >> These do not:
> >>
> >> listen on lo port 25
> >> listen on lo port 465
> >>
> >> This also does not:
> >>
> >> listen on lo port submission
> >>
> >> Found it curious that `submission` may be used in place of a port
> >> number but not the other service names.
> >>
> > 
> > this is because `smtp' and `smtps` are keywords, so they must be quoted:
> > 
> > listen on lo port "smtp"
> > 
> > 
> Don't know if there's interest, but considering the port argument is
> non-optional and smtp and smtps are valid (and imho not unreasonable)
> port names I reckon we could add them explicitly so they can be used
> without quotes.
> 

You beat me to it, yes this makes sense.


> Index: parse.y
> ===
> RCS file: /cvs/src/usr.sbin/smtpd/parse.y,v
> retrieving revision 1.258
> diff -u -p -r1.258 parse.y
> --- parse.y   23 Aug 2019 19:05:01 -  1.258
> +++ parse.y   24 Aug 2019 20:14:40 -
> @@ -1863,6 +1863,38 @@ opt_if_listen : INET4 {
>   free($2);
>   listen_opts.port = ntohs(servent->s_port);
>   }
> + | PORT SMTP {
> + struct servent *servent;
> +
> + if (listen_opts.options & LO_PORT) {
> + yyerror("port already specified");
> + YYERROR;
> + }
> + listen_opts.options |= LO_PORT;
> +
> + servent = getservbyname("smtp", "tcp");
> + if (servent == NULL) {
> + yyerror("invalid port: smtp");
> + YYERROR;
> + }
> + listen_opts.port = ntohs(servent->s_port);
> + }
> + | PORT SMTPS{
> + struct servent *servent;
> +
> + if (listen_opts.options & LO_PORT) {
> + yyerror("port already specified");
> + YYERROR;
> + }
> + listen_opts.options |= LO_PORT;
> +
> + servent = getservbyname("smtps", "tcp");
> + if (servent == NULL) {
> + yyerror("invalid port: smtps");
> + YYERROR;
> + }
> + listen_opts.port = ntohs(servent->s_port);
> + }
>   | PORT NUMBER   {
>   if (listen_opts.options & LO_PORT) {
>   yyerror("port already specified");
> 

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: Service names in listen on directives

2019-08-24 Thread Gilles Chehade
On Sat, Aug 24, 2019 at 12:32:05PM -0700, Darren S. wrote:
> OpenBSD 6.5 amd64
> OpenSMTPD 6.5.0
> 
> port [port]
> Listen on the given port instead of the default port 25.
> 
> I wanted to confirm if service names are intended to be supported for
> `listen on` option in smtpd.conf.
> 
> These result in syntax failure:
> 
> listen on lo port smtp
> listen on lo port smtps
> 
> These do not:
> 
> listen on lo port 25
> listen on lo port 465
> 
> This also does not:
> 
> listen on lo port submission
> 
> Found it curious that `submission` may be used in place of a port
> number but not the other service names.
> 

this is because `smtp' and `smtps` are keywords, so they must be quoted:

listen on lo port "smtp"


-- 
Gilles Chehade     @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: tags on the portable branch?

2019-08-24 Thread Gilles Chehade
On Sat, Aug 24, 2019 at 04:19:11PM +0200, Harald Dunkel wrote:
> On 8/23/19 9:55 PM, John Cox wrote:
> > Hi
> > 
> > Whilst I know it doesn't help you I just git cloned that URL and the
> > tag checkout just worked for me.  What happens if you make another new
> > (temporary) repo with clone and try again?
> > 
> > Regards
> > 
> > John Cox
> > 
> 
> Using a new clone, as suggested: The tag "opensmtpd-6.4.2p1" is available,
> but it doesn't build on Debian sid (openssl 1.1.1c). Full build.log is
> attached.
> 

This is expected.

Version 6.4.x only builds with LibreSSL or OpenSSL 1.0.x

See:

https://poolp.org/posts/2019-07-27/july-2019-report-tons-of-smtpd-work-mostly/



-- 
Gilles Chehade     @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: table api question

2019-08-24 Thread gilles
24 août 2019 02:59 "Edgar Pettijohn"  a écrit:

> I am writing a table-lua, however the table_lua_update function doesn't 
> appear to be called.
> Here are relevant pieces of the code.
> 
> The lookup function works. However, it would be more ideal to have the 
> update() called early
> to fill in the tables for the other functions. As is the lookup() has to do 
> the work of both.
> 
> Any help is appreciated.
> 

update is called when you issue an `smtpctl table update ` command.

On a side note, I had this discussion with someone a few days ago but can't 
remember
who, so if it was you and you already know, disregard:

I have a plan for the next two releases to switch the implementation of tables 
to an
API similar to that of filters, so we can have tables become scripts that read 
lines
from stdin, write answers to stdout, be written in any language, etc..

Not discouraging you from writing something using the current API, it is not so 
much
work anyways, but just letting you know that in a relatively short term your 
code is
going to need a rewrite.



Re: tags on the portable branch?

2019-08-22 Thread Gilles Chehade
On Thu, Aug 22, 2019 at 10:24:30AM +0200, Harald Dunkel wrote:
> Hi folks,
> 
> would it be possible to set tags on the portable branch as well?
> Something like
> 
>   portable-6.4.1
> 
> would do.
> 
> This could help alot for creating some kind of "official" source
> package for Debian and Fedora/RedHat.
> 

you mean like this ? :-)

https://github.com/OpenSMTPD/OpenSMTPD/releases/tag/opensmtpd-6.4.2p1

-- 
Gilles Chehade @poolpOrg

https://www.poolp.org    patreon: https://www.patreon.com/gilles



Re: smtpd not passing data to rspamd

2019-08-22 Thread Gilles Chehade
On Wed, Aug 21, 2019 at 08:06:58PM +, Thomas Smith wrote:
> 
> ? Original Message ?
> On Wednesday, August 21, 2019 8:28 AM, Gilles Chehade  
> wrote:
> 
> > On Wed, Aug 21, 2019 at 03:22:39PM +, Thomas Smith wrote:
> >
> > > Hi,
> > > I've setup filter-rspamd with rspamd. Both appear to be running (smtpd 
> > > and rspamd), I'm able to query rspamd's controller, access the web UI; 
> > > smtpd is processing and delivering mail as expected.
> > > ps wuax | grep rspam
> > > root 86736 0.0 0.4 45236 4008 ?? I 6:30AM 0:00.05 rspamd: main process 
> > > (rspamd)
> > > _rspamd 32135 0.0 1.0 45344 10140 ?? S 6:30AM 0:00.23 rspamd: 
> > > rspamd_proxy process (localhost:11332) (rspamd)
> > > _rspamd 4059 0.0 1.4 45688 14632 ?? S 6:30AM 0:01.63 rspamd: controller 
> > > process (localhost:11334) (rspamd)
> > > _rspamd 16743 0.0 1.1 45384 11020 ?? S 6:30AM 0:00.33 rspamd: normal 
> > > process (localhost:11333) (rspamd)
> > > _smtpd 32851 0.0 0.4 105520 3624 ?? I 6:56AM 0:00.01 
> > > /usr/local/bin/filter-rspamd
> > > _smtpd 68802 0.0 0.1 844 808 ?? Ip 6:56AM 0:00.00 sh -c 
> > > /usr/local/bin/filter-rspamd
> > > However, I don't see any messages being processed by rspamd. Nor do I see 
> > > any indication that data is being sent to rspamd (nothing in the logs, no 
> > > stats appearing in the web UI).
> >
> > can you show full logs for a sample smtpd session that didn't go through 
> > rspamd ?
> 
> Is this what you're looking for?
> 
> Aug 21 12:42:22 host smtpd[71198]: 43e03ee20005a41f smtp connected 
> address=x.x.x.x host=***t.com
> Aug 21 12:42:23 host smtpd[71198]: 43e03ee20005a41f smtp message 
> msgid= size=338369 nrcpt=1 proto=ESMTP
> Aug 21 12:42:23 host smtpd[71198]: 43e03ee20005a41f smtp envelope 
> evpid= 
> from=<t.com> 
> to=<***.***>
> Aug 21 12:42:24 host smtpd[71198]: 43e03ee20005a41f smtp disconnected 
> reason=quit
> 
> The msgid reveals some additional data, but the server doesn't manage final 
> delivery--emails are received and relayed only. So the additional message 
> information is related to the outbound (relayed) email but I can provide if 
> needed.
> 

sorry but this is tricky to troubleshoot with so few logs, obfuscated on
top of it :-/

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



FLOSS Weekly 543 OpenSMTPD

2019-08-21 Thread Gilles Chehade
Hello everyone,

I was invited to talk a bit about SMTP and OpenSMTPD in FLOSS Weekly.

Here is the link in case you're interested:

 https://twit.tv/shows/floss-weekly/episodes/543

Cheers

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: smtpd not passing data to rspamd

2019-08-21 Thread Gilles Chehade
On Wed, Aug 21, 2019 at 03:22:39PM +, Thomas Smith wrote:
> Hi,
> 
> I've setup filter-rspamd with rspamd. Both appear to be running (smtpd and 
> rspamd), I'm able to query rspamd's controller, access the web UI; smtpd is 
> processing and delivering mail as expected.
> 
> ps wuax | grep rspam
> root 86736  0.0  0.4 45236  4008 ??  I   6:30AM0:00.05 rspamd: 
> main process (rspamd)
> _rspamd  32135  0.0  1.0 45344 10140 ??  S   6:30AM0:00.23 rspamd: 
> rspamd_proxy process (localhost:11332) (rspamd)
> _rspamd   4059  0.0  1.4 45688 14632 ??  S   6:30AM0:01.63 rspamd: 
> controller process (localhost:11334) (rspamd)
> _rspamd  16743  0.0  1.1 45384 11020 ??  S   6:30AM0:00.33 rspamd: 
> normal process (localhost:11333) (rspamd)
> _smtpd   32851  0.0  0.4 105520  3624 ??  I  6:56AM0:00.01 
> /usr/local/bin/filter-rspamd
> _smtpd   68802  0.0  0.1   844   808 ??  Ip  6:56AM0:00.00 sh -c 
> /usr/local/bin/filter-rspamd
> 
> However, I don't see any messages being processed by rspamd. Nor do I see any 
> indication that data is being sent to rspamd (nothing in the logs, no stats 
> appearing in the web UI).
> 

can you show full logs for a sample smtpd session that didnt go through rspamd ?


> smtpd.conf:
> filter "rspamd" proc-exec "/usr/local/bin/filter-rspamd"
> listen on egress tls hostname $mx_domain pki $mx_domain filter "rspamd"
> 
> 'smtpd -d -v':
> debug: smtp: listen on x.x.x.x port 25 flags 0x2401 pki "" ca ""
> 
> I also don't see any debug messages regarding rspamd.
> 

your config is correct


-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: Question about OpenSMTPD and Debian package and filters/spam filtering

2019-08-21 Thread Gilles Chehade
On Wed, Aug 21, 2019 at 12:50:10PM +0200, Michiel van Es wrote:
> Hi!
> 

Hi,


> I am running a small VPS with 1 GB memory with Debian 10 amd64 with OpenSMTPD 
> (6.0.3) for private email and am looking what my best options are to limit 
> spam.
> I know there are some filters from Joerg 
> (https://www.mail-archive.com/misc@opensmtpd.org/msg04402.html) but am not 
> sure if these will work with my version of OpenSMTPD (I get a syntax error 
> when trying the old filter syntax).
> 
> I can also relay everything to Amavisd/SpamAssassin but then email won???t 
> get blocked at the SMTP level, also ASSP or Rspamd is an option but they are 
> pretty resource intensive and will eat all my VPS memory ;) 
> 
> What would be my best option?
> 

6.0.3 is a fairly old version and there aren't many options available.

if you're forced to stick with that version, which suffers from at least
one denial of service as far as I know, your best option is to relay via
something like SpamPD so it can interface with SpamAssassin, but this is
not going to operate at SMTP level, it will happen at delivery time.

there will be no way of blocking at SMTP level before next release 6.6.0
that is going to happen in a few weeks, during October, so any option is
going to be post delivery: either as a custom MDA, or as a relay via for
some smtp proxy that will reinject in smtpd like the dkimproxy stuff.

your best option would really be to build from source 6.4.2: it will not
block at SMTP level but will provide mechanisms to ease interfacing with
spamassassin or rspamd for post-SMTP handling.

if you're not too easily scared, running the development version is good
too because it's very close to release now, very stable and will not get
much changes until October as I'm busy busy these days ;-)


> I like to do some DNSBL and SpamAsssassin checks if possible.
> 
> My config if that is to any use to give some insights:
> 
> pki server.pragmasec.nl certificate 
> "/etc/letsencrypt/live/pragmasec.nl/fullchain.pem"
> pki server.pragmasec.nl key "/etc/letsencrypt/live/pragmasec.nl/privkey.pem"
> listen on localhost
> listen on eth0 port 25 tls pki server.pragmasec.nl hostname 
> server.pragmasec.nl auth-optional
> listen on eth0 port 587 tls-require pki server.pragmasec.nl hostname 
> server.pragmasec.nl auth
> table vdomains file:/etc/mail/domains
> table vusers file:/etc/mail/vusers
> expire 7d
> limit mta inet4
> accept from any for domain  virtual  deliver to mda 
> "/usr/lib/dovecot/dovecot-lda -f %{sender} -a %{rcpt}"
> accept from local for any relay
> 
> Cheers,
> 
> Michiel
> 
> 
> 

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: forcing SMTP authentication

2019-08-21 Thread Gilles Chehade
On Wed, Aug 21, 2019 at 07:39:42AM +0200, Selmeci Tam?s wrote:
> Hello!
> 
> In brief: STARTTLS is enabled, there is a self-signed certificate for
> encryption (better than nothing), smarthost is used to send mails from
> my domain. My problem is that it still accepts SMTP connections (over
> TLS) without authentication. What I want:
> - anybody can send email to my email address in my domain (now it's
> working);
> - relaying through my SMTP server is allowed only after successful
> authentication (now anybody can relay through my server without
> authentication, e.g. to send spams). Authentication should be based on
> regular /etc/passwd file (local users of the computer). In order to
> hide the passwords, STARTTLS should be used;
> 
> It's a rather simple configuration, but I wasn't able to set it up. If
> I put 'auth' into the 'listen on' line, it needs authentication to any
> access of the SMTP server, so other machines (e.g. from google.com)
> can't send me mails. Using 'authenticated' in 'accept from' directives
> also didn't do the trick appropriately (it wasn't able to receive any
> mails at all).
> 
> Could you please help me out with this?
> 
> Thanks, regards,
> ---
> ---
> pki mail.486.hu certificate "/etc/smtpd/mail.486.hu.crt"
> pki mail.486.hu key "/etc/smtpd/mail.486.hu.key"
> 
> table cred file:/etc/smtpd/cred
> 
> listen on eth0  port 25 hostname mail.486.hu tls-require
> listen on localhost port 25 hostname mail.486.hu tls-require
> 

you should add:

listen on eth0 port 587 hostname mail.486.hu tls-require auth


> # Storing mails arriving at the domain '486.hu'.
> accept from any for domain 486.hu deliver to mbox
> 
> # If the recipient is out of domain '486.hu', the mail is relayed through the
> # smarthost using TLS and authentication, see 'cred' file.
> accept from any for ! domain 486.hu relay via
> tls+auth://t-onl...@mail.t-online.hu auth  
> 

That last rule is essentially "accept from any for (pretty much) any" so
you have created an open relay.

Replace the "from any" with "from local" so the rule reads as:

   accept from local for ! domain 486.hu relay via
  tls+auth://t-onl...@mail.t-online.hu auth  

This should be much better.

-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



Re: filter assistance requested

2019-08-20 Thread Gilles Chehade
On Mon, Aug 19, 2019 at 01:48:05PM -0500, Edgar Pettijohn wrote:
> Nothing urgent here. Probably can wait for filter documentation. However, 
> I've been
> playing with filters off and on with limited success. It seems like data-line 
> will 
> be the most useful to filter on so thats been my focus lately. I have the 
> following
> script:
> 
> #!/usr/bin/perl
> 
> use strict;
> use warnings;
> use diagnostics;
> 
> open(my $fh, '>', '/tmp/log.txt') or die $!;
> 
> # remove line buffering
> select(STDOUT);
> $|++;
> select($fh);
> $|++;
> 
> print STDOUT "register|filter|smtp-in|data-line\n";
> print STDOUT "register|ready\n";
> 
> while ( <> ) {
> chomp; # get rid of newline
> 
> my @report = split /\|/;
> 
> next if $report[0] eq 'config';
> 
> foreach (@report) {
> print $fh "$_\|"; # just to see whats there
> }
>   print $fh "\n";
> 
> my $inbody = 0;
> my ($sid, $token, $line);
> $sid = $report[5];
> $token = $report[6];
> $line = $report[$#report];
> if ($report[0] eq 'filter' and $report[3] eq 'smtp-in' and $report[4] 
> eq 'data-line') {
> die "invalid filter command" if (scalar @report < 7);
> if ($line eq '') { print $fh "end of headers\n"; $inbody++; }
> if ($line eq '.') { print $fh "end of message\n"; $inbody--; }
> print $fh "filter-dataline|$token|$sid|$line\n";
> print STDOUT "filter-dataline|$token|$sid|$line\n";
> }
> }
> 
> close $fh;
> 
> 0;
> 
> It prints the following in /tmp/log.txt after a 
> $ echo "HI" | mail edgar
> 
> filter|0|1566239933.835511|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|Received:
>  from localhost (deathstar.my.domain [local])|
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|Received: from localhost 
> (deathstar.my.domain [local])
> filter|0|1566239933.835523|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|
>  by deathstar.my.domain (OpenSMTPD) with ESMTPA id 7052ea5a|
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|  by 
> deathstar.my.domain (OpenSMTPD) with ESMTPA id 7052ea5a
> filter|0|1566239933.835529|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|
>  for ;|
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|  for 
> ;
> filter|0|1566239933.835533|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|
>  Mon, 19 Aug 2019 13:38:53 -0500 (CDT)|
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|  Mon, 19 Aug 2019 
> 13:38:53 -0500 (CDT)
> filter|0|1566239933.836673|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|From:
>  Edgar Pettijohn |
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|From: Edgar Pettijohn 
> 
> filter|0|1566239933.836681|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|Date:
>  Mon, 19 Aug 2019 13:38:53 -0500 (CDT)|
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|Date: Mon, 19 Aug 2019 
> 13:38:53 -0500 (CDT)
> filter|0|1566239933.836685|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|To:
>  edgar|
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|To: edgar
> filter|0|1566239933.836688|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|ef8747a12860387a
> filter|0|1566239933.836692|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|HI|
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|HI
> filter|0|1566239933.836695|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|.|
> end of message
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|.
> 
> I get the following response:
> deathstar$ sendmail: command failed: 550 5.7.1 Delivery not authorized, 
> message refused: Message is not RFC 2822 compliant
> 
> I see the same from a telnet session as well.
> 
> So there are 2 problems. First my script doesn't appear to acurately 
> determine that the headers are finished. Second mail
> doesn't go through. Any suggestions are appreciated.
> 

This error occurs when you don't have at least an empty line to separate
headers from the body.

Based on your output, it seems that you are generating a bad line:

> filter|0|1566239933.836688|smtp-in|data-line|c0002b41f6bd164d|ef8747a12860387a|
> filter-dataline|ef8747a12860387a|c0002b41f6bd164d|ef8747a12860387a

There should be a | at the end of your filter-dataline because it is not
valid as is, I'll check why smtpd did not hit a fatal.


-- 
Gilles Chehade @poolpOrg

https://www.poolp.orgpatreon: https://www.patreon.com/gilles



  1   2   3   4   5   6   7   8   9   >