Re: New filters auth and sign
On Sat, 01 Jun 2024 08:45:00 +0100, "Corey Hickman" wrote: > > does it have policy server included? for instance, when DKIM fails, the > policy can be set up to deny the message. > Right now it ignores DMARC as if it doesn't exist. Doing a DMARC lookup for domain and inserting it's results into the header is possible and not a big deal, but it has some issues. The first is parsing the From header. It is durable, but different MUA may follow different logic and parser for this can be quite complicated. And complicated means bugs. The second is more ideological. DMARC needs something that aggregates the results and sends out reports. It shouldn't be a filter for smtpd. But a filter can write it's decision to log, and something should harvest it to process and create reports that need to be sent. Anyway, forensic reports, which should be close to real-time and include a lot of things from the original email, is a much more complicated story. All this brings up the question of personal data / GDPR and DMARC. I know of a very good analysis of DMARC and GDPR in the case of German law [1], which can be summarized as a quote: The reports are fundamentally permitted and justified under data protection law. However, the principle of proportionality is to be complied with at all times. Based on this analysis, I assume that only aggregated reports can be used without legal headaches in the EU. But implementing only a part of DMARC seems as much worse than not implementing it at all, and implementing it in its entirety requires a lot of pieces in place, much more than just a filter. Thus, DMARC was discussed on the OpenBSD mailing lists a few months ago [2]. As a conclusion, I personally use the p=none policy, because I assume that my mail should be delivered, and To is not the final destination, it's a kind of starting direction of the mail's way to the recipient. Footnotes: [1] https://certified-senders.org/wp-content/uploads/2018/08/Report_DMARC_and_GDPR.pdf [2] https://marc.info/?l=openbsd-misc&m=171015367409290&w=2 -- wbr, Kirill
New filters auth and sign
Greetings,
I'd like to announce a two new filters for OpenSMTD which better to use
together: auth and sign.
auth is a filter which verify DKMI, ARC and SPF, and iprev. It adds
Authentication-Results header or ARC-Authentication-Results.
sign is a filter which adds DKMI or ARC signature, or ARC seal.
For example, I run configuration:
filter "auth" proc-exec "filter-auth"
listen on egress port smtp ... filter { admdscrub, "auth", dnsbl }
filter sign_ed25519 proc-exec "filter-sign -a ed25519-sha256 -D
/etc/mail/domains \
-s 20240125ed25519 -k /etc/mail/dkim/20240125.ed25519.key" user
_dkimsign group _dkimsign
filter sign_rsa proc-exec "filter-sign -a rsa-sha256 -D /etc/mail/domains \
-s 20240125rsa -k /etc/mail/dkim/20240125.rsa.key" user _dkimsign
group _dkimsign
filter arc_auth proc-exec "filter-auth -A"
filter arc_sign proc-exec "filter-sign -A -a rsa-sha256 -d mx.catap.net \
-s 20240125rsa -k /etc/mail/dkim/20240125.rsa.key" user _dkimsign
group _dkimsign
filter arc_seal proc-exec "filter-sign -S -a rsa-sha256 -d mx.catap.net \
-s 20240125rsa -k /etc/mail/dkim/20240125.rsa.key" user _dkimsign
group _dkimsign
filter sign chain { sign_ed25519 sign_rsa arc_auth arc_sign arc_seal }
listen on egress port submission ... filter sign
Here all incomming messages is autorised by adding Authentication-Results,
and all outcomming messages:
- signed by two DKMI signature with correct domain (list in /etc/mail/domains)
- signed by one ARC signature with domain mx.catap.net
- seal by one ARC seal with domain mx.catap.net
Yeah, it is possible to use different selectors for ARC signature and seal,
but I haven't tested it.
The code is based on Martijn van Duren's filter-dkimsign, filter-dkimverify
and filter-spf, and I also used some pices from spfwalk.c from OpenSMTPD.
Man pages for both filters are updated.
Thus, sign filter is drop-in replacment for filter-dkimsign.
Code available here:
- https://github.com/catap/opensmtpd-filter-auth
- https://github.com/catap/opensmtpd-filter-sign
I also attached ports for OpenBSD which I used to run it.
How stable it is? Well, enough to share and ask for feedback. It may
contains bugs, but it should be fine to use.
Produced signature was tested against gmail, yahoo, icloud.com and dkimpy
and it holds. Anyway, outlook.com fails on ARC signature with errors 35 or
47 (what does it mean?) and produced invalid signature as the next in ARC
chain (tested by dkimpy).
Thus, this email were sent via server which uses that filters, so, headers
from this email a good example.
--
wbr, Kirill
filters.tgz
Description: Binary data
Bypassing filters in specific conditions
Hello,
I have a OpenBSD 7.5 with OpenSMTPD+rspamd+postgresql+dovecot setup with
virtuals (users and domains) and credentials in a postgresql database.
In addition to theses virtual users, i do some redirections to other
mail servers for specific aliases, and in that specific situation, i was
wondering if i could make opensmtpd completely bypass filters
(especially the rspamd filter).
Here is my smtpd config :
# This is the smtpd server system-wide configuration file.
# See smtpd.conf(5) for more information.
srs key REDACTED
srs key backup REDACTED
## Certs
pki gamindustri.fr cert "REDACTED"
pki gamindustri.fr key "REDACTED"
table domains postgres:/etc/mail/domains-pgsql.conf
table virtuals postgres:/etc/mail/virtuals-pgsql.conf
table passwd postgres:/etc/mail/passwd-pgsql.conf
table sympa file:/etc/mail/sympa.table
table sndrs postgres:/etc/mail/sndrs-pgsql.conf
table domains-max file:/etc/mail/domains-max.table
table addr-max file:/etc/mail/addr-max.table
filter "rdns" phase connect match !rdns disconnect "550 Incorrect
rDNS error"
filter "fcrdns" phase connect match !fcrdns disconnect "550 Incorrect
fcrDNS error"
filter "rspamd" proc-exec "filter-rspamd"
## Ports to listen on, and how to listen on them
listen on lo0 port 25 filter "rspamd"
listen on egress port 25 tls pki gamindustri.fr hostname
compa.gamindustri.fr filter { "rdns", "fcrdns", "rspamd" }
listen on egress port 465 smtps pki gamindustri.fr hostname
compa.gamindustri.fr mask-src received-auth senders
auth-optional filter "rspamd"
listen on egress port submission tls-require pki gamindustri.fr
hostname compa.gamindustri.fr mask-src received-auth senders
auth filter "rspamd"
action "dovecotFinal" lmtp "/var/dovecot/lmtp" rcpt-to virtual
action "send" relay srs
action "send-to-sympa" relay host smtp://REDACTED srs
action "send-to-max" relay host smtp://REDACTED srs
match from src REDACTED action "send"
match from any for domain rcpt-to action
"send-to-sympa"
match from any for domain action "send-to-max"
match from src REDACTED mail-from mailer-dae...@compa.gamindustri.fr
action "send"
match from src REDACTED action "send"
match from any for domain action "dovecotFinal"
match from auth for any action "send"
match from local for any action "send
RE: How to terminate smtpd filters?
I should have included more information. OS is FreeBSD 13.2. smtpd is 7.3.0
which is the latest in the pkg collection.
Originally, the only change I made to the filter, was add
#!/usr/local/bin/python3.9 and made afilter.py executable.
All other filters terminate correctly when smtpd exits.
filter afilter proc-exec "/data/afilter.py"
listen on bridge20 port smtp tls pki mail.tekrealm.net auth-optional
filter { afilter, trusted, check_dyndns, check_rdns,
check_fcrdns, senderscore, rspamd }
listen on bridge20 port smtps smtps pki mail.tekrealm.net auth
mask-src filter { afilter, rspamd }
action "outbound" relay helo mail.tekrealm.net filter "afilter"
Ultimately, I was able to get it to exit correctly after the confirmation that
it should exit when stdin closes.
I will submit a patch to the github repo after I beat on this for a few days or
so.
Thank you for your assistance.
-Andrew
-Original Message-
From: Tassilo Philipp
Sent: Tuesday, April 2, 2024 1:22 AM
To: gil...@poolp.org
Cc: and...@tekrealm.net; misc@opensmtpd.org
Subject: Re: How to terminate smtpd filters?
I agree with Gilles, your filter should react on stdin closing, but not sure
how your filter is set up.
Also, just a guess... are you running smtpd on Linux?
Linux doesn't kill children when the parent process dies, maybe that's related?
(To make it do that prctl(2) would need to be used w/
PR_SET_PDEATHSIG.)
On Tue, Apr 02, 2024 at 06:53:39AM +, gil...@poolp.org wrote:
> April 2, 2024 4:47 AM, and...@tekrealm.net wrote:
>
>> What signals a termination for smtpd filters?
>>
>> I'm using the code at
>> https://github.com/Beutlin/howto-opensmtpd-filters-and-reports,
>> Which works great, except for when smtpd gets shutdown. The script
>> continues to run and consumes up to 100% cpu time, while keeping the
>> smtpd parent? process running.
>>
>> I've tried adding a SIGTERM handler to the code which didn't work, as
>> well as I saw mentioned that the filter should exit on EOF, so I
>> tried wrapping
>> parser.dispatch() in
>> a try/except EOFError and using sys.exit. That didn't work either.
>>
>> I've read smtpd-filters, and looked at various other filters and I am
>> not understanding what tells the filter to shutdown.
>>
>
> The filter is connected to smtpd through its stdin, so it can
> terminate when there's an EOF on stdin.
>
> This is the proper way to do it and how all filters I wrote work but
> maybe a bug has crawled in the handling of filter termination and it
> went unnoticed, I don´t think I ever terminated smtpd in years besides system
> restarts.
>
> What system are you running on ?
>
Re: How to terminate smtpd filters?
I agree with Gilles, your filter should react on stdin closing, but not sure how your filter is set up. Also, just a guess... are you running smtpd on Linux? Linux doesn't kill children when the parent process dies, maybe that's related? (To make it do that prctl(2) would need to be used w/ PR_SET_PDEATHSIG.) On Tue, Apr 02, 2024 at 06:53:39AM +, gil...@poolp.org wrote: April 2, 2024 4:47 AM, and...@tekrealm.net wrote: What signals a termination for smtpd filters? I'm using the code at https://github.com/Beutlin/howto-opensmtpd-filters-and-reports, Which works great, except for when smtpd gets shutdown. The script continues to run and consumes up to 100% cpu time, while keeping the smtpd parent? process running. I've tried adding a SIGTERM handler to the code which didn't work, as well as I saw mentioned that the filter should exit on EOF, so I tried wrapping parser.dispatch() in a try/except EOFError and using sys.exit. That didn't work either. I've read smtpd-filters, and looked at various other filters and I am not understanding what tells the filter to shutdown. The filter is connected to smtpd through its stdin, so it can terminate when there's an EOF on stdin. This is the proper way to do it and how all filters I wrote work but maybe a bug has crawled in the handling of filter termination and it went unnoticed, I don´t think I ever terminated smtpd in years besides system restarts. What system are you running on ?
Re: How to terminate smtpd filters?
April 2, 2024 4:47 AM, and...@tekrealm.net wrote: > What signals a termination for smtpd filters? > > I'm using the code at > https://github.com/Beutlin/howto-opensmtpd-filters-and-reports, > Which works great, except for when smtpd gets shutdown. The script continues > to run and > consumes up to 100% cpu time, while keeping the smtpd parent? process > running. > > I've tried adding a SIGTERM handler to the code which didn't work, as well > as I saw > mentioned that the filter should exit on EOF, so I tried wrapping > parser.dispatch() in > a try/except EOFError and using sys.exit. That didn't work either. > > I've read smtpd-filters, and looked at various other filters and I am not > understanding > what tells the filter to shutdown. > The filter is connected to smtpd through its stdin, so it can terminate when there's an EOF on stdin. This is the proper way to do it and how all filters I wrote work but maybe a bug has crawled in the handling of filter termination and it went unnoticed, I don´t think I ever terminated smtpd in years besides system restarts. What system are you running on ?
How to terminate smtpd filters?
What signals a termination for smtpd filters? I'm using the code at https://github.com/Beutlin/howto-opensmtpd-filters-and-reports, Which works great, except for when smtpd gets shutdown. The script continues to run and consumes up to 100% cpu time, while keeping the smtpd parent? process running. I've tried adding a SIGTERM handler to the code which didn't work, as well as I saw mentioned that the filter should exit on EOF, so I tried wrapping parser.dispatch() in a try/except EOFError and using sys.exit. That didn't work either. I've read smtpd-filters, and looked at various other filters and I am not understanding what tells the filter to shutdown. Regards, -Andrew
Re: Example of smtpd-filters
On 2023/10/20 21:52:41 +0200, Sagar Acharya wrote: > I'm simply unable to start. > > I have to know where to get the mail from, format of text, which process to > give the return to, in what format, i.e. are there any standard return values > for accept or reject mail. I'd like to stress the point that a key difference of OpenSMTPD' filters is that they DO NOT EXIT. A filter is just a program (or a script) that is executed by smtpd at start and never exits. If a filter quits, it's a non-recoverable failure and smtpd dies too. The filter and smtpd talk via a simple protocol: smtpd writes lines to the filter standard input and the filter writes its responses to the standard output. Tassilo already shared some very sample code to get you started, for the rest you can refer to smtpd-filters[0] which describes the protocol. Feel free (of course!) to ask for clarification for any aspect not clearly covered in the manual page. Thanks, Omar Polo [0]: https://man.openbsd.org/smtpd-filters.7
Re: Example of smtpd-filters
You basically write a script that processes incoming lines on stdin, and
writes back to stdout. The protocol is described in smtpd-filters(7).
This basic idea in lua would be something like:
for line in io.lines() do
if line == 'in' then
io.write('out')
end
end
Now handle the lines you want to handle, and respond to them how it's
described in smtpd-filters(7).
On Fri, Oct 20, 2023 at 09:52:41PM +0200, Sagar Acharya wrote:
I'm simply unable to start.
I have to know where to get the mail from, format of text, which
process to give the return to, in what format, i.e. are there any
standard return values for accept or reject mail.
Thanking you
Sagar Acharya
https://humaaraartha.in/selfdost/selfdost.html
20 Oct 2023, 23:47 by tphil...@potion-studios.com:
Post your script that you got so far, explain where you get stuck/confused, and
someone will probably be happy to help.
On Fri, Oct 20, 2023 at 02:05:13PM +0200, Sagar Acharya wrote:
Can you please help me with a sample script for accepting or rejecting mail in
lua.
Say there is myfilter.lua within /etc/smtpd/
Can one please help with a sample whose pseudocode goes like
fetch(mail)if mailbody contains foo, reject,
else if mailbody contains bar, accept.
Such a sample script would help very much to support users write their own custom filters. Thanking you
Sagar Acharya
https://humaaraartha.in/selfdost/selfdost.html
Re: Example of smtpd-filters
I'm simply unable to start. I have to know where to get the mail from, format of text, which process to give the return to, in what format, i.e. are there any standard return values for accept or reject mail. Thanking you Sagar Acharya https://humaaraartha.in/selfdost/selfdost.html 20 Oct 2023, 23:47 by tphil...@potion-studios.com: > Post your script that you got so far, explain where you get stuck/confused, > and someone will probably be happy to help. > > > On Fri, Oct 20, 2023 at 02:05:13PM +0200, Sagar Acharya wrote: > >> Can you please help me with a sample script for accepting or rejecting mail >> in lua. >> >> Say there is myfilter.lua within /etc/smtpd/ >> >> Can one please help with a sample whose pseudocode goes like >> >> fetch(mail)if mailbody contains foo, reject, >> else if mailbody contains bar, accept. >> >> Such a sample script would help very much to support users write their own >> custom filters. Thanking you >> Sagar Acharya >> https://humaaraartha.in/selfdost/selfdost.html >>
Re: Example of smtpd-filters
Post your script that you got so far, explain where you get stuck/confused, and someone will probably be happy to help. On Fri, Oct 20, 2023 at 02:05:13PM +0200, Sagar Acharya wrote: Can you please help me with a sample script for accepting or rejecting mail in lua. Say there is myfilter.lua within /etc/smtpd/ Can one please help with a sample whose pseudocode goes like fetch(mail)if mailbody contains foo, reject, else if mailbody contains bar, accept. Such a sample script would help very much to support users write their own custom filters. Thanking you Sagar Acharya https://humaaraartha.in/selfdost/selfdost.html
Example of smtpd-filters
Can you please help me with a sample script for accepting or rejecting mail in lua. Say there is myfilter.lua within /etc/smtpd/ Can one please help with a sample whose pseudocode goes like fetch(mail)if mailbody contains foo, reject, else if mailbody contains bar, accept. Such a sample script would help very much to support users write their own custom filters. Thanking you Sagar Acharya https://humaaraartha.in/selfdost/selfdost.html
Re: Can OpenSMTPD filters rewrite MAIL FROM based on a message header?
May 18, 2023 4:45 PM, "Pēteris Caune" wrote: > Hello, > Hello, > Is it possible to configure OpenSMTPD in some way so that it would grab a > specific header from an > email message, and use it in the MAIL FROM field? > Nope, not doable but that's because DATA which contains the headers is emitted by the client AFTER that MAIL FROM and RCPT TO are both accepted, so it's already too late to rewrite them. > From reading the smtpd-filters man page, and the palant.info article [1] it > seems that the filter > would first receive the MAIL FROM value and have a chance to rewrite it, and > only afterwards see > the message itself, including its headers. I don't have hands-on experience > with writing filters > yet, so perhaps I'm completely misunderstanding how filters work. > Filters are actually hooks that happen during an SMTP session, so they are triggered as SMTP commands are being received from a client, they allow "man-in-the-middle"-ing sessions as if they were plugged between the client and the server and validating/altering commands in real time. If you _really_ wanted to use headers to alter a MAIL FROM, you would have to actually write an SMTP proxy that sits _before_ the filters so that it buffers the client session then play your updated session to OpenSMTPD. There is no way to do what you want from within a session without some kind of proxy buffering upfront. > For context, what I'm trying to accomplish: > > I have a web app that sends automated notifications to its users, and needs > to handle bounce > messages (to disable future automated notifications when mailboxes disappear, > or users mark email > as spam instead of unsubscribing, etc.) > unsure I understand :-/ > I am currently using a commercial SMTP relay service which supports bounce > notifications via > webhooks: when an email bounces (immediate or delayed bounce), the service > sends a HTTP POST > request to my configured webhook address, with bounce details in the request > body. When sending > email, I can include a specially named header in the messages, and the > webhook callbacks will > contain the value of that custom header. This way, I can associate bounce > notifications with events > in my web app. > > I'm looking into replicating this setup with OpenSMTPD. My idea is to deliver > bounce notifications > to the web app using either a "mda curl -XPOST @- > https://webhook-address-here"; action, or a "|curl > -XPOST @- https://webhook-address-here"; alias in the aliases table. The > webhook payload would > contain the entire message, and it would be the webhook handler's > responsibility to parse the > delivery report and handle it. > > But I need to pass some custom data from the original message to the bounce > notification, and so > I'm thinking I could rewrite the MAIL FROM address and include the custom > data in there. Ideally, > if feasible, I'd like to make OpenSMTPD do it. Alternatively, I could patch > my web app to put the > custom data in the MAIL FROM address from the very start, when submitting the > message. > > TL;DR: can OpenSMTPD filters rewrite MAIL FROM based on a header of the mail > message? And, related, > does what I'm trying to do here make any sense? Is there a perhaps a simpler > solution that I am not > seeing? > > Thanks! > Sorry, I don't understand the use-case but that's probably because I'm both tired and haven't done much mail in a while :-)
Can OpenSMTPD filters rewrite MAIL FROM based on a message header?
Hello, Is it possible to configure OpenSMTPD in some way so that it would grab a specific header from an email message, and use it in the MAIL FROM field? >From reading the smtpd-filters man page, and the palant.info article [1] it >seems that the filter would first receive the MAIL FROM value and have a >chance to rewrite it, and only afterwards see the message itself, including >its headers. I don't have hands-on experience with writing filters yet, so >perhaps I'm completely misunderstanding how filters work. For context, what I'm trying to accomplish: I have a web app that sends automated notifications to its users, and needs to handle bounce messages (to disable future automated notifications when mailboxes disappear, or users mark email as spam instead of unsubscribing, etc.) I am currently using a commercial SMTP relay service which supports bounce notifications via webhooks: when an email bounces (immediate or delayed bounce), the service sends a HTTP POST request to my configured webhook address, with bounce details in the request body. When sending email, I can include a specially named header in the messages, and the webhook callbacks will contain the value of that custom header. This way, I can associate bounce notifications with events in my web app. I'm looking into replicating this setup with OpenSMTPD. My idea is to deliver bounce notifications to the web app using either a "mda curl -XPOST @- https://webhook-address-here"; action, or a "|curl -XPOST @- https://webhook-address-here"; alias in the aliases table. The webhook payload would contain the entire message, and it would be the webhook handler's responsibility to parse the delivery report and handle it. But I need to pass some custom data from the original message to the bounce notification, and so I'm thinking I could rewrite the MAIL FROM address and include the custom data in there. Ideally, if feasible, I'd like to make OpenSMTPD do it. Alternatively, I could patch my web app to put the custom data in the MAIL FROM address from the very start, when submitting the message. TL;DR: can OpenSMTPD filters rewrite MAIL FROM based on a header of the mail message? And, related, does what I'm trying to do here make any sense? Is there a perhaps a simpler solution that I am not seeing? Thanks! [1] https://palant.info/2023/03/08/converting-incoming-emails-on-the-fly-with-opensmtpd-filters/
smtp-out reporting / outbound filters
Heho, i am currently looking at adding MTA-STS/DANE support to my mailer; However, these are not supported in opensmtpd. Given my limited coding abilities, i figured it might make more sense to try implementing that as a filter, given that [1] mentions outbound filters. however, man smtpd-filters(7) no longer mentions outbound filters. Is that feature gone or does someone have docs on using it sitting around? With best regards, Tobias [1]https://poolp.org/posts/2019-12-24/december-2019-opensmtpd-and-filters-work-articles-and-goodies/
Questions regarding filters
I'm new to OpenSMTPd (on OpenBSD 7.0-release) and when I started looking into filters I came up with some questions that the smtpd.conf manpage doesn't seem to answer. - for 'filter ... proc ...' and 'filter ... proc-exec ...' where in the processing of the message is the filter invoked and how does it interact with OpenSMTPd (what info is it given, what info does it return, etc; from the DKIM example it's evidently able to add a header to the message)? [How it interacts may be something only filter developers really care about, but when it's invoked is relevant for everyone.] - for decision 'rewrite value' is 'value' a fixed string, a string that may include references to matches in a regex (if a condition was '... regex '; this would probably require that there be exactly one condition with 'regex'), or what? Presumably the rewritten value affects only the envelope(s), not the headers in the message itself. [The need for regex references occurred to me because I was thinking about standardizing the domain part of MAIL FROM and RCPT TO, eliminating mention of specific hosts and leaving just my domain's name, which would require getting the username from the regex match.] - in the phase definitions I originally read 'data' as meaning immediately after the DATA command is received and 'commit' as after the '.' that terminates the message body, but the comment after the list of decisions that junking must happen 'before a message is committed' makes me think that was wrong. What is actually meant? - in the list of decisions 'session or transaction' is stated without making it clear what the difference is, and I don't see a clear statement of this anywhere else. What exactly is the difference? A nit: in the phase definition for 'commit' there's an extra 'is'. BTW, despite my questions I really like OpenSMTPd. Many thanks for creating it. Dave -- Dave Anderson
Re: what happened to smtpd-filters.7 ?
Hi Martin, thank you very much for your response. I stumbled over this lost man page looking for additional information about the filters mentioned on https://man.openbsd.org/smtpd.conf. Apparently there are a few more unused source files in the git repository. They are very hard to detect without running make. Regards Harri
Re: what happened to smtpd-filters.7 ?
filters are implemented in lka_filter.c. According to cvs log filter.c is removed in 2017 and was probably part of the first filter attempt. smtpd-filters.7 has never been hooked up to the build. Probably because it needs a little more scrutiny. But most in there can be used. martijn@ On Fri, 2021-03-19 at 07:56 +0100, Harald Dunkel wrote: > Hi folks, > > looking at github there is a file "smtpd-filters.7" and "filter.c" > in smtpd, but apparently they are not used at build or install time. > configure.ac doesn't mention them, either, so I wonder whats the > story here? Have they been forgotten? Obsolete code? > > > Regards > Harri >
what happened to smtpd-filters.7 ?
Hi folks, looking at github there is a file "smtpd-filters.7" and "filter.c" in smtpd, but apparently they are not used at build or install time. configure.ac doesn't mention them, either, so I wonder whats the story here? Have they been forgotten? Obsolete code? Regards Harri
Re: how to watch opensmtpd filters at work?
On 1/7/21 3:03 PM, Martijn van Duren wrote: Could you show your config, steps to reproduce and expected behaviour? Because I'm not entirely sure what you try to achieve. I was trying to see which rules in smtpd.conf match. "smtpctl trace all" didn't work. Problem was, I hadn't enabled debug logging in syslog.conf. This rendered "smtpctl trace rules" useless. I would strongly recommend to mention this in the man page. Its easy to forget. Regards Harri
Re: how to watch opensmtpd filters at work?
On Jan 7, 2021 5:45 AM, Harald Dunkel wrote:Hi folks, for debugging I would like to know which "match" line does actually match the incoming EMails. Is there some option for opensmtpd to watch it? "-v" seems to be insufficient. Every insightful comment would be highly appreciated. Regards Harri I think you are looking forsmtpctl trace rulesEdgar
Re: how to watch opensmtpd filters at work?
Could you show your config, steps to reproduce and expected behaviour? Because I'm not entirely sure what you try to achieve. On Thu, 2021-01-07 at 13:24 +0100, Harald Dunkel wrote: > On 1/7/21 1:03 PM, Martijn van Duren wrote: > > Your question isn't really specific, but my best guess is that -Tfilters > > will do the trick. > > > > I tried "smtpctl trace all", but there was no visual effect. > > > Regards > Harri >
Re: how to watch opensmtpd filters at work?
On 1/7/21 1:03 PM, Martijn van Duren wrote: Your question isn't really specific, but my best guess is that -Tfilters will do the trick. I tried "smtpctl trace all", but there was no visual effect. Regards Harri
Re: how to watch opensmtpd filters at work?
Your question isn't really specific, but my best guess is that -Tfilters will do the trick. martijn@ On Thu, 2021-01-07 at 12:45 +0100, Harald Dunkel wrote: > Hi folks, > > for debugging I would like to know which "match" line does > actually match the incoming EMails. Is there some option for > opensmtpd to watch it? "-v" seems to be insufficient. > > Every insightful comment would be highly appreciated. > > > Regards > Harri >
how to watch opensmtpd filters at work?
Hi folks, for debugging I would like to know which "match" line does actually match the incoming EMails. Is there some option for opensmtpd to watch it? "-v" seems to be insufficient. Every insightful comment would be highly appreciated. Regards Harri
Re: smtpd-filters.7 patch
Missed a typo "s/strenght/strength/" On 01/25/20 14:12, Edgar Pettijohn wrote: On 01/25/20 14:08, gil...@poolp.org wrote: The diff reads ok but I wonder why you removed this sentence: -No decision is ever taken by the report stream. I think it made it a bit more clear that reporting is informative only. I felt that the line stating it was a one-way stream covered it. Mainly it just made me stumble on the line and have to reread it a couple of times. Edgar diff --git a/smtpd/smtpd-filters.7 b/smtpd/smtpd-filters.7 index 1e1a27ef..bf563174 100644 --- a/smtpd/smtpd-filters.7 +++ b/smtpd/smtpd-filters.7 @@ -89,22 +89,21 @@ to inform in real-time about events that are occurring in the daemon. The report events do not expect an answer from .Nm , -it is just meant to provide them with informations. +it is just meant to provide them with information. A filter should be able to replicate the .Xr smtpd 8 -state for a session by gathering informations coming from report events. -No decision is ever taken by the report stream. +state for a session by gathering information coming from report events. .Pp The filter stream is a two-way stream which allows .Xr smtpd 8 to query .Nm about what it should do with a session at a given phase. -The filter requests expects an answer from +The filter requests expect an answer from .Nm , .Xr smtpd 8 will not let the session move forward until then. -A decision must always be taken by the filter stream. +A decision must always be made by the filter stream. .Pp It is sometimes possible to rely on filter requests to gather information, but because a reponse is expected by @@ -112,13 +111,13 @@ but because a reponse is expected by this is more costly than using report events. The correct pattern for writing filters is to use the report events to create a local state for a session, -then use filter requests to take decisions based on this state. +then use filter requests to make decisions based on this state. The only case when using filter request instead of report events is correct, is when a decision is required for the filter request and there is no need for more information than that of the event. .Sh PROTOCOL The protocol is straightforward, -it consists of a human-readable line exchanges between +it consists of human-readable line exchanges between .Nm and .Xr smtpd 8 @@ -165,7 +164,7 @@ will be documented in the sections below. .Sh CONFIGURATION During the initial handshake, .Xr smtpd 8 -will emit a serie of configuration keys and values. +will emit a series of configuration keys and values. The list is meant to be ignored by .Nm that do not require it and consumed gracefully by filters that do. @@ -265,7 +264,7 @@ This event is generated upon successful negotiation of TLS. .Pp .Ar tls-string contains a colon-separated list of TLS properties including the TLS version, -the cipher suite used by the session and the cipher strenght in bits. +the cipher suite used by the session and the cipher strength in bits. .It Ic link-disconnect This event is generated upon disconnection of the client. .It Ic link-auth : Ar username Ar result @@ -514,7 +513,7 @@ filter|0.5|1576146008.006103|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d filter|0.5|1576146008.006105|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d|. .Ed .Pp -They are expected to produce an output stream similarly terminate by a single +They are expected to produce an output stream similarly terminated by a single dot. A filter may inject, suppress,
Re: smtpd-filters.7 patch
On 01/25/20 14:08, gil...@poolp.org wrote: The diff reads ok but I wonder why you removed this sentence: -No decision is ever taken by the report stream. I think it made it a bit more clear that reporting is informative only. I felt that the line stating it was a one-way stream covered it. Mainly it just made me stumble on the line and have to reread it a couple of times. Edgar
Re: smtpd-filters.7 patch
The diff reads ok but I wonder why you removed this sentence: -No decision is ever taken by the report stream. I think it made it a bit more clear that reporting is informative only.
smtpd-filters.7 patch
diff --git a/smtpd/smtpd-filters.7 b/smtpd/smtpd-filters.7 index 1e1a27ef..3cdb10e1 100644 --- a/smtpd/smtpd-filters.7 +++ b/smtpd/smtpd-filters.7 @@ -89,22 +89,21 @@ to inform in real-time about events that are occurring in the daemon. The report events do not expect an answer from .Nm , -it is just meant to provide them with informations. +it is just meant to provide them with information. A filter should be able to replicate the .Xr smtpd 8 -state for a session by gathering informations coming from report events. -No decision is ever taken by the report stream. +state for a session by gathering information coming from report events. .Pp The filter stream is a two-way stream which allows .Xr smtpd 8 to query .Nm about what it should do with a session at a given phase. -The filter requests expects an answer from +The filter requests expect an answer from .Nm , .Xr smtpd 8 will not let the session move forward until then. -A decision must always be taken by the filter stream. +A decision must always be made by the filter stream. .Pp It is sometimes possible to rely on filter requests to gather information, but because a reponse is expected by @@ -112,13 +111,13 @@ but because a reponse is expected by this is more costly than using report events. The correct pattern for writing filters is to use the report events to create a local state for a session, -then use filter requests to take decisions based on this state. +then use filter requests to make decisions based on this state. The only case when using filter request instead of report events is correct, is when a decision is required for the filter request and there is no need for more information than that of the event. .Sh PROTOCOL The protocol is straightforward, -it consists of a human-readable line exchanges between +it consists of human-readable line exchanges between .Nm and .Xr smtpd 8 @@ -165,7 +164,7 @@ will be documented in the sections below. .Sh CONFIGURATION During the initial handshake, .Xr smtpd 8 -will emit a serie of configuration keys and values. +will emit a series of configuration keys and values. The list is meant to be ignored by .Nm that do not require it and consumed gracefully by filters that do. @@ -514,7 +513,7 @@ filter|0.5|1576146008.006103|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d filter|0.5|1576146008.006105|smtp-in|data-line|7641df9771b4ed00|1ef1c203cc576e5d|. .Ed .Pp -They are expected to produce an output stream similarly terminate by a single +They are expected to produce an output stream similarly terminated by a single dot. A filter may inject, suppress,
Re: Questions About Filters
Hello Gilles, Thank you for your response. With regards to the question about negating ‘rdns’, can I explain how I understand it and maybe you can confirm either way. Using the example, filter f01 phase connect match !rdns disconnect "550 missing rDNS" On connection (connect) we check (match) if reverse dns is invalid (!rdns) and if so then disconnect the session. For invalid, this would mean no reverse dns, or an incorrectly configured reverse dns. I did read the man page about ’smtpd.conf’, more so about filters and wanted be sure I comprehend it. Thanks Nino > On 4 Jan 2020, at 11:07 am, gil...@poolp.org wrote: > > January 4, 2020 12:25 AM, "Antonino Sidoti" wrote: > >> Hello, >> > > Hello, > > >> I have some basic questions about filters? >> >> What do we need to negate the rdns for the following command? >> >> filter f01 phase connect match !rdns disconnect "550 missing rDNS” >> > > I'm unsure I understand this question, the example you show negates rdns, > this is what I use myself to junk incoming sessions without rdns. > > >> Can someone please explain the difference between reject and disconnect when >> used in a filter? >> > > Very simple. > > When you use `reject` the command is rejected but the session isn't > disconnected. > If a client had multiple mails for you, rejecting a mail can allow it to > submit a > different mail before it gets disconnected. > > When you use `disconnect` the client gets disconnected after the rejection, > so it > has to connect again. > > >> Many thanks >> >> Nino >
Re: Questions About Filters
January 4, 2020 12:25 AM, "Antonino Sidoti" wrote: > Hello, > Hello, > I have some basic questions about filters? > > What do we need to negate the rdns for the following command? > > filter f01 phase connect match !rdns disconnect "550 missing rDNS” > I'm unsure I understand this question, the example you show negates rdns, this is what I use myself to junk incoming sessions without rdns. > Can someone please explain the difference between reject and disconnect when > used in a filter? > Very simple. When you use `reject` the command is rejected but the session isn't disconnected. If a client had multiple mails for you, rejecting a mail can allow it to submit a different mail before it gets disconnected. When you use `disconnect` the client gets disconnected after the rejection, so it has to connect again. > Many thanks > > Nino
Questions About Filters
Hello, I have some basic questions about filters? What do we need to negate the rdns for the following command? filter f01 phase connect match !rdns disconnect "550 missing rDNS” Can someone please explain the difference between reject and disconnect when used in a filter? Many thanks Nino
Re: Question about OpenSMTPD and Debian package and filters/spam filtering
On 8/21/19 12:50 PM, Michiel van Es wrote: I am running a small VPS with 1 GB memory with Debian 10 amd64 with OpenSMTPD (6.0.3) Hello, can you really use Buster's official opensmptd package? I tried it about 3 weeks ago and it was broken out of the box for me (can't really remember what was the issue at the moment). I had to use pinning and install stretch package.
Re: Filters and rctp-to rewrite.
On 9/9/19 7:16 PM, Reio Remma wrote: > On 09.09.2019 20:03, Giovanni Bechis wrote: >>> I'm currently using amavisd-new with the quarantine feature, but I'm >>> itching to switch to Rspamd (greylisting here I come!). >>> >> amavisd-new 2.12 has rspamd support, have you tried it ? > > Curious! I see the project has again switched hands, if you mean this one? > > https://gitlab.com/amavis/amavis > exactly, this is the new official amavis home. > I'll have a look at it. Then again with Rspamd and filter-rspamd we could cut > down a lot of complexity. >
Re: Filters and rctp-to rewrite.
On Mon, Sep 09, 2019 at 07:48:16PM +0300, Reio Remma wrote: > On 09.09.2019 18:13, Martijn van Duren wrote: > On 9/9/19 3:37 PM, Reio Remma wrote: >> Hello! >> >> Slowly digging into filters. >> >> Now I'm curious if it's possible to modify the recipient after say spam >> check in data-line? I get the impression that rewriting rcpt-to at that >> stage is impossible, but my goal would be to redirect/quarantine high >> scoring spam to a special e-mail address. >> >> Would it be doable somehow? >> >> Thanks! >> Reio >> > It is not. > > What you might be able to do is add an additional header and somehow let > an lmtp server make the decision based on the header. > > I haven't used lmtp myself, no clue if this actually works, but it's > worth investigating :-) > > Please reply to the threat if you managed to make it work. > > Thanks Martijn and Gilles for the confirmation! > > I'm currently using amavisd-new with the quarantine feature, but I'm itching > to switch to Rspamd (greylisting here I come!). Rspamd can let you interface Amavis. I did for someone I'm managing mail for so that you don't have to plug a ton of different tools directly to OpenSMTPD but let Rspamd broker these different filters and produce a result. > I'm using the quarantine to keep an eye on mails with a medium spam score so > we won't lose the occasional legit mail with a higher than normal spam > score. Additionally I can train these borderline mails correctly as > ham/spam. I will soon extend the filter-rspamd to include symbols it matched. With this, you'll be able to do delivery-time classification and move to a quarantine folder if you find the proper symbol. > I now see Rspamd has a metadata exporter feature I could probably use to > copy spammy mails to the quarantine mail address. Dunno about that > There are also Dovecot's sieve scripts. I'll have to see which work better. I'm not a big fan of Sieve, it is far more complex than it should be. I use it for Inbox -> SPAM, Spam -> Inbox training but quite franly if a classification can happen at delivery time through MDA, I'd rather let a fdm ruleset classify. -- Gilles Chehade @poolpOrg https://www.poolp.org patreon: https://www.patreon.com/gilles
Re: Filters and rctp-to rewrite.
On 09.09.2019 20:03, Giovanni Bechis wrote: I'm currently using amavisd-new with the quarantine feature, but I'm itching to switch to Rspamd (greylisting here I come!). amavisd-new 2.12 has rspamd support, have you tried it ? Curious! I see the project has again switched hands, if you mean this one? https://gitlab.com/amavis/amavis I'll have a look at it. Then again with Rspamd and filter-rspamd we could cut down a lot of complexity. Reio
Re: Filters and rctp-to rewrite.
On 9/9/19 6:48 PM, Reio Remma wrote: > On 09.09.2019 18:13, Martijn van Duren wrote: >> On 9/9/19 3:37 PM, Reio Remma wrote: >>> Hello! >>> >>> Slowly digging into filters. >>> >>> Now I'm curious if it's possible to modify the recipient after say spam >>> check in data-line? I get the impression that rewriting rcpt-to at that >>> stage is impossible, but my goal would be to redirect/quarantine high >>> scoring spam to a special e-mail address. >>> >>> Would it be doable somehow? >>> >>> Thanks! >>> Reio >>> >> It is not. >> >> What you might be able to do is add an additional header and somehow let >> an lmtp server make the decision based on the header. >> >> I haven't used lmtp myself, no clue if this actually works, but it's >> worth investigating :-) >> >> Please reply to the threat if you managed to make it work. > > Thanks Martijn and Gilles for the confirmation! > > I'm currently using amavisd-new with the quarantine feature, but I'm itching > to switch to Rspamd (greylisting here I come!). > amavisd-new 2.12 has rspamd support, have you tried it ? > I'm using the quarantine to keep an eye on mails with a medium spam score so > we won't lose the occasional legit mail with a higher than normal spam score. > Additionally I can train these borderline mails correctly as ham/spam. > > I now see Rspamd has a metadata exporter feature I could probably use to copy > spammy mails to the quarantine mail address. > > There are also Dovecot's sieve scripts. I'll have to see which work better. > > Thanks, > Reio >
Re: Filters and rctp-to rewrite.
On 09.09.2019 18:13, Martijn van Duren wrote: On 9/9/19 3:37 PM, Reio Remma wrote: Hello! Slowly digging into filters. Now I'm curious if it's possible to modify the recipient after say spam check in data-line? I get the impression that rewriting rcpt-to at that stage is impossible, but my goal would be to redirect/quarantine high scoring spam to a special e-mail address. Would it be doable somehow? Thanks! Reio It is not. What you might be able to do is add an additional header and somehow let an lmtp server make the decision based on the header. I haven't used lmtp myself, no clue if this actually works, but it's worth investigating :-) Please reply to the threat if you managed to make it work. Thanks Martijn and Gilles for the confirmation! I'm currently using amavisd-new with the quarantine feature, but I'm itching to switch to Rspamd (greylisting here I come!). I'm using the quarantine to keep an eye on mails with a medium spam score so we won't lose the occasional legit mail with a higher than normal spam score. Additionally I can train these borderline mails correctly as ham/spam. I now see Rspamd has a metadata exporter feature I could probably use to copy spammy mails to the quarantine mail address. There are also Dovecot's sieve scripts. I'll have to see which work better. Thanks, Reio
Re: Filters and rctp-to rewrite.
September 9, 2019 3:37 PM, "Reio Remma" wrote: > Hello! > Hello, > Slowly digging into filters. > > Now I'm curious if it's possible to modify the recipient after say spam check > in data-line? I get > the impression that rewriting rcpt-to at that stage is impossible, but my > goal would be to > redirect/quarantine high scoring spam to a special e-mail address. > By the time you start receiving DATA, the RCPT TO decision has already been taken in the SMTP transaction, so that would basically be a jump back in time. > Would it be doable somehow? > If you want to rewrite the RCPT TO, then not doable without your filter issuing a SMTP transaction itself by connecting and playing a session. If you want to junk (add X-Spam header), then you can simply have you filter do a buffering of DATA and prepend header on commit (what filter-rspamd does).
Filters and rctp-to rewrite.
Hello! Slowly digging into filters. Now I'm curious if it's possible to modify the recipient after say spam check in data-line? I get the impression that rewriting rcpt-to at that stage is impossible, but my goal would be to redirect/quarantine high scoring spam to a special e-mail address. Would it be doable somehow? Thanks! Reio
Re: Question about OpenSMTPD and Debian package and filters/spam filtering
> On 21 Aug 2019, at 13:58, Gilles Chehade wrote:
>
> On Wed, Aug 21, 2019 at 12:50:10PM +0200, Michiel van Es wrote:
>> Hi!
>>
>
> Hi,
>
>
>> I am running a small VPS with 1 GB memory with Debian 10 amd64 with
>> OpenSMTPD (6.0.3) for private email and am looking what my best options are
>> to limit spam.
>> I know there are some filters from Joerg
>> (https://www.mail-archive.com/misc@opensmtpd.org/msg04402.html) but am not
>> sure if these will work with my version of OpenSMTPD (I get a syntax error
>> when trying the old filter syntax).
>>
>> I can also relay everything to Amavisd/SpamAssassin but then email won???t
>> get blocked at the SMTP level, also ASSP or Rspamd is an option but they are
>> pretty resource intensive and will eat all my VPS memory ;)
>>
>> What would be my best option?
>>
>
> 6.0.3 is a fairly old version and there aren't many options available.
>
> if you're forced to stick with that version, which suffers from at least
> one denial of service as far as I know, your best option is to relay via
> something like SpamPD so it can interface with SpamAssassin, but this is
> not going to operate at SMTP level, it will happen at delivery time.
That’s interesting since Debian has a good track record of back porting
security fixes in their stable packages.
I will ask the maintainer if he applied the patch or upgraded the package to
latest version.
For now I use spampd which works fine for bayesian spam detection.
>
> there will be no way of blocking at SMTP level before next release 6.6.0
> that is going to happen in a few weeks, during October, so any option is
> going to be post delivery: either as a custom MDA, or as a relay via for
> some smtp proxy that will reinject in smtpd like the dkimproxy stuff.
I will wait for 6.6.0 ;)
>
> your best option would really be to build from source 6.4.2: it will not
> block at SMTP level but will provide mechanisms to ease interfacing with
> spamassassin or rspamd for post-SMTP handling.
>
> if you're not too easily scared, running the development version is good
> too because it's very close to release now, very stable and will not get
> much changes until October as I'm busy busy these days ;-)
Might give that a try, thanks :)
>
>
>> I like to do some DNSBL and SpamAsssassin checks if possible.
>>
>> My config if that is to any use to give some insights:
>>
>> pki server.pragmasec.nl certificate
>> "/etc/letsencrypt/live/pragmasec.nl/fullchain.pem"
>> pki server.pragmasec.nl key "/etc/letsencrypt/live/pragmasec.nl/privkey.pem"
>> listen on localhost
>> listen on eth0 port 25 tls pki server.pragmasec.nl hostname
>> server.pragmasec.nl auth-optional
>> listen on eth0 port 587 tls-require pki server.pragmasec.nl hostname
>> server.pragmasec.nl auth
>> table vdomains file:/etc/mail/domains
>> table vusers file:/etc/mail/vusers
>> expire 7d
>> limit mta inet4
>> accept from any for domain virtual deliver to mda
>> "/usr/lib/dovecot/dovecot-lda -f %{sender} -a %{rcpt}"
>> accept from local for any relay
>>
>> Cheers,
>>
>> Michiel
>>
>>
>>
>
> --
> Gilles Chehade @poolpOrg
>
> https://www.poolp.orgpatreon: https://www.patreon.com/gilles
Re: Question about OpenSMTPD and Debian package and filters/spam filtering
On Wed, Aug 21, 2019 at 12:50:10PM +0200, Michiel van Es wrote:
> Hi!
>
Hi,
> I am running a small VPS with 1 GB memory with Debian 10 amd64 with OpenSMTPD
> (6.0.3) for private email and am looking what my best options are to limit
> spam.
> I know there are some filters from Joerg
> (https://www.mail-archive.com/misc@opensmtpd.org/msg04402.html) but am not
> sure if these will work with my version of OpenSMTPD (I get a syntax error
> when trying the old filter syntax).
>
> I can also relay everything to Amavisd/SpamAssassin but then email won???t
> get blocked at the SMTP level, also ASSP or Rspamd is an option but they are
> pretty resource intensive and will eat all my VPS memory ;)
>
> What would be my best option?
>
6.0.3 is a fairly old version and there aren't many options available.
if you're forced to stick with that version, which suffers from at least
one denial of service as far as I know, your best option is to relay via
something like SpamPD so it can interface with SpamAssassin, but this is
not going to operate at SMTP level, it will happen at delivery time.
there will be no way of blocking at SMTP level before next release 6.6.0
that is going to happen in a few weeks, during October, so any option is
going to be post delivery: either as a custom MDA, or as a relay via for
some smtp proxy that will reinject in smtpd like the dkimproxy stuff.
your best option would really be to build from source 6.4.2: it will not
block at SMTP level but will provide mechanisms to ease interfacing with
spamassassin or rspamd for post-SMTP handling.
if you're not too easily scared, running the development version is good
too because it's very close to release now, very stable and will not get
much changes until October as I'm busy busy these days ;-)
> I like to do some DNSBL and SpamAsssassin checks if possible.
>
> My config if that is to any use to give some insights:
>
> pki server.pragmasec.nl certificate
> "/etc/letsencrypt/live/pragmasec.nl/fullchain.pem"
> pki server.pragmasec.nl key "/etc/letsencrypt/live/pragmasec.nl/privkey.pem"
> listen on localhost
> listen on eth0 port 25 tls pki server.pragmasec.nl hostname
> server.pragmasec.nl auth-optional
> listen on eth0 port 587 tls-require pki server.pragmasec.nl hostname
> server.pragmasec.nl auth
> table vdomains file:/etc/mail/domains
> table vusers file:/etc/mail/vusers
> expire 7d
> limit mta inet4
> accept from any for domain virtual deliver to mda
> "/usr/lib/dovecot/dovecot-lda -f %{sender} -a %{rcpt}"
> accept from local for any relay
>
> Cheers,
>
> Michiel
>
>
>
--
Gilles Chehade @poolpOrg
https://www.poolp.orgpatreon: https://www.patreon.com/gilles
Question about OpenSMTPD and Debian package and filters/spam filtering
Hi!
I am running a small VPS with 1 GB memory with Debian 10 amd64 with OpenSMTPD
(6.0.3) for private email and am looking what my best options are to limit spam.
I know there are some filters from Joerg
(https://www.mail-archive.com/misc@opensmtpd.org/msg04402.html) but am not sure
if these will work with my version of OpenSMTPD (I get a syntax error when
trying the old filter syntax).
I can also relay everything to Amavisd/SpamAssassin but then email won’t get
blocked at the SMTP level, also ASSP or Rspamd is an option but they are pretty
resource intensive and will eat all my VPS memory ;)
What would be my best option?
I like to do some DNSBL and SpamAsssassin checks if possible.
My config if that is to any use to give some insights:
pki server.pragmasec.nl certificate
"/etc/letsencrypt/live/pragmasec.nl/fullchain.pem"
pki server.pragmasec.nl key "/etc/letsencrypt/live/pragmasec.nl/privkey.pem"
listen on localhost
listen on eth0 port 25 tls pki server.pragmasec.nl hostname server.pragmasec.nl
auth-optional
listen on eth0 port 587 tls-require pki server.pragmasec.nl hostname
server.pragmasec.nl auth
table vdomains file:/etc/mail/domains
table vusers file:/etc/mail/vusers
expire 7d
limit mta inet4
accept from any for domain virtual deliver to mda
"/usr/lib/dovecot/dovecot-lda -f %{sender} -a %{rcpt}"
accept from local for any relay
Cheers,
Michiel
Re: Filters guidance request
Yes to clarify further, I bought myself another release cycle ;-) The filter code in 6.5 is voluntarily undocumented as both grammar and protocol didn't have enough time to "rest" for us to spot things which would appear better after a few months of being in use. Now that the stable release is out I will start documenting and making some of the changes I wanted so that it's "stable" in 6.6. If you're a developer, you can use filters in 6.5, you just need to be advanced enough to read code. On Sat, Apr 27, 2019 at 01:03:57AM +0200, Martijn van Duren wrote: > See https://poolp.org/, there are some things available, but as far as > I'm aware everything is still experimental. > > martijn@ > > On 4/26/19 9:45 PM, Aham Brahmasmi wrote: > > Namaste misc, > > > > I was wondering whether the absence of filters on the 6.5 [1] and > > current smtpd.conf(5) [2] manpages along with the modest OpenSMTPD 6.5.0 > > release notes [3] should be read into. > > > > In other words, are filters ready for general use? > > > > Dhanyavaad. > > > > Regards, > > ab > > [1] - https://man.openbsd.org/OpenBSD-6.5/smtpd.conf > > [2] - https://man.openbsd.org/smtpd.conf > > [3] - https://www.openbsd.org/65.html > > -|-|-|-|-|-|-|-- > > > > -- > You received this mail because you are subscribed to misc@opensmtpd.org > To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org > -- Gilles Chehade @poolpOrg https://www.poolp.org tip me: https://paypal.me/poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: Filters guidance request
See https://poolp.org/, there are some things available, but as far as I'm aware everything is still experimental. martijn@ On 4/26/19 9:45 PM, Aham Brahmasmi wrote: > Namaste misc, > > I was wondering whether the absence of filters on the 6.5 [1] and > current smtpd.conf(5) [2] manpages along with the modest OpenSMTPD 6.5.0 > release notes [3] should be read into. > > In other words, are filters ready for general use? > > Dhanyavaad. > > Regards, > ab > [1] - https://man.openbsd.org/OpenBSD-6.5/smtpd.conf > [2] - https://man.openbsd.org/smtpd.conf > [3] - https://www.openbsd.org/65.html > -|-|-|-|-|-|-|-- > -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Filters guidance request
Namaste misc, I was wondering whether the absence of filters on the 6.5 [1] and current smtpd.conf(5) [2] manpages along with the modest OpenSMTPD 6.5.0 release notes [3] should be read into. In other words, are filters ready for general use? Dhanyavaad. Regards, ab [1] - https://man.openbsd.org/OpenBSD-6.5/smtpd.conf [2] - https://man.openbsd.org/smtpd.conf [3] - https://www.openbsd.org/65.html -|-|-|-|-|-|-|-- -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: OpenSMTPD filters
Yay, Christmas is pretty early this year. ;) > Hi, > I have started committing filters support to OpenBSD today in order > to get them nice and ready for the next major release. > The only part missing at this point is DATA filtering which I'll > probably finish by the end of November. > Filters are in development meaning that keywords will change, > protocol will change, they are not intended to be used by lambda > users yet, only by developers willing to change their code every few days. > I wrote about them in an overview here: > https://poolp.org/posts/2018-11-03/opensmtpd-released-and-upcoming-filters-preview/ > Feel free to ask questions :-)b??yǢ??m?+ -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
OpenSMTPD filters
Hi,I have started committing filters support to OpenBSD today in order to get them nice and ready for the next major release.The only part missing at this point is DATA filtering which I'll probably finish by the end of November.Filters are in development meaning that keywords will change, protocol will change, they are not intended to be used by lambda users yet, only by developers willing to change their code every few days.I wrote about them in an overview here:https://poolp.org/posts/2018-11-03/opensmtpd-released-and-upcoming-filters-preview/Feel free to ask questions :-)
Re: Death of filters?
On Thu, Sep 21, 2017 at 5:09 PM, Gilles Chehade wrote: > we have something else which we will disclose shortly Cool, looking forward! -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: Death of filters?
On Thu, Sep 21, 2017 at 01:41:51PM +0200, Jason A. Donenfeld wrote: > Hey, > > I noticed you've removed support for filters in the latest CVS. Does > this mean that avenue of development is totally dead? Or do you have > something else that will be released with the next version? > development is not dead, we took a different approach to the problem so the existing code is no longer relevant and we removed it. we have something else which we will disclose shortly -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Death of filters?
Hey, I noticed you've removed support for filters in the latest CVS. Does this mean that avenue of development is totally dead? Or do you have something else that will be released with the next version? Jason -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: Simple Filters
On Wed, Aug 09, 2017 at 07:42:50PM +1000, Damian McGuckin wrote: > > Where are simple filters at? I noticed they are still experimental? Has > anything progressed recently? > > Just looking for something that will allow blocking on Sender, Recipient, > and Subject (with pattern matching for all of these). > Yes, there's active work in that area with a roadmap to include a filter layer in OpenSMTPD 6.3 (to be released around May 2018). They're likely going to be finished before we release OpenSMTPD 6.2 this year in November but too close to make it to the release. We'll start to ship them in snapshots as soon as we can though. I'll talk about that at EuroBSDCon > As a potential alternative, how much extra load is placed on the OpenSMTPD > server by lots, i.e. tens, of lines of the 'reject from ..' mechanism with > explicit sender/recipient names or domains. > not much, ruleset matching is quite fast and using SQL backend there are probably some tricks to match patterns with just a few lines anyways. -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Simple Filters
Where are simple filters at? I noticed they are still experimental? Has anything progressed recently? Just looking for something that will allow blocking on Sender, Recipient, and Subject (with pattern matching for all of these). As a potential alternative, how much extra load is placed on the OpenSMTPD server by lots, i.e. tens, of lines of the 'reject from ..' mechanism with explicit sender/recipient names or domains. Regards - Damian Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037 Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here Views & opinions here are mine and not those of any past or present employer -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: What happened to filters?
On Thu, 17 Nov 2016, Gilles Chehade wrote: Nope but you'd have better luck asking on m...@openbsd.org rather than this list which is unrelated to sendmail :-) Thanks. I will be one of the first testers of OpenSMPTD once there is a mechanism (integral to it) which can filter on RBLs. Regards - Damian Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037 Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here Views & opinions here are mine and not those of any past or present employer -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: What happened to filters?
On Thu, Nov 17, 2016 at 12:16:04PM +1100, Damian McGuckin wrote: > > I was looking to run with 'OpenSMTPD' but I really need the ability to use > RBLs. > > Anyway, if I turn off and 'smtpd' daemon by setting > > smtpd_flags=NO > > in > > /etc/rc.conf.local > > and install the 'sendmail' package, nothing, I firstly had to manually > create > > /var/log/maillog > > but sendmail does not even log anything in the file when I try to > send email. > > Has anybody had to do this? > > When installing 'sendmail' automatically play with the bits that > mailwrapper needs? > > Is there any trick to making 'sendmail' work? > Nope but you'd have better luck asking on m...@openbsd.org rather than this list which is unrelated to sendmail :-) -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: What happened to filters?
I was looking to run with 'OpenSMTPD' but I really need the ability to use RBLs. Anyway, if I turn off and 'smtpd' daemon by setting smtpd_flags=NO in /etc/rc.conf.local and install the 'sendmail' package, nothing, I firstly had to manually create /var/log/maillog but sendmail does not even log anything in the file when I try to send email. Has anybody had to do this? When installing 'sendmail' automatically play with the bits that mailwrapper needs? Is there any trick to making 'sendmail' work? Regards - Damian Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037 Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here Views & opinions here are mine and not those of any past or present employer -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: What happened to filters?
* Gilles Chehade le [02-09-2016 14:57:46 +0200]: > On Fri, Sep 02, 2016 at 01:57:18PM +0200, Thuban wrote: > > Hello, > > I just upgraded openbsd 5.9 to 6.0. As announced on the upgrade page, > > some parts of opensmtpd-extras disappeared. Okay. > > > > I'm actually very surprised to see that simple filters like regexp, or > > the very useful spamassassin are gone. > > > > This experimental API has been enabled for developers to help us find if > there are shortcomings, design errors and bugs in the filter layer so we > can make this whole mechanism stable both in terms of the interface, and > the reliability of the daemon. > > Instead, filters were written to please users primarily. > > And since there were written to please the users and not to help us make > the API better several filters actually worked-around limitations rather > than helping fix them. Some went as far as tweaking fields of structures > that were meant to be opaque to filters. > > Most filters were broken in one or many ways making the daemon unstable, > yet people kept telling other people to run this or that filter. Then we > receive bug reports and have to investigate if they are because of layer > or because of half-baked filter. > > The very useful spamassassin filter was also very broken, several issues > were fixed in the last few months that were specifically filter related, > and there are still a few issues with it. The API is not ready for users > and we can't cope with troubleshooting issues that aren't related to the > API itself but with individual filters. > > To make it crystal clear: > >ANY setup running with filters is broken. > > Some setups will crash, others will leak descriptors, others will block, > and some will simply work fine as long as you don't enter an error path. > > Some filters may work, I don't know, all the ones I read were bad. > > > > Anyway, is there any plan to include them in opensmtpd-extras in the > > future? > > > > Yes, when we have solved some of the shortcomings with the API and we're > happy enough with it that we actually start writing filters. > > Today, we need to be able to learn what's wrong with the API and make as > many changes as we want without people yelling that their setup is kaput > which is not what's been happening lately. > > > > How can I replace these filters? > > > > spamassassin can be used with spampd as a proxy, I don't know if there's > a similar tool for regex. > > Thank you for these very interesting explanations. I'll find my way with spampd. Thank you again an dkeep on the good work. Regards, thuban signature.asc Description: PGP signature
Re: What happened to filters?
On Fri, Sep 02, 2016 at 01:57:18PM +0200, Thuban wrote: > Hello, > I just upgraded openbsd 5.9 to 6.0. As announced on the upgrade page, > some parts of opensmtpd-extras disappeared. Okay. > > I'm actually very surprised to see that simple filters like regexp, or > the very useful spamassassin are gone. > This experimental API has been enabled for developers to help us find if there are shortcomings, design errors and bugs in the filter layer so we can make this whole mechanism stable both in terms of the interface, and the reliability of the daemon. Instead, filters were written to please users primarily. And since there were written to please the users and not to help us make the API better several filters actually worked-around limitations rather than helping fix them. Some went as far as tweaking fields of structures that were meant to be opaque to filters. Most filters were broken in one or many ways making the daemon unstable, yet people kept telling other people to run this or that filter. Then we receive bug reports and have to investigate if they are because of layer or because of half-baked filter. The very useful spamassassin filter was also very broken, several issues were fixed in the last few months that were specifically filter related, and there are still a few issues with it. The API is not ready for users and we can't cope with troubleshooting issues that aren't related to the API itself but with individual filters. To make it crystal clear: ANY setup running with filters is broken. Some setups will crash, others will leak descriptors, others will block, and some will simply work fine as long as you don't enter an error path. Some filters may work, I don't know, all the ones I read were bad. > Anyway, is there any plan to include them in opensmtpd-extras in the > future? > Yes, when we have solved some of the shortcomings with the API and we're happy enough with it that we actually start writing filters. Today, we need to be able to learn what's wrong with the API and make as many changes as we want without people yelling that their setup is kaput which is not what's been happening lately. > How can I replace these filters? > spamassassin can be used with spampd as a proxy, I don't know if there's a similar tool for regex. -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: What happened to filters?
* Thuban [160902 13:57]: > Hello, > I just upgraded openbsd 5.9 to 6.0. As announced on the upgrade page, > some parts of opensmtpd-extras disappeared. Okay. > > I'm actually very surprised to see that simple filters like regexp, or > the very useful spamassassin are gone. > > Anyway, is there any plan to include them in opensmtpd-extras in the > future? > How can I replace these filters? As for SA I am using spampd from ports and some relaying rules. HTH, Christian -- May you be peaceful, may you live in safety, may you be free from suffering, and may you live with ease. -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
What happened to filters?
Hello, I just upgraded openbsd 5.9 to 6.0. As announced on the upgrade page, some parts of opensmtpd-extras disappeared. Okay. I'm actually very surprised to see that simple filters like regexp, or the very useful spamassassin are gone. Anyway, is there any plan to include them in opensmtpd-extras in the future? How can I replace these filters? Regards, -- /Thuban/ signature.asc Description: PGP signature
about filters
Ehlo, With last release we made the experimental filters API available so that developers could start writing filters, spot shortcomings in the API and let people test them to try to spot bugs we missed. As we expected, several issues were spotted and we work on fixing them. This is all great but lately a trend has emerged: People find a filter that does what they want and they completely ignore the fact that they are experimental because it solves an immediate need. We could care less, but now we see more and more situations where a user has a question regarding a feature and someone will just tell them about this or that filter which "solves" their issue. Now that filters have been packaged by third-parties, it gets worse. New users may be told by someone to run filter-foobar, they will find it in their package manager, install it, and end up with an unstable system without even knowing they're running experimental code... So... We don't want to disable filters, we're at a stage where it needs to run on some *aware* user machines, and we can't simply stop publishing as we now have lots of users who would experience a regression on their setup. I have fixed the configure to make it VERY obvious for packagers that an add-on is not meant to be used on a stable setup. I have also added a very annoying runtime warning that will trigger when a filter is called and which will make it clear in logs that a filter is being used despite being experimental. I hope it helps reduce the wreck that my mailbox has become :-) -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: [Filters] share data between callbacks
On Wed, Mar 23, 2016 at 08:54:55PM +0100, frit...@alokat.org wrote: > On Wed, Mar 23, 2016 at 09:49:24AM +0100, Gilles Chehade wrote: > > On Wed, Mar 23, 2016 at 12:21:03AM +0100, Joerg Jung wrote: > > > On Tue, Mar 22, 2016 at 10:57:00PM +0100, frit...@alokat.org wrote: > > > > Hi, > > > > > > > > is it possible to share data between callback functions in a > > > > python-based filter? > > > > I'm looking for something like this one (from the clamav filter): > > > > - filter_api_set_udata > > > > - filter_api_get_udata > > > > > > I think you could just create your own python dict which associates user > > > data with the given message ID. > > > > > > > yes, that's what I do myself, i use the on_connect/on_disconnect to > > store a per-session dictionnary where i can store stuff I want to > > retain between callbacks > > > > Thanks, I'll give it a try. > > Btw: why do I not see the filter log calls (e.g. from filter-trace), if I use > $ echo "foo" | mail -s "bar" > but I can see how the mail is processed. > > If I use: > $ telnet localhost 25 > and do all the staff manually, I see the filter's output. Yes, expected. Local enqueued mail usually bypasses filters. You may want to read about "listen on socket" in smtpd.conf(5) to change that behavior. > --f. > > > -- > > Gilles Chehade > > > > https://www.poolp.org @poolpOrg > > > > -- > You received this mail because you are subscribed to misc@opensmtpd.org > To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org > -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: [Filters] share data between callbacks
On Wed, Mar 23, 2016 at 09:49:24AM +0100, Gilles Chehade wrote: > On Wed, Mar 23, 2016 at 12:21:03AM +0100, Joerg Jung wrote: > > On Tue, Mar 22, 2016 at 10:57:00PM +0100, frit...@alokat.org wrote: > > > Hi, > > > > > > is it possible to share data between callback functions in a python-based > > > filter? > > > I'm looking for something like this one (from the clamav filter): > > > - filter_api_set_udata > > > - filter_api_get_udata > > > > I think you could just create your own python dict which associates user > > data with the given message ID. > > > > yes, that's what I do myself, i use the on_connect/on_disconnect to > store a per-session dictionnary where i can store stuff I want to > retain between callbacks > Thanks, I'll give it a try. Btw: why do I not see the filter log calls (e.g. from filter-trace), if I use $ echo "foo" | mail -s "bar" but I can see how the mail is processed. If I use: $ telnet localhost 25 and do all the staff manually, I see the filter's output. --f. > -- > Gilles Chehade > > https://www.poolp.org @poolpOrg > -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: [Filters] share data between callbacks
On Wed, Mar 23, 2016 at 12:21:03AM +0100, Joerg Jung wrote: > On Tue, Mar 22, 2016 at 10:57:00PM +0100, frit...@alokat.org wrote: > > Hi, > > > > is it possible to share data between callback functions in a python-based > > filter? > > I'm looking for something like this one (from the clamav filter): > > - filter_api_set_udata > > - filter_api_get_udata > > I think you could just create your own python dict which associates user > data with the given message ID. > yes, that's what I do myself, i use the on_connect/on_disconnect to store a per-session dictionnary where i can store stuff I want to retain between callbacks -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: [Filters] share data between callbacks
On Tue, Mar 22, 2016 at 10:57:00PM +0100, frit...@alokat.org wrote: > Hi, > > is it possible to share data between callback functions in a python-based > filter? > I'm looking for something like this one (from the clamav filter): > - filter_api_set_udata > - filter_api_get_udata I think you could just create your own python dict which associates user data with the given message ID. > If it's not, how can I compile a C-based filter? > gcc(1) says: smtpd-api.h no such file or directory. According to filter_api(3) > it has to be included. > > --f. > > -- > You received this mail because you are subscribed to misc@opensmtpd.org > To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org > -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
[Filters] share data between callbacks
Hi, is it possible to share data between callback functions in a python-based filter? I'm looking for something like this one (from the clamav filter): - filter_api_set_udata - filter_api_get_udata If it's not, how can I compile a C-based filter? gcc(1) says: smtpd-api.h no such file or directory. According to filter_api(3) it has to be included. --f. -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: Filters
On Sat, Jan 02, 2016 at 04:44:09PM +1100, Damian McGuckin wrote: > > I would like to read something before 'playing' is done. Old, but still mostly valid: https://poolp.org/0xa871/The-state-of-filters Also, all you need can be found in man pages and the most recent opensmtpd-extras comes with man pages. -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: Filters
On Sat, 2 Jan 2016, Sunil Nimmagadda wrote: Filters require you to be -current. On OpenBSD the simplest way to test... Thanks. That's extremely very useful. That was never mentioned anywhere. I'll wait until 5.9 comes out. We never run '-current' to protect sites. Regards - Damian Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037 Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here Views & opinions here are mine and not those of any past or present employer -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: Filters
> On Sat, 2 Jan 2016, Sunil Nimmagadda wrote: > > >> Also, in the package you mention, there appears to be no documentation > >> about how to use the filters. > > > > man smptd.conf, specifically these sections... > > filter name filter [arguments] > > listen on [filter name] ... > > I pulled down 5.7.3, from October 2015, from the web-site and there is no > > filter name filter [arguments] > > nor any examples. > > >> Where are the source files for these extra packages? Is there some > >> documentation there, either explicitly or embedded in the sources? > > > > https://github.com/OpenSMTPD/OpenSMTPD-extras/tree/master/extras/wip/filter s > > > > All the filters have a manpage. > > What is the simplest way to use 'git' to pull down the lot. > > The tarballs on the web site and the CVS stuff seems way out of date. Filters require you to be -current. On OpenBSD the simplest way to test... 1. Install a recent snapshot. 2. pkg_add opensmtpd-extras. 3. smtpd.conf similar to... ... filter dnsbl dnsbl listen on ... filter dnsbl ... ... 4. # smtpd -dv -Tfilters (and share logs if you encounter any errors). The portable version is actively worked on to fix a few minor issues; the best way to test it on other platforms is to wait for a snapshot to be published in coming days. -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: Filters
On Sat, 2 Jan 2016, Sunil Nimmagadda wrote: Also, in the package you mention, there appears to be no documentation about how to use the filters. man smptd.conf, specifically these sections... filter name filter [arguments] listen on [filter name] ... I pulled down 5.7.3, from October 2015, from the web-site and there is no filter name filter [arguments] nor any examples. Where are the source files for these extra packages? Is there some documentation there, either explicitly or embedded in the sources? https://github.com/OpenSMTPD/OpenSMTPD-extras/tree/master/extras/wip/filters All the filters have a manpage. What is the simplest way to use 'git' to pull down the lot. The tarballs on the web site and the CVS stuff seems way out of date. Regards - Damian Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037 Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here Views & opinions here are mine and not those of any past or present employer -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: Filters
> > Hi guys, > > If you do not mind, I will answer both in one. > > Giovanni: > > On Sat, 2 Jan 2016, Giovanni .. wrote: > > > You should at least pkg_add opensmtpd-extras to start playing with > > filters. In base there is only the opensmtpd core implementation. > > Yes. I noticed that. But on the web-site, > > www.opensmtpd.org > > there is absolutely nothing about opensmtpd-extras. > > Also, in the package you mention, there appears to be no documentation > about how to use the filters. man smptd.conf, specifically these sections... filter name filter [arguments] listen on [filter name] ... > > I have not installed it because I do not know what it will do to my little > OpenBSD 5.8 test system. At this time, I am trying to budget people-time > so I would like to read something before 'playing' is done. > > I do note that 'parse.y' in the base tree lexically analyses a grammar > which is far more extensive than that which is documented in the man > pages. > > Where are the source files for these extra packages? Is there some > documentation there, either explicitly or embedded in the sources? https://github.com/OpenSMTPD/OpenSMTPD-extras/tree/master/extras/wip/filters All the filters have a manpage. -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
RE: Filters
Hi guys, If you do not mind, I will answer both in one. Giovanni: On Sat, 2 Jan 2016, Giovanni .. wrote: You should at least pkg_add opensmtpd-extras to start playing with filters. In base there is only the opensmtpd core implementation. Yes. I noticed that. But on the web-site, www.opensmtpd.org there is absolutely nothing about opensmtpd-extras. Also, in the package you mention, there appears to be no documentation about how to use the filters. I have not installed it because I do not know what it will do to my little OpenBSD 5.8 test system. At this time, I am trying to budget people-time so I would like to read something before 'playing' is done. I do note that 'parse.y' in the base tree lexically analyses a grammar which is far more extensive than that which is documented in the man pages. Where are the source files for these extra packages? Is there some documentation there, either explicitly or embedded in the sources? Mohammad: On Sat, 2 Jan 2016, Mohammad wrote: I have just finished building a FreeBSD mail server with OpenSMTPD, ClamAV, amavisd-new and SpamAssassin. The reason I managed to do it with OpenSMTPD and didn't need Postfix is this replacement to RBLs http://bgp-spamd.net/index.html I think it's a better alternative to RBLs since your MTA doesn't have to do a DNS query for every incoming message. I actually agree. Hessler's and Beck's approach using BGP is an elegant solution. However, we have been using MAPS RBLs for nearly 2 decades. After several buyouts, MAPS is now owned by Trend and called an ERS, email reputation service. These are accepted by our users so changing, at least for now, is not an option. Also, too many of my clients use the same ERS so we need to be using the same as them. BTW, I noticed that you used SpamAssassin, something that has hooks for DNS BL processing, the generic term for a MAPS Realtime Black List. I want to avoid SpamAssassin for simplicity, at least on the OpenBSD system (see below). So while your suggestion is a good one, other criteria rule it out as an alternative for now. Thanks - Damian Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037 Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here Views & opinions here are mine and not those of any past or present employer -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
RE: Filters
-Original Message- From: Giovanni Bechis [mailto:giova...@paclan.it] Sent: Friday, January 1, 2016 12:34 PM To: Damian McGuckin ; misc@opensmtpd.org Subject: Re: Filters Il 1 gennaio 2016 08:09:17 CET, Damian McGuckin ha scritto: > >Hi everybody, > >New to this list although I have been using OpenBSD thought since 2.1. >Contributed hardware (long ago) to the OpenBSD project. > >I am trying to figure out how to use DNS BLs with OpenSMTPD. Until I >can do that, I do not want to deploy it. > >I can see the API code in the source try but not the instructions on >how to use it. > >I found the document 'opensmtpd-LinuxCon2015.conf' by Giovanni Bechis >and it seems to imply that filters are operational. In fact, it says > > "there are filters available for dnsbl, regex matching, > Spam Assassin, and Clamav integration and much more" > >However, except for some limited images in the presentation, I cannot >find any documentation. Even if I grep 'dnsbl' in the current release, >nothing is there. > >Any pointers as to where I can find this information? > >Thanks - Damian > >Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW >2037 >Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted >here Views & opinions here are mine and not those of any past or >present employer You should at least pkg_add opensmtpd-extras to start playing with filters. In base there is only the opensmtpd core implementation. Cheers Giovanni -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org I have just finished building a FreeBSD mail server with OpenSMTPD, ClamAV, amavisd-new and SpamAssassin. The reason I managed to do it with OpenSMTPD and didn't need Postfix is this replacement to RBLs http://bgp-spamd.net/index.html I think it's a better alternative to RBLs since your MTA doesn't have to do a DNS query for every incoming message. HTH b��yǢ��m�+&j)[yƮ�쨹���r��y�h�+ kiv��N�r��zǧu���[h�+ ��칻 �&ޢ���kiv��
Re: Filters
Il 1 gennaio 2016 08:09:17 CET, Damian McGuckin ha scritto: > >Hi everybody, > >New to this list although I have been using OpenBSD thought since 2.1. >Contributed hardware (long ago) to the OpenBSD project. > >I am trying to figure out how to use DNS BLs with OpenSMTPD. Until I >can >do that, I do not want to deploy it. > >I can see the API code in the source try but not the instructions on >how >to use it. > >I found the document 'opensmtpd-LinuxCon2015.conf' by Giovanni Bechis >and >it seems to imply that filters are operational. In fact, it says > > "there are filters available for dnsbl, regex matching, > Spam Assassin, and Clamav integration and much more" > >However, except for some limited images in the presentation, I cannot >find >any documentation. Even if I grep 'dnsbl' in the current release, >nothing >is there. > >Any pointers as to where I can find this information? > >Thanks - Damian > >Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW >2037 >Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted >here >Views & opinions here are mine and not those of any past or present >employer You should at least pkg_add opensmtpd-extras to start playing with filters. In base there is only the opensmtpd core implementation. Cheers Giovanni -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Filters
Hi everybody, New to this list although I have been using OpenBSD thought since 2.1. Contributed hardware (long ago) to the OpenBSD project. I am trying to figure out how to use DNS BLs with OpenSMTPD. Until I can do that, I do not want to deploy it. I can see the API code in the source try but not the instructions on how to use it. I found the document 'opensmtpd-LinuxCon2015.conf' by Giovanni Bechis and it seems to imply that filters are operational. In fact, it says "there are filters available for dnsbl, regex matching, Spam Assassin, and Clamav integration and much more" However, except for some limited images in the presentation, I cannot find any documentation. Even if I grep 'dnsbl' in the current release, nothing is there. Any pointers as to where I can find this information? Thanks - Damian Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037 Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here Views & opinions here are mine and not those of any past or present employer -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
opensmptd-extra filters
Hello. Ist there any good beginner howto the filter in the add-on package should work? Thx. Martin smime.p7s Description: S/MIME Cryptographic Signature
Re: logging & filters
> smtpd -d gives me this: > [...] > info: filter-clamav: result stream: Eicar-Test-Signature FOUND > warn: clamav_filter: on_eom: REJECT virus id=44fa746c81ec2474 > [...] > But i can't get any logging from it in syslog. Is that by design, or > am i missing something obvious? For future reference: It's by design. Since i like having clamav messages in my logs this helped: # diff filter_api.c.orig filter_api.c 767c767 < log_init(-1); --- > log_init(0); -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: logging & filters
On Fri, Jul 10, 2015 at 08:01:52AM BST, Pete wrote:
> >> But i can't get any logging from it in syslog. Is that by design, or
> >> am i missing something obvious?
>
> > Yes, you are :^)
>
> > man 5 syslog.conf
>
> > Hint: facility, level and prog.
>
> Well, probably not something _that_ obvious. ;)
> Even with below config nothing turns up.
> But when looking at the code, even mail.info should display it.
How so? Nowhere does it state that it uses 'mail' facility.
> filter_clamav.c:
> [...]
> log_info("info: filter-clamav: result %s", l);
>
> I need a bigger cluestick.
The example above only mentions the 'info' *level*. What you're after is
the 'prog'.
> /etc/syslog.conf:
> *.notice;auth,authpriv,cron,ftp,kern,lpr,mail,user.none /var/log/messages
> kern.debug;syslog,user.info /var/log/messages
> auth.info /var/log/authlog
> authpriv.debug /var/log/secure
> cron.info /var/cron/log
> daemon.info /var/log/daemon
> ftp.info/var/log/xferlog
> lpr.debug /var/log/lpd-errs
> mail.info /var/log/maillog
> mail.* /var/log/maillog.debug
> #uucp.info /var/log/uucp
> *.* /var/log/messages.all
Obviously, it also depends on how your 'filter-clamav' is
configured - if logging is enabled at all then, based on the above
'/etc/syslog.conf', your logs should end up in '/var/log/messages.all'.
Regards,
Raf
--
You received this mail because you are subscribed to misc@opensmtpd.org
To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: logging & filters
>> But i can't get any logging from it in syslog. Is that by design, or
>> am i missing something obvious?
> Yes, you are :^)
> man 5 syslog.conf
> Hint: facility, level and prog.
Well, probably not something _that_ obvious. ;)
Even with below config nothing turns up.
But when looking at the code, even mail.info should display it.
filter_clamav.c:
[...]
log_info("info: filter-clamav: result %s", l);
I need a bigger cluestick.
/etc/syslog.conf:
*.notice;auth,authpriv,cron,ftp,kern,lpr,mail,user.none /var/log/messages
kern.debug;syslog,user.info /var/log/messages
auth.info /var/log/authlog
authpriv.debug /var/log/secure
cron.info /var/cron/log
daemon.info /var/log/daemon
ftp.info/var/log/xferlog
lpr.debug /var/log/lpd-errs
mail.info /var/log/maillog
mail.* /var/log/maillog.debug
#uucp.info /var/log/uucp
*.* /var/log/messages.all
--
You received this mail because you are subscribed to misc@opensmtpd.org
To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: logging & filters
On Thu, Jul 09, 2015 at 03:16:54PM BST, z...@gmx.net wrote: > Hi, > > I have filter-clamav (and others) plugged into 5.7.1 (OpenBSD 5.7) and > it's working fine so far. Thanks for that, it's awesome. > > smtpd -d gives me this: > [...] > info: filter-clamav: result stream: Eicar-Test-Signature FOUND > warn: clamav_filter: on_eom: REJECT virus id=44fa746c81ec2474 > [...] > > But i can't get any logging from it in syslog. Is that by design, or > am i missing something obvious? Yes, you are :^) man 5 syslog.conf Hint: facility, level and prog. > syslog.conf > [...] > mail.info/var/log/maillog > [...] Regards, Raf -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
logging & filters
Hi, I have filter-clamav (and others) plugged into 5.7.1 (OpenBSD 5.7) and it's working fine so far. Thanks for that, it's awesome. smtpd -d gives me this: [...] info: filter-clamav: result stream: Eicar-Test-Signature FOUND warn: clamav_filter: on_eom: REJECT virus id=44fa746c81ec2474 [...] But i can't get any logging from it in syslog. Is that by design, or am i missing something obvious? syslog.conf [...] mail.info/var/log/maillog [...] -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
the state of filters
ohai, I know some of you are _VERY_ interested in filters, so I wrote a blurb on my blog to discuss the state of filters in OpenSMTPD. https://poolp.org/0xa871/The-state-of-filters Feel free to react here ;-) -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
smtpd extra filters
Hello misc@, I'm trying to set up the filter-dkim-signer from the OpenSMTPD-extras repository on my OpenBSD 5.5 machine. Unfortunately I can't seem to find the instructions to hook it up in my smtpd.conf. It would be much appreciated if someone could inform me on how to set this up, or at least point me in the right direction. Sincerely, Martijn van Duren -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: wanna help with filters ?
On Mon, Apr 14, 2014 at 04:20:27PM +0200, Gilles Chehade wrote: > Ohai, > > In June, we will be having a private hackathon with Charles and Eric, to > work on the filter API and infrastructure. This will happen at my place, > it's kind of unofficial and it's unrelated to OpenBSD's hackathons. > > [...] > > From JUN-16 to JUN-22, you can join us on #OpenSMTPD @ irc.freenode.net Just clarifying something as a couple people misunderstood: - Charles, Eric and I will be having a hackthon at my place; - You guys are invited to participate on IRC during our hackathon; I can't possibly invite everyone to come over to my huge mansion :-) But it's nice to seee that there is interest in an IRL event so I'll try to keep that in mind for when we're bigger -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: wanna help with filters ?
Count me in. Bryan -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: wanna help with filters ?
On Mon, Apr 14, 2014 at 06:18:59PM +0200, Mathieu - wrote: > You can count me in. Been quite busy these days, hence I couldn't help, > it will be a good way to step in again. > > Btw, do you have ideas of filters or will we have to come up with some? > you will have to come up with some ;-) some people who have been sitting in the channel have already mentionned several ideas in the past and they may not all have the skills to write these filters so unless you have an idea, probably someone can shoot an idea at you. -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: wanna help with filters ?
You can count me in. Been quite busy these days, hence I couldn't help, it will be a good way to step in again. Btw, do you have ideas of filters or will we have to come up with some? -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: wanna help with filters ?
Count me in. -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: wanna help with filters ?
On Mon, Apr 14, 2014 at 04:20:27PM +0200, Gilles Chehade wrote: > In June, we will be having a private hackathon with Charles and Eric, to > work on the filter API and infrastructure. This will happen at my place, > it's kind of unofficial and it's unrelated to OpenBSD's hackathons. > [...] > > Let us know by replying to this mail if you intend to participate, so we > have an idea how many people would be joining us ;-) > As I mistakenly cross-posted to m...@openbsd.org (I fat-fingered the mutt command) I know C and Lua pretty well, and can help out for some of that time. My schedule is dependent on things I can plan for yet, but I'll check IRC and announce my availability sometime during that time period. -- jdv -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: wanna help with filters ?
ALL RIGHT now time to wesel out of work! On Mon, Apr 14, 2014 at 7:20 AM, Gilles Chehade wrote: > Ohai, > > In June, we will be having a private hackathon with Charles and Eric, to > work on the filter API and infrastructure. This will happen at my place, > it's kind of unofficial and it's unrelated to OpenBSD's hackathons. > > Why do you care ? > > If you want filters support in OpenSMTPD, well ... >this is a great opportunity for you to actually help us ! > > During this hackathon, we intend to make the API usable by the public so > we kind of need the public to make sure we're doing it right ;-) > > How do you help ? > > From JUN-16 to JUN-22, you can join us on #OpenSMTPD @ irc.freenode.net > > mark these dates > > If you know C, Perl, Python or Lua, you can write filters with us, then > we can spot functions that we need to add to the API to make your lives > simpler and we can spot bugs in our API that make your filters explode. > If you have questions with the API, we'll help. > > > Don't know how to code ? > > You can help us in other ways, by testing filters written by those that > can code, by testing snapshots we publish and make sure that they don't > break when run without filters, etc ... > > > What's in for you ? > > Filters obviously ... > ... and _maybe_ some goodies for some of you ! > > > Let us know by replying to this mail if you intend to participate, so we > have an idea how many people would be joining us ;-) > > -- > Gilles Chehade > > https://www.poolp.org @poolpOrg > > -- > You received this mail because you are subscribed to misc@opensmtpd.org > To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org > > -- Jason Barbier Pro Patria Vigilans
wanna help with filters ?
Ohai, In June, we will be having a private hackathon with Charles and Eric, to work on the filter API and infrastructure. This will happen at my place, it's kind of unofficial and it's unrelated to OpenBSD's hackathons. Why do you care ? If you want filters support in OpenSMTPD, well ... this is a great opportunity for you to actually help us ! During this hackathon, we intend to make the API usable by the public so we kind of need the public to make sure we're doing it right ;-) How do you help ? >From JUN-16 to JUN-22, you can join us on #OpenSMTPD @ irc.freenode.net mark these dates If you know C, Perl, Python or Lua, you can write filters with us, then we can spot functions that we need to add to the API to make your lives simpler and we can spot bugs in our API that make your filters explode. If you have questions with the API, we'll help. Don't know how to code ? You can help us in other ways, by testing filters written by those that can code, by testing snapshots we publish and make sure that they don't break when run without filters, etc ... What's in for you ? Filters obviously ... ... and _maybe_ some goodies for some of you ! Let us know by replying to this mail if you intend to participate, so we have an idea how many people would be joining us ;-) -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this mail because you are subscribed to misc@opensmtpd.org To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
Re: Intro and filters
Hi, On Thu, May 30, 2013 at 12:09:55AM +0200, Beat Vontobel wrote: > Hi, > > a few weeks ago I finally got around to test OpenSMTPD and run a > server since (actually two, the gateway relaying to my internal > instance) for the private account I'm sending this message from. So > far it has been a very pleasant "just works" experience! > Main goal achieved then ! > One of the few bugs I hit (reject with senders) has been fixed in > 5.3.2 literally the very moment I was scratching my head on why it > wasn't working, returned to the website to maybe find some info or > check out the latest version from CVS, just in time to see the new > release announced. :) Thanks to the devs for the great work! > You're welcome ;-) > After running it successfully for some time, I browsed the sources > today and discovered the very clean filter API with some example > code. However, it's not mentioned in the man pages and even a "grep > filter" hardly returns anything from the build/make scripts. What's > the easiest way to build with filters to already play around with them > a little bit? (I'm currently using the portable version on Linux.) > Filters are not quite ready, the API is not stable (functions will come, change and go) and anything you'd write today may not work next week. We don't perform the reliability checks with this code enabled so we do not even know if a smtpd instance can last an hour with it running (we have not run it enough to know if there isn't a huge memory / fd leak). For that reason, the feature is not documented yet and you can't enable it without touching some code to "plug" the mechanism. Obviously, I will not tell how to enable it because by experience this means that several people will just plug it and assume it to work, asking us tons of questions about why this and that ;-) For what it's worth, besides "externalized" scheduler, filters is the next "big" thing we have in our pipe. -- Gilles Chehade https://www.poolp.org @poolpOrg -- You received this email because you are subscribed to mailing list: misc@opensmtpd.org To unsubscribe, send mail with subject: [misc@opensmtpd.org] unregister
Intro and filters
Hi, a few weeks ago I finally got around to test OpenSMTPD and run a server since (actually two, the gateway relaying to my internal instance) for the private account I'm sending this message from. So far it has been a very pleasant "just works" experience! One of the few bugs I hit (reject with senders) has been fixed in 5.3.2 literally the very moment I was scratching my head on why it wasn't working, returned to the website to maybe find some info or check out the latest version from CVS, just in time to see the new release announced. :) Thanks to the devs for the great work! After running it successfully for some time, I browsed the sources today and discovered the very clean filter API with some example code. However, it's not mentioned in the man pages and even a "grep filter" hardly returns anything from the build/make scripts. What's the easiest way to build with filters to already play around with them a little bit? (I'm currently using the portable version on Linux.) Cheers and thx again! Beat -- You received this email because you are subscribed to mailing list: misc@opensmtpd.org To unsubscribe, send mail with subject: [misc@opensmtpd.org] unregister
