Re: How to write a milter with access to carddav
We've successfully written milters (Google pymilter) that check a MySQL database to accept inbound mails for anyone who is in this client's CRM system. Since the postfix box is a front-end to an Exchange server, the milter triggers a FILTER relay:192.168.x.x response to pass the email to Exchange without any further processing. If your CardDav server is local, and uses MySQL as the backend, it's easier to have a milter query MySQL directly. Depending on the size of the DB, you will need to do some query optimization, get your indexes set properly, etc... that will lower the query times and not add much latency. Our milter queries a 250K row table of contacts in 10ms, which adds around 30ms latency to the inbound email based on log times. If we had to do that over http with a multi-round trip 401 authorization and / or oath, that latency would skyrocket by comparison. My two cents. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com <mailto:mich...@highpoweredhelp.com> On 03/12/2018 04:20 AM, André Rodier wrote: > Hello Bastian, > > Thanks for the advice, I will consider it. I am pretty sure to know how to do > this. > > However, because the CardDav server is on the same host, I think it should > not be an issue. > > I made a few tests, and the performances are even better than some anti-spam > milters like SpamAssassin... > > The other reason is I want to add headers to the email, and let the users > decide how they want to process personal emails. > > Perhaps I can add headers with a global sieve filter as well, and have the > same result. > > André. > > On 12 March 2018 05:20:00 GMT+00:00, Bastian Blank > <bastian+postfix-users=postfix@waldi.eu.org> wrote: >> On Fri, Mar 09, 2018 at 11:53:00AM +, André Rodier wrote: >>> I would like to know if there is any milter for postfix that would >> let >>> me query a CardDav server? >> Well, don't. Milter is latency sensitive and it will break mail >> delivery if you don't manage to get it right. >> >> If you use the correct Sieve implementation you can ask it to run stuff >> for you: >> >> https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-extprograms.txt >> >> Bastian
RE: bloc domains with all variants of tld
This is interesting, and I am curious to know the answer as well. We have an application where denying all tlds and then allowing in certain ones would be very useful. (Default reject, allow only whitelisted). Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com<mailto:mich...@highpoweredhelp.com> From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of wodel youchi Sent: Monday, November 6, 2017 4:26 AM To: postfix-us...@cloud9.net Subject: bloc domains with all variants of tld Hi, We need to bloc some incoming emails from certain domains. How to write rules to bloc a domain with all its variant of tld? if we want to bloc the example domain we write the rules like this example.com<http://example.com> REJECT example.fr<http://example.fr> REJECT example.de<http://example.de> REJECT etc... How to write one rule to express all tlds? something like exemple.* REJECT Regards. [https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif]<https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=webmail> Garanti sans virus. www.avast.com<https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=webmail>
Milter order?
*TL;DR* - my milter works, but I want it to operate /after/ smtpd_helo_restrictions, smtpd_recipient_restrictions, and smtpd_client_restrictions have done their magic because it's logging information from spam that gets filtered out by those guys (and also by spamassassin). Here's my config line: smtpd_milters = unix:/var/run/spamass/spamass.sock unix:/var/run/opendkim/opendkim.sock local:/var/run/mcdbcache/mcdb.sock Background: We are caching the to and from fields for a CRM system. Additionally, we are tagging inbound and outbound mail with an SMTP header and a tag in the body so we can associated conversations with projects and vice versa. So, I need full access to the envelope and the body (and attachments, which is another thread entirely, so let's not get sidetracked there). In re-reading the FILTER_README and MILTER_READMEs, it almost seems as if I need to change this from a milter to a filter, but that is a content filter, which I don't really want to do. I don't want to change the disposition of any email that has made it past the restrictiosn and spamassassin. Just clone it for further processing later. I want to cache / copy the emails as they come in, and do not want to filter anything. What's my next step? I am hoping there is a config that I have missed that will apply the milter /after /the restrictions have been processed... -- Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com <mailto:mich...@highpoweredhelp.com>
Re: Stopping compromised accounts
What about an outbound milter that would do nothing other than read the to and from fields, and then store the message meta data along with a timestamp in a database? You could then run queries to find the total number of emails sent per user, and an average send rate (over the whole organization or on a per-user basis). Then, aberration like a spike can be easily seen. You could choose the granularity. For faster detection of "stuff that needs to be looked at" you could do this on an hourly report basis... When something is outside parameters, the cron job that runs the queries can email you. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 12/06/2016 03:50 PM, John Fawcett wrote: > On 12/06/2016 02:52 AM, Alex wrote: >> Hi, >> >> I have a postfix-3.0.5 system with a few hundred users. They have >> access to submission, webmail, and dovecot to send and receive mail. >> >> On occasion, user's local desktop are compromised, and with it their >> account on this system. This leads to their local desktop using the >> submission service to send hundreds or thousands of spam emails >> through this compromised account. >> >> They're only stopped after the user receives a ton of bounce messages, >> or we happen to see it somehow while watching logs. >> >> What mechanisms are available to say, control the number of messages >> sent per day or otherwise be made aware of a pattern of messages being >> sent by an account that could be indicative of account compromise? >> >> Thanks, >> Alex > You could use a policy server that can do rate limiting (such as > > policyd). This will reduce the impact of the problem without > > stopping it altogether. > > I use a home grown log parser script that can trigger account > > blocking if there are too many successful logins from different > > ips in a short space of time or too many logins in general. > > It won't be for everyone (since it has PHP as a prerequisite) > > but if that's not an issue you might want to give it a try. > > http://www.voipsupport.it/wiki/index.php/CheckAuthLog > > John >
Re: What is the number means?
Linux man page numbers. http://unix.stackexchange.com/questions/3586/what-do-the-numbers-in-a-man-page-mean#3587 Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 12/02/2016 04:26 PM, Gao wrote: > Hi, > > I'd like ask a dumb question: I see there are many things in Postfix > which named as pipe(8), smtp(5), lmtp(8). So what is number 5 or 8 > mean? Version number? > > Gao >
Re: Customize log messages?
This is a great idea. This is a spam filter that is integrated into a CRM system, so I needed to parse and dump the information so it could be sucked up later. Here's what I ultimately created. It still needs some work (mainly because it re-reads the whole file every time, and I should use timestamps and a half-interval search algorithm to find the last-processed time stamp. I am relying on log rotate to make it not-too-terribly-big). https://github.com/mjmunger/postfix-log-parser Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 12/02/2016 01:30 AM, @ wrote: > On 11/30/16 2:35 PM, Michael Munger wrote: > >> I am writing a log parser so that when users complain "so and so sent me >> an email and I didn't get it" I can query the logs and find this with >> ease. Ultimately, I want ot make this self service through a web page. > I went a different way. Users can chose to receive a "DMR" (Daily Mail > Report) and that report can contain either all the rejected email > addresses that were not accepted for their account (or domain), all > the accepted emails they got, or both. > > I have a bash script that does it, and when a user wants this, I > simply set up a crontab for them. Usually after a week or so they want > it turned off. The script sends them a lightly styled HTML table in > the email. > > The heart of the script is: > > if [ "$REJECT" = 1 ]; then > echo 'IP addressClaimed > address' > bzgrep "$MATCHPAT" $LOGF | grep -i reject | egrep 'from=<[^>]+>' | > grep -v "Protocol error" | \ > grep -v "$EXCLUDE" | sort -u | sed 's/from=,[]:' | > grep -v rejected | \ > awk '{print "REJECTED class=\"right\">"$16""$20""}' > fi > > if [ "$ACCEPT" = 1 ]; then > echo 'Accepted ID style="width:6em;">TimeFrom' > bzgrep -E 'DATA|\"from=\"' $LOGF | grep -v "<>"| \ > awk '{print $6"\t"$3"\t"$17"\t"$16}' | grep -v ESMTP | \ > grep -v "to=<backup" | column -t | sort -k 2 | grep > "to=<.*$MATCHPAT" | \ > grep -v "$EXCLUDE" | sed 's/from//g' | sed 's/://' | tr -d > '=><' | > awk '{print ""$1" class=\"right\">"$2""$4""}' >fi > > For this to work > > smtpd_log_access_permit_actions = static:all > > must be set in main.cf. This makes your logs chattier, but provides me > with the line in the logs that I need to get this working. > > One user, in particular, was calling several times a week looking for > an email and now never calls. > > >
Re: Customize log messages?
On 12/01/2016 09:37 AM, Wietse Venema wrote: And I have made a note to log the sender when rejecting the (MAIL FROM) SIZE parameter. Wow. Wasn't expecting that! Thank you, sir.
Re: Customize log messages?
Bill: Thank you for both items. I shall pour over them. On 11/30/2016 11:49 PM, Bill Cole wrote: On 30 Nov 2016, at 20:20, Michael Munger wrote: First, there can be no TO address before the client sends MAIL FROM. Second, the size check is done before checking the sender address, presumably because it is more efficient that way. But I guess some code could be swapped around. My mistake. I thought: 552 5.3.4 Message size exceeds fixed limit; I did not know that the message size was declared prior to the MAIL FROM and RCPT TO commands. I figured it would come either after the DATA command or after the message was received so that it could calculate sizes. I'll rethink my strategy here. See https://tools.ietf.org/html/rfc1870 Maximum message size is advertised in the EHLO response and senders can announce message size as an extra argument to MAIL. Many SMTP clients honor the advertised maximum and simply QUIT the session if it is too small, others proceed to MAIL with the SIZE argument and get rejected explicitly there (that 552 reply.) I have been scouring the docs to determine what checks are perform when and in what order. I assume they follow the SMTP prtocol (HELO checks first, MAIL FROM checks next, RCPT TO checks next, and so on...). I found the architecture readme, which is excellent. Is there a list of which programs do which checks? I could assemble a list from there. Unless you (or someone) already had such a list? You can get most of this in the postconf(5) man page, and whatever isn't detailed there will be in the man pages for the various Postfix components. The Postfix README files are great for a high-level instructional view of how it all works, but for the full technical details you really need to look at man pages.
Re: Customize log messages?
> First, there can be no TO address before the client sends MAIL FROM. > Second, the size check is done before checking the sender address, > presumably because it is more efficient that way. But I guess some > code could be swapped around. My mistake. I thought: 552 5.3.4 Message size exceeds fixed limit; I did not know that the message size was declared prior to the MAIL FROM and RCPT TO commands. I figured it would come either after the DATA command or after the message was received so that it could calculate sizes. I'll rethink my strategy here. I have been scouring the docs to determine what checks are perform when and in what order. I assume they follow the SMTP prtocol (HELO checks first, MAIL FROM checks next, RCPT TO checks next, and so on...). I found the architecture readme, which is excellent. Is there a list of which programs do which checks? I could assemble a list from there. Unless you (or someone) already had such a list? Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com
Customize log messages?
I am writing a log parser so that when users complain "so and so sent me an email and I didn't get it" I can query the logs and find this with ease. Ultimately, I want ot make this self service through a web page. In a transaction like this: 119970-Nov 29 13:56:12 mcdb2 postfix/smtpd[12371]: disconnect from unknown[118.201.69.1] 119971-Nov 29 13:56:33 mcdb2 postfix/smtpd[12587]: connect from mail-lf0-f51.google.com[209.85.215.51] 119972:Nov 29 13:56:34 mcdb2 postfix/smtpd[12587]: NOQUEUE: reject: MAIL from mail-lf0-f51.google.com[209.85.215.51]: 552 5.3.4 Message size exceeds fixed limit; proto=ESMTP helo= 119973-Nov 29 13:56:35 mcdb2 postfix/smtpd[12587]: disconnect from mail-lf0-f51.google.com[209.85.215.51] The email is rejected without making reference to the 'from' and or 'to' addresss?. Is there a way I can configure this to include the from here? ideally, make it say: 119972:Nov 29 13:56:34 mcdb2 postfix/smtpd[12587]: NOQUEUE: reject: MAIL from mail-lf0-f51.google.com[209.85.215.51]: 552 5.3.4 Message size exceeds fixed limit; from=<someuser@gmailcom> to=<someu...@ourdomain.com> proto=ESMTP helo= -- Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com
Re: short circuit content_filters
As you have suggested, so I have done. For each of these, I have added the whitelist.cf and shortcircuit.cf files via check_sender_access as the /first thing/ these checks do so that domains on the whitelist are shortcircuited and routed without prejudice. smtpd_helo_restrictions = check_sender_access mysql:/etc/postfix/whitelist.cf, check_sender_access mysql:/etc/postfix/shortcircuit.cf, permit_mynetworks, permit_sasl_authenticated, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, reject_unknown_helo_hostname smtpd_recipient_restrictions = check_sender_access mysql:/etc/postfix/whitelist.cf, check_sender_access mysql:/etc/postfix/shortcircuit.cf, reject_invalid_hostname, reject_unauth_pipelining, reject_unknown_recipient_domain, reject_unauth_destination, permit_mynetworks, permit_sasl_authenticated, permit smtpd_client_restrictions = check_sender_access mysql:/etc/postfix/whitelist.cf, check_sender_access mysql:/etc/postfix/shortcircuit.cf, reject_rbl_client dnsbl.sorbs.net, reject_rbl_client b.barracudacentral.org, reject_rbl_client cbl.abuseat.org, reject_rbl_client bl.spamcop.net, reject_rbl_client zen.spamhaus.org, reject_rbl_client dnsbl-1.uceprotect.net, reject_rbl_client dnsrbl.org, reject_rbl_client bad.psky.me, check_sender_access mysql:/etc/postfix/blacklist.cf Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 11/28/2016 11:55 PM, Noel Jones wrote: > Don't forget to put your shortcircuit.cf in smtpd_{helo, sender, > recipient, data}_restrictions. > > Or even better, to prevent any unintentional open relay have your > shortcircuit.cf return "permit_auth_destination" rather than simply OK. > > > -- Noel Jones > > > On 11/28/2016 9:22 PM, Michael Munger wrote: >> Good timing. I was just trying to figure out why a domain on the >> whitelist was still caught by an rbl when you sent this. Thank you. >> >> To fix the issue, I have added an additional check_sender_access >> >> smtpd_client_restrictions = check_sender_access >> mysql:/etc/postfix/whitelist.cf, >> check_sender_access >> mysql:/etc/postfix/shortcircuit.cf, >> reject_rbl_client dnsbl.sorbs.net, >> reject_rbl_client b.barracudacentral.org, >> reject_rbl_client cbl.abuseat.org, >> reject_rbl_client bl.spamcop.net, >> reject_rbl_client zen.spamhaus.org, >> reject_rbl_client dnsbl-1.uceprotect.net, >> reject_rbl_client dnsrbl.org, >> reject_rbl_client bad.psky.me, >> check_sender_access >> mysql:/etc/postfix/blacklist.cf >> >> Contents of shortcircuit.cf: >> >> user = postfix >> password = foopass >> dbname = postfix >> query = SELECT 'OK' AS whitelist_action FROM whitelist WHERE >> whitelist_domain = '%s' >> hosts = 127.0.0.1 >> >> Michael Munger, dCAP, MCPS, MCNPS, MBSS >> High Powered Help, Inc. >> Microsoft Certified Professional >> Microsoft Certified Small Business Specialist >> Digium Certified Asterisk Professional >> mich...@highpoweredhelp.com >> On 11/28/2016 10:12 PM, Noel Jones wrote: >>> On 11/28/2016 8:14 PM, Michael Munger wrote: >>> >>> ... >>>> we simply changed the action >>>> from "OK" to "FILTER relay:192.168.10.81". This means that inbound >>>> mail, once it is discovered to be on a whitelist, is immediately >>>> relayed to our Exchange server. >>> No, the mail is not immediately relayed. It goes through the rest of >>> the configured restrictions, is queued, and is then relayed to the >>> next-hop specified by your FILTER statement. Of particular interest >>> is that FILTER -- similar to DUNNO -- does not skip further checking >>> such as rbl lookups or hostname restrictions. >>> >>> To both set the next-hop AND skip further postfix restriction >>> processing, you need two check_sender_access tables; the first to >>> return FILTER and the second to return OK to skip further checks. >>> >>> >>> >>> -- Noel Jones
Re: short circuit content_filters
Good timing. I was just trying to figure out why a domain on the whitelist was still caught by an rbl when you sent this. Thank you. To fix the issue, I have added an additional check_sender_access smtpd_client_restrictions = check_sender_access mysql:/etc/postfix/whitelist.cf, check_sender_access mysql:/etc/postfix/shortcircuit.cf, reject_rbl_client dnsbl.sorbs.net, reject_rbl_client b.barracudacentral.org, reject_rbl_client cbl.abuseat.org, reject_rbl_client bl.spamcop.net, reject_rbl_client zen.spamhaus.org, reject_rbl_client dnsbl-1.uceprotect.net, reject_rbl_client dnsrbl.org, reject_rbl_client bad.psky.me, check_sender_access mysql:/etc/postfix/blacklist.cf Contents of shortcircuit.cf: user = postfix password = foopass dbname = postfix query = SELECT 'OK' AS whitelist_action FROM whitelist WHERE whitelist_domain = '%s' hosts = 127.0.0.1 Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 11/28/2016 10:12 PM, Noel Jones wrote: > On 11/28/2016 8:14 PM, Michael Munger wrote: > > ... >> we simply changed the action >> from "OK" to "FILTER relay:192.168.10.81". This means that inbound >> mail, once it is discovered to be on a whitelist, is immediately >> relayed to our Exchange server. > No, the mail is not immediately relayed. It goes through the rest of > the configured restrictions, is queued, and is then relayed to the > next-hop specified by your FILTER statement. Of particular interest > is that FILTER -- similar to DUNNO -- does not skip further checking > such as rbl lookups or hostname restrictions. > > To both set the next-hop AND skip further postfix restriction > processing, you need two check_sender_access tables; the first to > return FILTER and the second to return OK to skip further checks. > > > > -- Noel Jones
Re: short circuit content_filters
> >> How do I configure Postfix to NOT use spamassassin to scan the email if >> it is coming from one of these whitelisted domains? > > Make the check_sender_access queries return a FILTER result > (specifying a suitable transport, likely 'local:$myhostname') which > will override your content_filter directive. See access(5) This is exactly what I wanted. For future searchers, here was the answer: Since FILTER (see: access(5)) must be the name of a service in master.cf, AND this is a mail gateway, we simply changed the action from "OK" to "FILTER relay:192.168.10.81". This means that inbound mail, once it is discovered to be on a whitelist, is immediately relayed to our Exchange server. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com
Re: short circuit content_filters
> Fix it in spamassassin. Use whitelist_from or better whitelist_from_dkim. > See "perldoc Mail::SpamAssassin::Conf" for config instructions. Seems legit. How do I configure SpamAssassin to look up the domain in MySQL? Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com
short circuit content_filters
TLDR: How do I configure Postfix to NOT use spamassassin to scan the email if it is coming from a domain that is whitelisted via check_sender_access? DETAILS: We maintain a white-list of company domains that are aggregated from all our employee's address books. We want all email that claims to be from those domains (even if they are spam, forged, contain cryptolocker, etc...). It's full-on, highway to the danger zone, unbridled email acceptance if your domain is on this whitelist. For the most part, the configs below work. check_sender_access queries our MySQL database and gets an "OK" for these domains. And, (I believe) once this is returned, the remainder of the checking stops for that step in the process. (Please correct me if I am wrong here, but that IS the desired behavior). We still have a few automated emails (these are order confirmations) that are getting caught by spamassassin DESPITE being on the whitelist because they are HTML only, and they talk about money if you fill out a form. And, Spamassassin is executing post-queue, where these rules do not apply post-queue, and only upon receipt. Example of a good message caught in the bad filter: Content analysis details: (2.8 points, 2.0 required) pts rule name description -- -- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [67.231.157.48 listed in wl.mailspike.net] 1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 0.0 HTML_MESSAGE BODY: HTML included in message 0.0 LOTS_OF_MONEY Huge... sums of money 0.6 HTML_MIME_NO_HTML_TAG HTML-only message, but there is no HTML tag 0.0 T_FILL_THIS_FORM_SHORT Fill in a short form with personal information 1.0 MONEY_FORM_SHORT Lots of money if you fill out a short form How do I configure Postfix to NOT use spamassassin to scan the email if it is coming from one of these whitelisted domains? Or, in the alternative, can I configure Postfix to add a header to emails that are on the whitelist, and then configure spam assassin to not check emails with that header? Relevant sections of master.cf:** # == # service type private unpriv chroot wakeup maxproc command + args # (yes) (yes) (yes) (never) (100) # == smtp inet n - - - - smtpd -o content_filter=spamassassin # ... other stuff ... # spamassassin unix - n n - - pipe user=debian-spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient} *Relevant sections for main.cf:* smtpd_recipient_restrictions = reject_invalid_hostname, reject_unauth_pipelining, reject_unknown_recipient_domain, reject_unauth_destination, permit_mynetworks, permit_sasl_authenticated, check_sender_access mysql:/etc/postfix/whitelist.cf, check_policy_service unix:private/policy-spf permit smtpd_helo_restrictions = permit_mynetworks, check_sender_access mysql:/etc/postfix/whitelist.cf, permit_sasl_authenticated, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, reject_unknown_helo_hostname # 1. We whitelist customers as best we can regardless of what a cesspool their email servers, IP address, or network neighborhood might be. # 2. Then, we let the RBLs do their job to lessen the load on the MySQL server. # 3. If the email passes all those checks, it's time to see if it is on our custom blacklists. If so, we reject, otherwise, we'll let it continue down the chain. smtpd_client_restrictions = check_sender_access mysql:/etc/postfix/whitelist.cf, reject_rbl_client dnsbl.sorbs.net, reject_rbl_client b.barracudacentral.org, reject_rbl_client cbl.abuseat.org, reject_rbl_client bl.spamcop.net, reject_rbl_client zen.spamhaus.org, reject_rbl_client dnsbl-1.uceprotect.net, reject_rbl_client ubl.unsubscore.com, reject_rbl_client dnsrbl.org, reject_rbl_client bad.psky.me, reject_rbl_client dnsbl-2.uceprotect.net check_sender_access mysql:/etc/postfix/blacklist.cf Incidentally, blacklist.cf rejects an email if it is coming from a domain that NONE of our customers have ever sent from before. This eliminates day-old-bread TLDs that are just spam factories. (.top for example is spewing forth untold mountains of spam right now based on our logs.). -- Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business
RE: OpenDKIM
If Amavis is signing your outbound mail with your (a) private key, and you've published the public key in your DNS so remote servers can verify, then you should be all set. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com -Original Message- From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of John Allen Sent: Friday, November 6, 2015 1:13 PM To: postfix-users@postfix.org Subject: OpenDKIM Is OpenDKIM worth while? I use amavis and it says it signs and verifies DKIM so do need anything else?
Tracking header?
I would like to inject a tracking header in an email before it is sent. Nothing malicious, just something I can use to uniquely identify a specific email that was sent from our server's web application. Here's the use case narrative: We build web based applications where we need to be able to audit: "Did so-and-so get an email?" When the user is outside our system, the only thing we can verify is that the recipient server on the other end responded with a 250 OK. So, I need to be able to pair that 250 OK response with a specific email that was generated out of our system. To do that, I am planning on writing a milter in python that will look at the headers (before or after it is sent, I haven't decided on that), and then figure out the post-send disposition (250 OK, 550 5.7.1 Rejected, etc...) and update the database. My current thought on how to do this would be to add a header to the email that corresponds to the id that email has in the database. Just an integer value. (THe secuirty guy in me says use a nonce instead. It could be either). So, the first part of my task is: can I add a header to the email that will safely be ignored by everything? Something like: "X-Tracking: 1234\r\n"? If the answer to that is "Yes", then my next task will be to research what hooks are available for a miter to look at the results of a sent mail so I can get the remote server response to store in the database. Does anyone have any thoughts on this? I am completely open to scrapping my current idea and adopting a better way. I have no pride in the concept. I just want something that works well and is an elegant approach. Thanks in advance, Michael -- Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com
Re: Postfix + Courier or Dovecot?
Thanks to everyone on their feedback. I'm going with Dovecot. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 06/18/2015 10:28 AM, Chris Adams wrote: Once upon a time, Eric Broch ebr...@whitehorsetc.com said: that courier chokes on large mailboxes, but I never experienced that. It always performed well for me. I recently switched an install of about 55,000 mailboxes (mostly telco/ISP customers) from Courier to Dovecot. The mail spool is on a backend accessed over NFS; the NFS network traffic dropped significantly (from around 300 megabits per second to about 20 megabits per second) after the switch. Our backend has an SSD read cache to (mostly) keep up with the load with Courier, and now with Dovecot, the SSD is essentially unused (the working set fits in the storage array RAM cache). Depending on your access patterns, Dovecot indexing can be a huge win. Also, Dovecot management is far superior. For example, we have mailbox quotas, which with Courier were just stored in a file in the Maildir. That file had to be regenerated periodically (scanning the whole mailbox), and there was no quick way to check the quota status (it took reading 55,000 files). Dovecot can store quotas in a database, so checking them now is one SELECT (that takes a fraction of a second) away.
Postfix + Courier or Dovecot?
Having Googled this extensively, and searched the postfix users archives, I have yet to come to a conclusion on which to use: courier or dovecot. I am leaning towards Dovecot because of it's large community, security focus, and a few off color emails in the archives (http://is.gd/fs49di). So the question is: which of these plays the best with Postfix? and most importantly: why? -- Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com
Re: Fwd: newbie faq - sorry: recipient lists
Martin: IMAP is managed by courier, which is a different list. So, it is not useful to reference that here except for the fact that it proves MySQL is able to look up your receiving account, which means that that account / the tables are setup properly. Your problem has nothing to do with IMAP. As Viktor said, we need logs and your main.cf file. Use pastebin to send. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 06/15/2015 09:35 AM, Martin S wrote: The setup works if I am loged in to the system, e.g. by imap through mail2web. Then I can send mail from my server to the world or between mailboxes on the server. Sending mail *FROM* your server to the world means that the recipient is in some other domain, not yours. You said it, not I. -- Yes and this works. This is not an issue. Which I also said. Sending mail to the server from the world outside results in 5.1.1 @: Recipient address rejected: User unknown in relay recipient table. Sending mail from outside *TO* your server, means that the recipient is in your domain, not someone else's. You said it, not I. -- Yes I am completely aware of this. If I send mail from my laptop (using mail address a) to mail address b (on the problem server) which is in a domain I am authorative for results in the error message. However if I am logged into the problem server using imap then sending mail from mail address c (on the problem server) which is in a domain I am authorative to mail address b works - meaning it can actually resolve the address and am able to find the address in the recipient table. /Martin S 2015-06-15 15:26 GMT+02:00 Viktor Dukhovni postfix-us...@dukhovni.org: On Mon, Jun 15, 2015 at 03:20:35PM +0200, Martin S wrote: OK maybe I was unclear. Or you're still confused or both. I am talking about a site that I am authorative for. It's my own domain. DNS points to this server. If you can't post logs that illustrate the various cases under discussion nobody can help you. The setup works if I am loged in to the system, e.g. by imap through mail2web. Then I can send mail from my server to the world or between mailboxes on the server. Sending mail *FROM* your server to the world means that the recipient is in some other domain, not yours. You said it, not I. Sending mail to the server from the world outside results in 5.1.1 @: Recipient address rejected: User unknown in relay recipient table. Sending mail from outside *TO* your server, means that the recipient is in your domain, not someone else's. You said it, not I. -- Viktor.
Re: newbie faq - sorry: recipient lists
Mar This is a different error than you were getting before. YOu were getting user unknown in recipient table before, which meant postfix could not figure out who you wanted to send mail to... or if it was even responsible for that domain / user's mail. Now, you have a problem with are you allowed to send me mail?. My first question is: did you overwrite all the config files you had before and use the ones specifically in the tutorial? I suspect not. This error comes about when the remote side is trying to authenticate with an authentication mechanism that the server side doesn't support. Here's why it works locally: When you send locally, you're sending via 127.0.0.1, which is clearly in your mynetworks (as it would be by default). This means you're doing IP authentication to send mail. Thus, it works as long as you're in your own sandbox. When you attempt to send mail from outside your machines self made sandbox of 127.0.0.1/8, then you are required to authenticate. Otherwise, spammers can use you as a relay. So, thus far, it's working properly. Now, on the remote side, when you try to authenticate, you're trying to use PLAIN (plaintext) ,and the server is rejecting that request. This tells me that your /etc/postfix/sasl/smtpd.conf file is incorrect, in valid, non-existent, or (at the very least) not setup the way the http://learnlinuxonline.com/servers/how-setup-linux-mail-server-debian-ubuntu dictates. If you double check that article, under the section How to Configure Saslauthd for our Linux Mail Server, you'll see that the second line of the configuration for the /etc/postfix/sasl/smtpd.conf line specifies: mech_list: PLAIN LOGIN Since your server is not allowing this, you need to check that out. Re-Read the section: Debugging Loggin Failures. If you are still trying to use your own configs, scrap them. Start with these tested configs, and you can modify it after you get it working. Double check every detail, and if it still doesn't work, post back here with your specific errors (error text required) as well as the relevant snipits from /var/log/syslog as applicable. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 06/14/2015 04:53 AM, Martin S wrote: OK, re-did it all using http://learnlinuxonline.com/servers/how-setup-linux-mail-server-debian-ubuntu Still having the same problem, it works locally but not from the rest of the world. E.g. using Postfix admin I can send mail to my account but using the http://www.mail2web.com service I can't access the account. Mailing from my laptop I get Your SMTP server does not support PLAIN. Choose a different authentication method. The server responded: 5.7.8 Error: authentication failed: authentication failure Going to look at it when i get home /Martin S 2015-06-13 17:36 GMT+02:00 Michael Munger mich...@highpoweredhelp.com: It's easier to do it right the first time than it us to unlock your install. http://learnlinuxonline.com/servers/how-setup-linux-mail-server-debian-ubuntu Sent from my Verizon Wireless 4G LTE smartphone Original message From: Martin S Date:06/13/2015 3:51 AM (GMT-05:00) To: postfix-us...@cloud9.net Subject: newbie faq - sorry: recipient lists hi, I've set up a mailserver (as my hosting mail is crap) using this site: http://www.server-world.info/en/note?os=CentOS_7p=httpdf=13 I am having problems with recipients. No matter how I set up my recipeints (local or virtual) I get Recipient address rejected: User unknown in relay recipient table Can someone point me to a reliable instruction for me to unfuck my installation? Regards, Martin S
Re: newbie faq - sorry: recipient lists
Check the section on the article that deals with the SQL for the virtual table, ave read the comments at the bottom as well. On June 14, 2015 2:02:02 PM EDT, Martin S shieldf...@gmail.com wrote: Hmm progress as now I'm getting 5.1.1 ADDRESS: Recipient address rejected: User unknown in relay recipient table Not finding much in log files Still on it though. /Martin S 2015-06-14 17:44 GMT+02:00 Martin S shieldf...@gmail.com: Oh my old configs are very very gone =) As the old system was Centos and the new one Debian they went the way of the Dodo with the complete reinstall ... I'll recheck the section now that I'm home. /Martin S 2015-06-14 15:57 GMT+02:00 Michael Munger mich...@highpoweredhelp.com: Mar This is a different error than you were getting before. YOu were getting user unknown in recipient table before, which meant postfix could not figure out who you wanted to send mail to... or if it was even responsible for that domain / user's mail. Now, you have a problem with are you allowed to send me mail?. My first question is: did you overwrite all the config files you had before and use the ones specifically in the tutorial? I suspect not. This error comes about when the remote side is trying to authenticate with an authentication mechanism that the server side doesn't support. Here's why it works locally: When you send locally, you're sending via 127.0.0.1, which is clearly in your mynetworks (as it would be by default). This means you're doing IP authentication to send mail. Thus, it works as long as you're in your own sandbox. When you attempt to send mail from outside your machines self made sandbox of 127.0.0.1/8, then you are required to authenticate. Otherwise, spammers can use you as a relay. So, thus far, it's working properly. Now, on the remote side, when you try to authenticate, you're trying to use PLAIN (plaintext) ,and the server is rejecting that request. This tells me that your /etc/postfix/sasl/smtpd.conf file is incorrect, in valid, non-existent, or (at the very least) not setup the way the http://learnlinuxonline.com/servers/how-setup-linux-mail-server-debian-ubuntu dictates. If you double check that article, under the section How to Configure Saslauthd for our Linux Mail Server, you'll see that the second line of the configuration for the /etc/postfix/sasl/smtpd.conf line specifies: mech_list: PLAIN LOGIN Since your server is not allowing this, you need to check that out. Re-Read the section: Debugging Loggin Failures. If you are still trying to use your own configs, scrap them. Start with these tested configs, and you can modify it after you get it working. Double check every detail, and if it still doesn't work, post back here with your specific errors (error text required) as well as the relevant snipits from /var/log/syslog as applicable. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 06/14/2015 04:53 AM, Martin S wrote: OK, re-did it all using http://learnlinuxonline.com/servers/how-setup-linux-mail-server-debian-ubuntu Still having the same problem, it works locally but not from the rest of the world. E.g. using Postfix admin I can send mail to my account but using the http://www.mail2web.com service I can't access the account. Mailing from my laptop I get Your SMTP server does not support PLAIN. Choose a different authentication method. The server responded: 5.7.8 Error: authentication failed: authentication failure Going to look at it when i get home /Martin S 2015-06-13 17:36 GMT+02:00 Michael Munger mich...@highpoweredhelp.com: It's easier to do it right the first time than it us to unlock your install. http://learnlinuxonline.com/servers/how-setup-linux-mail-server-debian-ubuntu Sent from my Verizon Wireless 4G LTE smartphone Original message From: Martin S Date:06/13/2015 3:51 AM (GMT-05:00) To: postfix-us...@cloud9.net Subject: newbie faq - sorry: recipient lists hi, I've set up a mailserver (as my hosting mail is crap) using this site: http://www.server-world.info/en/note?os=CentOS_7p=httpdf=13 I am having problems with recipients. No matter how I set up my recipeints (local or virtual) I get Recipient address rejected: User unknown in relay recipient table Can someone point me to a reliable instruction for me to unfuck my installation? Regards, Martin S -- Regards, Martin S -- Regards, Martin S -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Transparent Proxy?
Hello everyone, I need to setup an instance of postfix IN FRONT of an Exchange server, the purpose of which is to capture and archive all incoming email (and eventually outbound, but I'll tackle that later). Can someone point me to the section in the docs I should read for this? any example configs / howtos would be appreciated. One item of particular concern is header re-writes. I would prefer to have postfix retain the original sender IP address so that the spam filter on the Exchange box will continue to work as-is. (or I can write a script in Python... to do the re-writes if necessary) Any advice is welcomed. Thank you all in advance. -- Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com
Re: Transparent Proxy?
I was referring to TCP header re-writes. But, now that I think about it, the re-write would have to happen at the Exchange level after receipt. (Can't do it pre-send unless the entire message fit in a single packet. So... that's not going to work). Since this is the case, then I will probably have to migrate the spam filter to happen at the Postfix proxy, which requires we write a connector to their CRM system. Not a big deal, but not as easy as I was hoping. Now, back to the proxy setup? Where should I start reading? On 06/09/2015 12:11 PM, Viktor Dukhovni wrote: On Tue, Jun 09, 2015 at 12:04:56PM -0400, Michael Munger wrote: One item of particular concern is header re-writes. I would prefer to have postfix retain the original sender IP address ... What you call the sender IP address is not message (header) content. Rather, it is the IP address part of the TCP client's (address,port) pair associated with the TCP connection between the SMTP client and server. There is nothing that Postfix can do to preserve this when it connects to the Exchange server. The IP address Exchange will see will be that of the Postfix server. IP-based anti-spam processing will have to be performed by Postfix, not Exchange. -- Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com
Re: Transparent Proxy?
Re: Postfix must know users / use LDAP Excellent point, and I didn't think about that. Will do. Re: See google for details. +1 internets for awesome response. :-) Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 06/09/2015 12:55 PM, Noel Jones wrote: Also, postfix*must* know the valid recipients. If you simply accept all mail and let Exchange bounce the undeliverables, it won't take long for your server to be blacklisted as a backscatter source. Either periodically export a list of recipients from Exchange to postfix, or configure LDAP on the postfix server to query your AD. See google for details.
Re: Transparent Proxy?
Excellent advice, and I will read these throughly. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 06/09/2015 01:13 PM, Viktor Dukhovni wrote: On Tue, Jun 09, 2015 at 11:55:40AM -0500, Noel Jones wrote: Postfix isn't a proxy and can't be configured to behave as one. Maybe you're just using that term to refer to an email gateway/firewall, and not really expecting a proxy. Admittedly it is possible to configure a degenerate installation in which smtpd(8) proxies mail to another backend MTA without queueing. I've used this for TLS termination in front of an MTA with no TLS support. http://www.postfix.org/SMTPD_PROXY_README.html http://www.postfix.org/postconf.5.html#smtpd_proxy_filter With care, one can abuse proxy handoff to insert Postfix in front of another MTA. In such configurations one should generally leave the proxy options parameter: http://www.postfix.org/postconf.5.html#smtpd_proxy_options empty to let the backend MTA reject invalid recipients, unless Postfix has access to the recipient validation tables of the backend MTA (in which case it may be OK to use speed_adjust).
Re: Transparent Proxy?
You were clear. I understood that this was caveat emptor and that store - forward was the better way to go. Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com On 06/09/2015 05:26 PM, Viktor Dukhovni wrote: On Tue, Jun 09, 2015 at 04:49:50PM -0400, Michael Munger wrote: Admittedly it is possible to configure a degenerate installation in which smtpd(8) proxies mail to another backend MTA without queueing. I've used this for TLS termination in front of an MTA with no TLS support. http://www.postfix.org/SMTPD_PROXY_README.html http://www.postfix.org/postconf.5.html#smtpd_proxy_filter Excellent advice, and I will read these throughly. Note, I am not advocating this, just noting it is possible. Even with this, you still don't see the original client IP address as the origin of the SMTP connection at the backend MTA. Postfix sends XFORWARD metadata, but I don't think Microsoft Exchange supports this. You can use a proxy Postfix if you know what you're doing, but likely you should stick to store/forward. -- Viktor.
Re: Capture / Filter Remote Server Responses?
tldr: I want to be able to stick a filter / milter AFTER smtp(8) so I can save the remote server response code and associate it with an email that was sent so my PHP application can display the status of a notification sent to a user. Looking for docs on how to do this. Long story / background / additional info: I have an automated bug tracker that sends email notifications to users when something they are working on needs their attention. Sometimes, people complain they don't get the messages. So, I end up going through mail.log to see what the problem is, and invariably, I see the email was sent to their address with a 250 OK Queued as... response, which means it's not my problem. So, here's the question: I would like to add a python script that would be able to read / store the remote server responses for a given message. I will stuff an X-Header into the mail before I send it so that the Python script can associate the exact email with the exact notification that goes out. The point here, is that if a message with X-Header XYZ 1234 was sent, and we received a 250 OK back from the remote server, we know we've done everything we can to deliver the message. It's now outside our network. if, on the other hand, I get something like a 551 5.7.1, we know that something is wrong and we've been classified as spam. Or, as I recently saw with one of our users: 550 5.1.1 user unknown because the user in question is unable to type their own email address properly... .twice. (The form requires you confirm it, so they really had to try to fail). *If I can insert the python script as a filter (or milter?) AFTER smtp(8), that would be perfect. I could read the X-header injected by the PHP application to get the message ID as the PHP application knows it by, and then read the response code to find out the disposition of the message.* ** Where in the docs can I find information on doing this? Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com
RE: Capture / Filter Remote Server Responses?
This seems to get me about 80% of the way there. I can send from a specific mailbox, so the response will be delivered to that mailbox, and I can check it programmatically. So that's great. It still leaves me with the question of: how do I associate this particular reply with a notification that I sent? My initial idea was to use X-Header and stuff a unique ID in that header, so I could make the association, but this documentation appears to say: I'm not going to send you X-Headers back... or body content. So, what are your suggestions on associating the reports to the original notif? Where can I stuff an ID to do this? Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com -Original Message- From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of Wietse Venema Sent: Wednesday, May 20, 2015 11:03 AM To: Postfix users Subject: Re: Capture / Filter Remote Server Responses? Michael Munger: tldr: I want to be able to stick a filter / milter AFTER smtp(8) so I can save the remote server response code and associate it with an email that was sent so my PHP application can display the status of a notification sent to a user. Postfix has an option to record SMTP remote server responses, and to mail that back to the sender, formatted as a standard internet delivery status notification. You request this with /usr/sbin/sendmail -v recipient See http://www.postfix.org/ADDRESS_REWRITING_README.html#debugging for an example and narrative text. There currently is no option to request this in SMTP submissions. This should be an access-restricted feature for obvious reasons. Wietse
Re: Capture / Filter Remote Server Responses?
Thank you very much. I'll take it from here, and let you know how it goes. (I'll reply the list with a how-to for future searchers). On 05/20/2015 01:25 PM, Wietse Venema wrote: Michael Munger: This seems to get me about 80% of the way there. I can send from a specific mailbox, so the response will be delivered to that mailbox, and I can check it programmatically. So that's great. It still leaves me with the question of: how do I associate this particular reply with a notification that I sent? My initial idea was to use X-Header and stuff a unique ID in that header, so I could make the association, but this documentation appears to say: I'm not going to send you X-Headers back... or body content. The documentation shows that Postfix sends the message HEADERS back. You can also specify an Envelope-ID on submission, and then you get it back via the delivery status notification. SENDMAIL(1) General Commands ManualSENDMAIL(1) ... -f sender Set the envelope sender address. This is the address where delivery problems are sent to. With Postfix versions before 2.1, the Errors-To: message header overrides the error return address. ... -V envid Specify the envelope ID for notification by servers that support DSN. See RFC 3461 for more information about the envelope ID. Note that remote MTAs will report non-delivery or delays to the same sender address that you use for reporting remote SMTP server replies, though not all MTAs use the standardized format. Wietse -- Michael Munger, dCAP, MCPS, MCNPS, MBSS High Powered Help, Inc. Microsoft Certified Professional Microsoft Certified Small Business Specialist Digium Certified Asterisk Professional mich...@highpoweredhelp.com