[courier-users] Weird messages received
Sometimes I receive an email to my admin account stating that the email I sent to an [EMAIL PROTECTED] was not sent. The message has an attachment and when you open them down eventually you find a spam email that is FROM the [EMAIL PROTECTED] to a [EMAIL PROTECTED] How is this happening? What do I look at? From an external account I teleneted in and sent a message from the [EMAIL PROTECTED] to [EMAIL PROTECTED] And it worked... so I thought it is an open relay. But when I try any of the websites etc. to check for this none of them can find an open relay on my mail server. This is a delivery status notification from icarus, running the Courier mail server, version 0.47. The original message was received on Thu, 18 Oct 2007 23:46:17 -0600 from localhost (localhost [127.0.0.1]) --- UNDELIVERABLE MAIL Your message to the following recipients cannot be delivered: [EMAIL PROTECTED] _ Are you ready for Windows Live Messenger Beta 8.5 ? Get the latest for free today! http://entertainment.sympatico.msn.ca/WindowsLiveMessenger- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
Gordan Bobic wrote: But now that you mentioned it - is there a way to make Courier make an additional check? e.g. it receives a message: From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Normally, this is not too plausible to check if from is for a non-locally hosted domain, but if from is from a locally hosted domain, can we make Courier check if from is deliverable, and if not, reject with unknown sender or some such? That's not one of the checks that Courier performs, AFAIK. To carry out that check, you need to roll out your own filter. Actually, it is not a bad idea, since some spammers appear to be convinced that they get a better likelihood to deliver their thing if the sender's faked address features the same domain as the recipient(s). In case you really want to check From and To as sketched above, you may do that using a global filter. If you meant to check MAIL FROM, a.k.a. Return-Path instead, you might also do that using the maildropfilter. It is invoked for each RCPT TO, before receiving the body of the message. This filtering feature can be deployed running maildrop in embedded mode, which is done by writing maildrop's full path in the maildropfilter config file. However, one can point to whatever executable from that config file. The running program should read the SENDER environment variable and exit, say, 1 if it is a non-existing local domain. See http://www.courier-mta.org/localmailfilter.html On a separate note, is it possible to get Courier to do return path verification? i.e. for the from address, look up mx, connect, and do: HELO, MAIL FROM, RCPT TO, QUIT, just to see if the FROM address is deliverable? Ditto for how to implement that using a filter. However, I cannot imagine what would happen in case the remote domain filters according to the same principle, or uses greylisting, or is temporarily down. - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
Gordan Bobic wrote: On Fri, 19 Oct 2007, Alessandro Vesely wrote: Gordan Bobic wrote: But now that you mentioned it - is there a way to make Courier make an additional check? e.g. it receives a message: From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Normally, this is not too plausible to check if from is for a non-locally hosted domain, but if from is from a locally hosted domain, can we make Courier check if from is deliverable, and if not, reject with unknown sender or some such? That's not one of the checks that Courier performs, AFAIK. To carry out that check, you need to roll out your own filter. Are there any docs on how to write filters for courier? It is different writing global filters than local ones. The former can be written in Perl, Python, or C. Besides Courier examples, see http://search.cpan.org/dist/Courier-Filter/ http://phantom.dragonsdawn.net/~gordon/courier-pythonfilter/latest/README Local filters come in two flavors: delivery and recipient a.k.a. whitelisting API. Both are based on Maildrop, which is extensively documented. However, there are few examples about the recipient filters. They use the embedded mode of Maildrop, and can reject a message without taking the burden of sending the relevant DSN to the sender. In both flavors, local filters can read recipient-specific recipes. [...] On a separate note, is it possible to get Courier to do return path verification? i.e. for the from address, look up mx, connect, and do: HELO, MAIL FROM, RCPT TO, QUIT, just to see if the FROM address is deliverable? Ditto for how to implement that using a filter. However, I cannot imagine what would happen in case the remote domain filters according to the same principle, or uses greylisting, or is temporarily down. If the remote domain filters on the same principle, spam will get through if the sender address is valid. But that's to be expected. However, they may issue a connection to your server to verify the MAIL FROM that you are using for the purpose of verifying the MAIL FROM that... Hm... perhaps one should use an empty MAIL FROM for doing that check. Greylists are also OK, as it'll respond with a transient failure (which is probably as good as a yes). Permanent failures (e.g. no such user), OTOH, are a definite no. Fine. If the site is down, then that's questionable, especially if all their MX-es are out of action. Secondary MXes usually ignore if a mailbox exists on the primary server. It is probably better to skip this check when the primary is down. But it seems daft to be implementing all this in a filter. It would effectively mean implementing almost a full SMTP server. Granted, this could probably be done with a few hundred lines of Bash or Perl, but it still seems silly... Yes, it would work better integrated into the server. Consider that Courier, if configured to do so, has already looked up the MX record for the sender's domain. And how would one filter ALL incoming email using such a filter (e.g. big virtual domain mail server)? That can have a quite heavy impact on the throughput. Just consider how long a mail queue can grow if mail is being rejected in delivery mode: that means it is not very common to get a quick no such user. One should use rather severe timeouts and default to a yes when anything goes wrong. But then how effective would that be? Spammers are probably good at picking up problematic MAIL FROMs. - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
Johnny C. Lam wrote: Gordan Bobic wrote: It would also be worth checking if MAIL FROM: is the same as From: and RCPT TO: is the same as To:. Can anyone think of why these would ever be inconsistent in a valid email? The first argument to MAIL FROM: is the mailbox where errors and DSNs are sent. This is different from the From: header, which should be the mailbox of the author of the message and also the default reply-to address. These two can easily not match when address extensions and mailing lists are involved. To recap, mailing lists and robots. IME, messages from non-occasional mailing list and several robots sooner or later are stored in their own imap folder(s). In case doing so is common, there is an interesting point to consider: it is very easy to discriminate between vanilla human-to-human messages on the one hand and robots and occasional mailing lists on the other. Comparing that with the overwhelming intricacies involved in the spam vs. ham discrimination, I'd say the former is cleaner than the latter. It also clarifies the meaning of SPF. - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
M Core wrote: That's not one of the checks that Courier performs, AFAIK. To carry out that check, you need to roll out your own filter. Are there any docs on how to write filters for courier? I agree, I can't find any good examples on how to fix this problem or setup that filter. It mentions using the rcptfilter file, but courier doesn't have any docs on how to setup this file. http://www.courier-mta.org/courierperlfilter.html http://www.courier-mta.org/courierfilter.html The documentation is available either through the Courier website or you should be able to find these manpages on your system, i.e. man courierfilter man courierperlfilter Cheers, -- Johnny C. Lam - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
Gordan Bobic wrote: But now that you mentioned it - is there a way to make Courier make an additional check? ... Normally, this is not too plausible to check if from is for a non-locally hosted domain, but if from is from a locally hosted domain, can we make Courier check if from is deliverable, and if not, reject with unknown sender or some such? You can make courier do almost anything. :) http://phantom.dragonsdawn.net/~gordon/courier-pythonfilter/testfilters/localsenders.py I haven't tested that, but that's more or less how you could test local sender addresses. On a separate note, is it possible to get Courier to do return path verification? i.e. for the from address, look up mx, connect, and do: HELO, MAIL FROM, RCPT TO, QUIT, just to see if the FROM address is deliverable? Yes, the dialback pythonfilter does that. It wasn't usable for a long time, due to limitations in Python, but should be these days. - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
M Core wrote: I'm just not quite sure how to go about fixing it. Could someone give me a few more details. How do you use the script that they are talking about? The man page for 'courierperlfilter' should have all of the required information. Alternatively, you can look at pythonfilter: http://phantom.dragonsdawn.net/~gordon/courier-pythonfilter/ This filter does pretty much the same thing as the example you linked to: http://phantom.dragonsdawn.net/~gordon/courier-pythonfilter/testfilters/deliveredto.py - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
Gordan Bobic wrote: Are there any docs on how to write filters for courier? The courierfilter manpage spells out exactly what a filter must do. It would also be worth checking if MAIL FROM: is the same as From: and RCPT TO: is the same as To:. Can anyone think of why these would ever be inconsistent in a valid email? The first argument to MAIL FROM: is the mailbox where errors and DSNs are sent. This is different from the From: header, which should be the mailbox of the author of the message and also the default reply-to address. These two can easily not match when address extensions and mailing lists are involved. The first argument to RCPT TO: very often doesn't match the To: header if you receive the mail due to a Bcc: or have aliases or mail forwarding or mailing lists. If you want to not receive these spams, then reject these messages by better checking of the envelope (MAIL and RCPT) or do body inspection. Do not try to force message headers to match the envelope -- there is a reason why those are different concepts. Cheers, -- Johnny C. Lam - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
On Fri, 19 Oct 2007, Marcin 'Rambo' Roguski wrote: It would also be worth checking if MAIL FROM: is the same as From: and RCPT TO: is the same as To:. Can anyone think of why these would ever be inconsistent in a valid email? Well, that might kill of many of PHP generated mail: originally mine is sent as [EMAIL PROTECTED] while my robots (like newsletter, would use other address). Yes, but can anyone think of a valid (i.e. not caused by a poorly considered or implemented system) case where the envelope should be different to the from header? Gordan - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
On Fri, 19 Oct 2007, Alessandro Vesely wrote: Gordan Bobic wrote: But now that you mentioned it - is there a way to make Courier make an additional check? e.g. it receives a message: From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Normally, this is not too plausible to check if from is for a non-locally hosted domain, but if from is from a locally hosted domain, can we make Courier check if from is deliverable, and if not, reject with unknown sender or some such? That's not one of the checks that Courier performs, AFAIK. To carry out that check, you need to roll out your own filter. Are there any docs on how to write filters for courier? Actually, it is not a bad idea, since some spammers appear to be convinced that they get a better likelihood to deliver their thing if the sender's faked address features the same domain as the recipient(s). In case you really want to check From and To as sketched above, you may do that using a global filter. It would also be worth checking if MAIL FROM: is the same as From: and RCPT TO: is the same as To:. Can anyone think of why these would ever be inconsistent in a valid email? If you meant to check MAIL FROM, a.k.a. Return-Path instead, you might also do that using the maildropfilter. It is invoked for each RCPT TO, before receiving the body of the message. This filtering feature can be deployed running maildrop in embedded mode, which is done by writing maildrop's full path in the maildropfilter config file. However, one can point to whatever executable from that config file. The running program should read the SENDER environment variable and exit, say, 1 if it is a non-existing local domain. See http://www.courier-mta.org/localmailfilter.html Thanks. :-) On a separate note, is it possible to get Courier to do return path verification? i.e. for the from address, look up mx, connect, and do: HELO, MAIL FROM, RCPT TO, QUIT, just to see if the FROM address is deliverable? Ditto for how to implement that using a filter. However, I cannot imagine what would happen in case the remote domain filters according to the same principle, or uses greylisting, or is temporarily down. If the remote domain filters on the same principle, spam will get through if the sender address is valid. But that's to be expected. Greylists are also OK, as it'll respond with a transient failure (which is probably as good as a yes). Permanent failures (e.g. no such user), OTOH, are a definite no. If the site is down, then that's questionable, especially if all their MX-es are out of action. But it seems daft to be implementing all this in a filter. It would effectively mean implementing almost a full SMTP server. Granted, this could probably be done with a few hundred lines of Bash or Perl, but it still seems silly... And how would one filter ALL incoming email using such a filter (e.g. big virtual domain mail server)? Gordan - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
[courier-users] Weird messages received
To expand on this problem I think it is possibly this: http://www.nabble.com/Spam-with-Delivered-To-header-t3689885.html I'm just not quite sure how to go about fixing it. Could someone give me a few more details. How do you use the script that they are talking about? This seems like what I have: postmaster sends me a user unknown email. Inside that email is another email that is addressed to a known user, from an unknown user that was sent from the post master to the unknown user saying This is a delivery status notification from icarus, running the Courier mail server, version 0.47. The original message was received on Fri, 19 Oct 2007 00:16:55 -0600 from danielpwns ([:::98.18.15.171]) --- UNDELIVERABLE MAIL Your message to the following recipients cannot be delivered: [EMAIL PROTECTED]: Mail loop - already have my Delivered-To: header. AND attached to that email is a spam email... from [EMAIL PROTECTED] to a known user at my domain. _ R U Ready for Windows Live Messenger Beta 8.5? Try it today! http://entertainment.sympatico.msn.ca/WindowsLiveMessenger- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
That's not one of the checks that Courier performs, AFAIK. To carry out that check, you need to roll out your own filter. Are there any docs on how to write filters for courier? I agree, I can't find any good examples on how to fix this problem or setup that filter. It mentions using the rcptfilter file, but courier doesn't have any docs on how to setup this file. _ R U Ready for Windows Live Messenger Beta 8.5? Try it today! http://entertainment.sympatico.msn.ca/WindowsLiveMessenger- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
It would also be worth checking if MAIL FROM: is the same as From: and RCPT TO: is the same as To:. Can anyone think of why these would ever be inconsistent in a valid email? Well, that might kill of many of PHP generated mail: originally mine is sent as [EMAIL PROTECTED] while my robots (like newsletter, would use other address). -- Football is a game designed to keep coalminers off the streets. -- Jimmy Breslin - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
On Fri, 19 Oct 2007, M Core wrote: Sometimes I receive an email to my admin account stating that the email I sent to an [EMAIL PROTECTED] was not sent. The message has an attachment and when you open them down eventually you find a spam email that is FROM the [EMAIL PROTECTED] to a [EMAIL PROTECTED] How is this happening? What do I look at? From an external account I teleneted in and sent a message from the [EMAIL PROTECTED] to [EMAIL PROTECTED] And it worked... so I thought it is an open relay. But when I try any of the websites etc. to check for this none of them can find an open relay on my mail server. It's not an open relay. It accepts email for [EMAIL PROTECTED] That's where the bounce went to, because somebody forged the envelope from header to your valid account. But now that you mentioned it - is there a way to make Courier make an additional check? e.g. it receives a message: From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Normally, this is not too plausible to check if from is for a non-locally hosted domain, but if from is from a locally hosted domain, can we make Courier check if from is deliverable, and if not, reject with unknown sender or some such? On a separate note, is it possible to get Courier to do return path verification? i.e. for the from address, look up mx, connect, and do: HELO, MAIL FROM, RCPT TO, QUIT, just to see if the FROM address is deliverable? Gordan - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Weird messages received
I've been receiving these kinds of messages myself. And, in fairly large numbers. In my case, when I trace it back to the source, I discover that someone is using an invalid user name from my domain as the envelope sender address when sending spam. A non-delivery message then bounces back to the invalid user name from the server of the intended spam recipient. This non-delivery message is attached to the weird message, and clearly identifies the IP address of the bot net system that sent the original spam. There really is some mechanism here that I would like to have the power to change. These weird messages clearly occur as reports of non-delivery being sent to postmaster. For me, the postmaster, the quantity of non-delivery messages overwhelms me to the point that I ignore real non-delivery messages that I would like to be alert to. Just as clear is the fact that these messages are originating inside of Courier. My own Courier software is the current stable Debian binary version of Courier 0.53.3 retrieved by apt-get, and non-delivery messages to postmaster are enabled. Just why Courier believes that these particular kinds of non-delivery messages should be originated remains a mystery to me... - Original Message - From: Gordan Bobic [EMAIL PROTECTED] To: courier-users@lists.sourceforge.net Sent: Friday, October 19, 2007 12:57 AM Subject: Re: [courier-users] Weird messages received On Fri, 19 Oct 2007, M Core wrote: Sometimes I receive an email to my admin account stating that the email I sent to an [EMAIL PROTECTED] was not sent. The message has an attachment and when you open them down eventually you find a spam email that is FROM the [EMAIL PROTECTED] to a [EMAIL PROTECTED] How is this happening? What do I look at? From an external account I teleneted in and sent a message from the [EMAIL PROTECTED] to [EMAIL PROTECTED] And it worked... so I thought it is an open relay. But when I try any of the websites etc. to check for this none of them can find an open relay on my mail server. It's not an open relay. It accepts email for [EMAIL PROTECTED] That's where the bounce went to, because somebody forged the envelope from header to your valid account. But now that you mentioned it - is there a way to make Courier make an additional check? e.g. it receives a message: From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Normally, this is not too plausible to check if from is for a non-locally hosted domain, but if from is from a locally hosted domain, can we make Courier check if from is deliverable, and if not, reject with unknown sender or some such? On a separate note, is it possible to get Courier to do return path verification? i.e. for the from address, look up mx, connect, and do: HELO, MAIL FROM, RCPT TO, QUIT, just to see if the FROM address is deliverable? Gordan - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now http://get.splunk.com/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
