Re: [courier-users] identlookup vs noidentlookup
Kristian Duus Østergaard writes: When sending mail, Courier has no control over whether the remote mail server issues its own ident query, Courier will wait 5 minutes for the remote server's initial response, and that is configurable wth the esmtptimeouthelo config setting. No but in my case the problem comes from the combination that the sending server has a timeout of less than 30 seconds AND drops the ident packages making courier use the full timeout of 30 seconds. Most receiving servers perform forward and reverse DNS lookup on the peer's IP address. This is even more popular than ident lookups. It's not uncommon for DNS lookups to stall for various reasons. When testing connections to remote servers, I find quite often that remote servers take a fairly long time to issue their greeting, even if the connection goes thru promptly, while they struggle with their own DNS servers. Senders which are so impatient are going to be fairly broken in many other ways, too. pgpcaA1PS5d2M.pgp Description: PGP signature -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] identlookup vs noidentlookup
On 2013-05-24 12:56, Sam Varshavchik wrote: Kristian Duus Østergaard writes: When sending mail, Courier has no control over whether the remote mail server issues its own ident query, Courier will wait 5 minutes for the remote server's initial response, and that is configurable wth the esmtptimeouthelo config setting. No but in my case the problem comes from the combination that the sending server has a timeout of less than 30 seconds AND drops the ident packages making courier use the full timeout of 30 seconds. Most receiving servers perform forward and reverse DNS lookup on the peer's IP address. This is even more popular than ident lookups. It's not uncommon for DNS lookups to stall for various reasons. When testing connections to remote servers, I find quite often that remote servers take a fairly long time to issue their greeting, even if the connection goes thru promptly, while they struggle with their own DNS servers. Senders which are so impatient are going to be fairly broken in many other ways, too. I probably didn't write it - but I have a wireshark dump that proves that it is the ident that times out. You are off course correct that it might very well be broken in other ways. Regards Kristian -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] identlookup vs noidentlookup
On 24/05/2013 02:24, Kristian Duus Østergaard wrote:
Hi,
My smtp server is currently using identlookup and I think it is one reason
that
I don't receive a ton of Spam.
Unfortunately some of my users receive mails from a domain that has a very
short
timeout and drops identlookups at the firewall, instead of rejecting them.
This
results in no mails coming through to my users from the domain in question and
me getting asked how many other domains does this happen from. My own
approximate count indicates that only 1.6% of the failing connections are from
legit servers.
So my questions are really :
What is your experience with identlookups ?
Should I stop using it on my server and risk more Spam ?
When you discover a problem with a server what do you do ?
Do any of you have automated scripts to inform the postmaster in the other
end that you do have a server and it actually can respond ?
Does courier have any filtering function for this very special scenario ?
Sorry for the long rant..
Regards
Kristian Duus Østergaard
Hi all,
I know that Courier lets you set server-specific options, as well as
domain specific.
Is it possible currently, or would it be possible to implement, this for
the {no,}identlookup option?
This would allow you to more easily work-around this broken server by
telling Courier to use identlookups by default, but not for connections
from this host.
I have turned identlookup off on my server, but I use the greylisting
module from PythonFilter.
As the server is only used by myself, the initial delay of 5+ minutes is
worth virtually no spam.
Email was never designed as a time-critical service, and should NEVER be
treated as such. But that's another discussion for another day.
The better solution to your problem, Kristian, is definitely to (attempt
to) contact the broken server's admin to advise them of this issue. But
postmaster@ or admin@ might not be set as valid destinations, or not
delivered to a mailbox which is read on a regular/routine basis.
You could also put a firewall rule in which REJECTs the identlookup
packets sent from your email server toward theirs. This would
work-around the impatience as well.
Good luck.
Cheers,
Tim Lyth
--
Try New Relic Now We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
___
courier-users mailing list
courier-users@lists.sourceforge.net
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
[courier-users] identlookup vs noidentlookup
Hi, My smtp server is currently using identlookup and I think it is one reason that I don't receive a ton of Spam. Unfortunately some of my users receive mails from a domain that has a very short timeout and drops identlookups at the firewall, instead of rejecting them. This results in no mails coming through to my users from the domain in question and me getting asked how many other domains does this happen from. My own approximate count indicates that only 1.6% of the failing connections are from legit servers. So my questions are really : What is your experience with identlookups ? Should I stop using it on my server and risk more Spam ? When you discover a problem with a server what do you do ? Do any of you have automated scripts to inform the postmaster in the other end that you do have a server and it actually can respond ? Does courier have any filtering function for this very special scenario ? Sorry for the long rant.. Regards Kristian Duus Østergaard -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] identlookup vs noidentlookup
On 23.05.13 18:24, Kristian Duus Østergaard wrote: My smtp server is currently using identlookup and I think it is one reason that I don't receive a ton of Spam. I have configured the same on my former employer's SMTP servers because of the same reason. I found it useful - if the machine provides ident, SMTP transaction starts imediately and if client is firewalled, it has to wait, which helps against spambots. Clients have to use different ports (and different SMTP server) even... Unfortunately some of my users receive mails from a domain that has a very short timeout and drops identlookups at the firewall, instead of rejecting them. This results in no mails coming through to my users from the domain Such SMTP client violates the SMTP protocol, which requires waiting at least 300 seconds for SMTP welcome greeting. Inform the clients (and remote postmaster) that remote SMTP server is misconfigured and violates the standard which results in problems. in question and me getting asked how many other domains does this happen from. My own approximate count indicates that only 1.6% of the failing connections are from legit servers. So my questions are really : What is your experience with identlookups ? Should I stop using it on my server and risk more Spam ? When you discover a problem with a server what do you do ? Commented above. I recommend using ident on SMTP server to avoid much of spam. Do any of you have automated scripts to inform the postmaster in the other end that you do have a server and it actually can respond ? no automation Does courier have any filtering function for this very special scenario ? no filtering, but for special cases you can drop SMTP connections when outgoing, which will result in immediate dropped ident connection, instead of waiting to time out. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. REALITY.SYS corrupted. Press any key to reboot Universe. -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] identlookup vs noidentlookup
On Thu, May 23, 2013 at 6:24 PM, Kristian Duus Østergaard krist...@duus.com wrote: Hi, Hi there! I'll just start with noting that I come from a different school of MTA administration than Matus Uhlar, so I've learned things differently. My smtp server is currently using identlookup and I think it is one reason that I don't receive a ton of Spam. Unfortunately some of my users receive mails from a domain that has a very short timeout and drops identlookups at the firewall, instead of rejecting them. This results in no mails coming through to my users from the domain in question and me getting asked how many other domains does this happen from. My own approximate count indicates that only 1.6% of the failing connections are from legit servers. So my questions are really : What is your experience with identlookups ? Ident lookups have historically been associated with spamming, as a more or less efficient way of identifying which addresses are valid and therefore okay to send spam to. It has also been associated with targeted attacks against specific accounts across protocols, e.g. for FTP, SSH etc. Relying on ident lookups therefore is to rely on that most MTA admins open up for ident lookups, such as Matus Uhlar obviously is doing. In my experience, this is futile. Many legitimate email providers block ident lookups. Should I stop using it on my server and risk more Spam ? Those are two questions. In my opinion, you should stop using it. This is not likely to be the last time you experience problems with legitimate email related to blocked/blackholed ident lookups. But only you can really answer which balance of spam blocked vs. legitimate email received is best for your service. It would also depend a bit on volume, a high volume may result in a high rate of ident lookups, which may be considered bad. Whether you actually risk more spam by disabling this feature is an unresolved question, as far as I can tell there are mostly opinions around; I'm not aware of any thorough research on this in particular, but I could of course be wrong. I often am. When you discover a problem with a server what do you do ? That depends on whose server it is. If it is one of the big freemail providers with no functional technical contact points (certain companies whose names begin with G, M, and Y come to mind), then I typically will inform the user that they're using such a dysfunctional service, and that they must expect a certain degree of problems. If it is someone using a certain firewall product with the initial 'B', likewise. Otherwise, I may provide a slightly more technical explanation that can be forwarded to the sending party's email administrator. This works in an astonishing amount of cases, but still too few to make a real difference. Do any of you have automated scripts to inform the postmaster in the other end that you do have a server and it actually can respond ? Nopes, automation for information to third parties in cases like these may stand at risk for unnecessary backscattering, all depending on how well the script catches the conceivable use cases. Does courier have any filtering function for this very special scenario ? I don't. Sorry for the long rant.. That was not a rant. :) -- Jan -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] identlookup vs noidentlookup
On 23.05.13 20:07, Jan Ingvoldstad wrote: Ident lookups have historically been associated with spamming, as a more or less efficient way of identifying which addresses are valid and therefore okay to send spam to. It has also been associated with targeted attacks against specific accounts across protocols, e.g. for FTP, SSH etc. Relying on ident lookups therefore is to rely on that most MTA admins open up for ident lookups, such as Matus Uhlar obviously is doing. In my experience, this is futile. Many legitimate email providers block ident lookups. This is not what I was saying. I have said that if someone provides IDENT lookups, the response will be used and the client is rewarded with avoiding the timeout when waiting to timeout. Of course, using SMTP delay of e.g. 30 seconds may help you even if ident is working, so I encourage you to implement this anti-spam measure (and drop all clients who send any content before your server displays the greeting). I think courier has no such measure currently. I don't think anyone sane would block IDENT lookups. You may simply not provide it. Note that IDENT response is for your (sender's, smtp client's) use, not for the recipient (SMTP server) admin's - the server will just log it and in case of spamming the header will be passed to you, who can in addition see the IDENT response, if you have decided to provide it. Should I stop using it on my server and risk more Spam ? Those are two questions. In my opinion, you should stop using it. This is not likely to be the last time you experience problems with legitimate email related to blocked/blackholed ident lookups. You in fact say there's no need to turn ident lookups off. The SMTP client MUST wait at least 300 seconds, because delays may happen because of other reasons, while timeout for TCP connect is uaually 30 to 60 seconds. If client drops the connection sooner, it's clearly problem of the client. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. The box said 'Requires Windows 95 or better', so I bought a Macintosh. -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] identlookup vs noidentlookup
On Thu, May 23, 2013 at 9:47 PM, Matus UHLAR - fantomas uh...@fantomas.skwrote: This is not what I was saying. I have said that if someone provides IDENT lookups, the response will be used and the client is rewarded with avoiding the timeout when waiting to timeout. I'm sorry for interpreting you in the way that you yourself let your own servers behave in the way that you expect from others. Of course, using SMTP delay of e.g. 30 seconds may help you even if ident is working, so I encourage you to implement this anti-spam measure (and drop all clients who send any content before your server displays the greeting). I think courier has no such measure currently. I don't think anyone sane would block IDENT lookups. Then we heartily disagree. You may simply not provide it. It is common for non-provided services to be firewalled off with DROP rules rather than DENY or even ACCEPT. This comes from hard-learned lessons about permitting services on a whitelisting basis rather than blacklisting basis; this reduces risk of damage or loss of service from several types of attacks. It is good network maintenance practice. You in fact say there's no need to turn ident lookups off. No, I don't. The SMTP client MUST wait at least 300 seconds, I'm afraid you're misremembering the standard. RFC 2821 and 5321 specify that timeouts MUST be supported, but only SHOULD be at least 5 minutes for the initial 220 message. https://tools.ietf.org/html/rfc2821#page-56 https://tools.ietf.org/html/rfc5321#page-65 A SHOULD requirement weighs very heavily, of course, but it means that the RFC authors and the community have accepted the bitter fact that RFC 821 was very vague about several technological aspects that have later proven to be of importance, and taken that into account when specifying the SHOULD requirement rather than a MUST as you believe. -- Jan -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] identlookup vs noidentlookup
Sam Varshavchik writes: And, with each passing here, the value of ident lookups gets smaller, and Make that …passing year…. Serves me right for trying to reply to my email, while watching Youtube at the same time… pgpA6JdRAf64w.pgp Description: PGP signature -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] identlookup vs noidentlookup
On 2013-05-24 02:07, Sam Varshavchik wrote: Kristian Duus stergaard writes: Hi, My smtp server is currently using identlookup and I think it is one reason that I don't receive a ton of Spam. Unfortunately some of my users receive mails from a domain that has a very short timeout and drops identlookups at the firewall, instead of rejecting them. This results in no mails coming through to my users from the domain in question and me getting asked how many other domains does this happen from. My own The success or failure of ident lookup has no effect on whether the mail from the sender is accepted, or not. Has no impact, whatsoever. The only thing that ident lookup does is record some additional data in mail headers. And, with each passing here, the value of ident lookups gets smaller, and smaller. identd is only to be found on multiuser servers, which haven't been exactly popular, in a long, long time. As others have stated, Courier will timeout on its ident query after 30 seconds. Legitimate mail servers will wait several minutes before timing out. There's also some confusion here regarding when ident lookup matters. Courier makes an ident query when receiving mail. And Courier will timeout after 30 seconds. When sending mail, Courier has no control over whether the remote mail server issues its own ident query, Courier will wait 5 minutes for the remote server's initial response, and that is configurable wth the esmtptimeouthelo config setting. No but in my case the problem comes from the combination that the sending server has a timeout of less than 30 seconds AND drops the ident packages making courier use the full timeout of 30 seconds. -- Mvh. Kristian Duus stergaard Kristian Duus Consult email: d...@kristian-duus.dk mobil: +45 22114772 -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] identlookup vs noidentlookup
On 2013-05-24 05:57, Andrew Burnette wrote: On 05/23/2013 12:24 PM, Kristian Duus Østergaard wrote: Hi, My smtp server is currently using identlookup and I think it is one reason that I don't receive a ton of Spam. Unfortunately some of my users receive mails from a domain that has a very short timeout and drops identlookups at the firewall, instead of rejecting them. This results in no mails coming through to my users from the domain in question and me getting asked how many other domains does this happen from. My own approximate count indicates that only 1.6% of the failing connections are from legit servers. So my questions are really : What is your experience with identlookups ? Should I stop using it on my server and risk more Spam ? When you discover a problem with a server what do you do ? Do any of you have automated scripts to inform the postmaster in the other end that you do have a server and it actually can respond ? Does courier have any filtering function for this very special scenario ? Sorry for the long rant.. Regards Kristian Duus Østergaard Consider it an effective orthogonal version of greylisting (which often causes other greater problems unfortunately)? Enabled, it typically tends to hold the inbound smtp connection open ~30 seconds before a HELO smtp conversation is allowed to begin. Turns out numerous virus/malware bots drop their tcp connection right at the 30 second mark. (yet another reason also why port 587 is a better choice for local relaying end user clients/senders such they do not experience the hold time, entirely a different issue though and not helpful to your experience) It's very effective in reducing spam in my system, testing with either setting a couple years ago the effective cut rate of 2/3 or more. In combo with good RBL selection, I see 90% of connection attempts dropped/rejected/etc, and still my users receive their good mail. My users complained more when it was disabled, and many have been shielded for so long against spam they don't understand why users of other systems complain about spam:-) It may be possible to manipulate your firewall to respond with a bogus lovel affirmative indent for just the domain name of the impatient MTA. Just a thought, but not terribly complicated depending upon what you front your servers with (even a simple ufw rule might do the trick?) Many variables in that idea only you might be able to decide if it's feasible or not. In the same respect, it would be nice to have a simple BGP feed to block various known bad neighborhoods out there on the Internet Good luck, andy I think your take on this is closer to my own sentiments - using ident lookups as an accepted greylisting technique and I think this is what Matus is also doing. As for whether or not to provide valid ident lookups my take is that all you really need is to reject the connection. Courier will then accept the reject and continue immediately as far as I understand/have tested. I totally agree with Sam's and Jan's comment that ident lookups as a verifying technique probably is a thing of the past. As for Jan's comment on dropping instead of rejecting - I think the idea behind dropping used to work. But for servers connected directly to the internet I don't think it makes much sense any more as it's relatively easy to identify all the open ports anyway. So to sum it up - I think I'll leave my server the way it is and try to politely tell the admin of the server that even though the specification says SHOULD be 300 seconds it's probably a good idea not to lower it, even if it makes your queues empty faster. Regards Kristian -- Try New Relic Now We'll Send You this Cool Shirt New Relic is the only SaaS-based application performance monitoring service that delivers powerful full stack analytics. Optimize and monitor your browser, app, servers with just a few lines of code. Try New Relic and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
