Re: [Declude.JunkMail] Gateways and CMDSPACE conundrum
Sandy et al., Regarding how peering is handled, that sucks! It was a bit of a kludge anyway, more than most at least. I just got mail bombed on both servers by three different ISP relays. The recipient address was invalid (sent to and from itself), and if I had MS SMTP/ORF configured on both machines to blacklist invalid addresses (instead of just the domain on the backup as is done currently), this would have stopped that attack cold without me lifting a finger. Instead I was stuck scanning as many as 15 incoming messages per second, or at least trying to do so, but not succeeding. Worse yet, the destination server was bouncing NDR's back through our server and each of those were being virus scanned despite the original being in plain text. I've also noticed that there are a couple hundred E-mails a day in the backup's BadMail directory for locally hosted domains with non-existant accounts. I'm only hosting about 300 accounts in total, and this is all to just those addresses and not the gatewayed domains. Address validation would stop these needless bounces to forged addresses from my servers and help to clean up the Internet. I have a feeling that the need for CMDSPACE detection falls far short of the need for address validation for gatewayed domains. ORF seems to be a great tool for this because I can do things like configure a local RBL for instance to block virus sending machines on the gateway by maintaining a single zone, along with sender and recipient blacklisting. ORF of course is a very limited spam blocking tool at the moment and not appropriate for such needs. I'm still thinking about approaching IMail for a solution to recipient blacklisting on gatewayed domains using an 'everything but' methodology. How unrealistic do you think that would be??? It might just be easier to ask VAMSoft for CMDSPACE header logging and dealing with the two separate pieces of technology. Matt Sanford Whiteman wrote: With a recent IMail release, you can now set up peering to use RCPT TO to test incoming messages for valid senders. Right, but the resulting envelope behavior is not different from the old VRFY scenario, AFAIK. As long as IMail does envelope rejection for peered domains that fail validation, this setup should work. There's never been real-time validation and rejection with peering. With just IMail servers, the idea is that a message can enter a "farm" of peers and will only be bounced (not rejected) after the message has been spooled and delivery attempted at every peer. Once you add an MS SMTP server into the mix, you only have one-way peering. Maybe I'm not clear on what you're suggesting, but I don't see it shedding any light on your issue. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re[2]: [Declude.JunkMail] Gateways and CMDSPACE conundrum
I'm eagerly awaiting the results of your research :) Well, I wish I had better news, but the problem (they'd call it a feature, and for once I think I'd agree) is that MS SMTP normalizes the envelope fields as part of of normal message flow. Thus, even though the IIS and ORF logs will show extra spaces, they're the only place where the actual protocol commands are preserved. ORF doesn't do anything special to change this data; it just automatically changes after the message is submitted, and subsequent message inspection during MS SMTP transport will not show any of the original formatting errors. The only way you could retain the protocol-level data is by plugging in another protocol event sink alongside ORF, but running more than one custom PES on the same system is pretty un-heard-of (I believe it's possible using different priorities, but I've never seen it in practice). So you are pretty much stuck with (a) giving up, or (b) trying the log inspection option. I really wouldn't trust the latter, since you might actually have to throttle _down_ processing to make sure that the logs can be parsed in time to intercept the message...not good. Sorry I don't have a fix. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Gateways and CMDSPACE conundrum
Sandy, Am I correct in assuming that you attempted something similar to the following script on the VAMSoft site? Envelope header information http://www.vamsoft.com/orf/tools.asp#smtpenvl This is how they add headers to the messages containing the MAIL FROM and RCPT To data. I get the feeling from reading the code that they are using some sort of pre-normalized data that MS SMTP spits out, though I'm not at all sure about that. If this data isn't normalized, then this would seem to do the trick with a little bit of modification. Thanks, Matt Sanford Whiteman wrote: I'm eagerly awaiting the results of your research :) Well, I wish I had better news, but the problem (they'd call it a feature, and for once I think I'd agree) is that MS SMTP "normalizes" the envelope fields as part of of normal message flow. Thus, even though the IIS and ORF logs will show extra spaces, they're the only place where the actual protocol commands are preserved. ORF doesn't do anything special to change this data; it just automatically changes after the message is submitted, and subsequent message inspection during MS SMTP transport will not show any of the original formatting errors. The only way you could retain the protocol-level data is by plugging in another protocol event sink alongside ORF, but running more than one custom PES on the same system is pretty un-heard-of (I believe it's possible using different priorities, but I've never seen it in practice). So you are pretty much stuck with (a) giving up, or (b) trying the log inspection option. I really wouldn't trust the latter, since you might actually have to throttle _down_ processing to make sure that the logs can be parsed in time to intercept the message...not good. Sorry I don't have a fix. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re[2]: [Declude.JunkMail] Gateways and CMDSPACE conundrum
Am I correct in assuming that you attempted something similar to the following script on the VAMSoft site? In the same vein, yes (though actually part of an existing compiled sink that we wrote). This is how they add headers to the messages containing the MAIL FROM and RCPT To data. I get the feeling from reading the code that they are using some sort of pre-normalized data that MS SMTP spits out, though I'm not at all sure about that. Yes, MS SMTP's envelopefields collection is all normalized, and that's all that's available after submission. That's what I was double-checking, --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Gateways and CMDSPACE conundrum
Sandy, Well, I haven't yet given up. For one, I could ask that VAMSoft if they could allow for header tagging of this type. There is another kludge though that I am thinking might be of use here... With a recent IMail release, you can now set up peering to use RCPT TO to test incoming messages for valid senders. You could set up each gateway domain to peer off of the ORF enabled instance of MS SMTP for validation with RCPT TO, and then it would handle delivery according to your host file settings (MS SMTP has better queue settings anyway). As long as IMail does envelope rejection for peered domains that fail validation, this setup should work. I do understand of course that this would be somewhat limiting in terms of capacity due to the delay while checking the RCPT TO, however if installed on the same box or network, this should be very quick. I'm hoping that down the line IMail will have the ability to do envelope rejection for gatewayed domains, and if that comes, you could just strip out the MS SMTP/ORF part without needing to totally retool your environment. What do you think? Matt Sanford Whiteman wrote: Am I correct in assuming that you attempted something similar to the following script on the VAMSoft site? In the same vein, yes (though actually part of an existing compiled sink that we wrote). This is how they add headers to the messages containing the MAIL FROM and RCPT To data. I get the feeling from reading the code that they are using some sort of pre-normalized data that MS SMTP spits out, though I'm not at all sure about that. Yes, MS SMTP's envelopefields collection is all normalized, and that's all that's available after submission. That's what I was double-checking, --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re[2]: [Declude.JunkMail] Gateways and CMDSPACE conundrum
With a recent IMail release, you can now set up peering to use RCPT TO to test incoming messages for valid senders. Right, but the resulting envelope behavior is not different from the old VRFY scenario, AFAIK. As long as IMail does envelope rejection for peered domains that fail validation, this setup should work. There's never been real-time validation and rejection with peering. With just IMail servers, the idea is that a message can enter a farm of peers and will only be bounced (not rejected) after the message has been spooled and delivery attempted at every peer. Once you add an MS SMTP server into the mix, you only have one-way peering. Maybe I'm not clear on what you're suggesting, but I don't see it shedding any light on your issue. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
[Declude.JunkMail] Gateways and CMDSPACE conundrum
Scott, Sandy and others that might have an idea about what do to, I've been moving towards a setup that allows me to do envelope rejection of invalid addresses for gatewayed domains due to repeated instances where spammers have dictionary attacked servers that accept all valid domains without an attempt to validate the user. I have setup a gateway to use MS SMTP with VAMSoft's ORF which allows me to blacklist all addresses but those that match the list I provide it. I have unfortunately noticed though that the CMDSPACE dependencies (a space in the RCPT TO command) are not preserved when passing through the gateway. The conundrum that I face is the conflict between the following two requirements/wishes: 1) SMTP Envelope Rejection of Invalid Addresses - This is a requirement from what I can determine on any gateway spam filtering solution. This protects from haphazard dictionary attack harvesting of invalid addresses, blocks problematic Joe-Jobs, avoids scanning of dead and undeliverable gatewayed accounts which can result in bounce messages to forged addresses when such messages pass the spam filtering mechanism and are sent to the destination server. 2) CMDSPACE Detection - Given that this test will hit on about 50% of the spam that our system sees, and is about 99.9% accurate on our system, it would be very unfortunate to loose the ability to detect this condition when handled by a gateway designed to provide the ability to do SMTP Envelope Rejection. In the very least, it appears that MS SMTP reformats the RCPT TO command and eliminates the space. I imagine that it is also possible that the gateway is also capable of modifying broken headers that may trip BADHEADERS or SPAMHEADERS, though I haven't verified this. I'm not yet married to one particular solution for either need. I'm sure that there are other solutions out there besides ORF that can be used to do envelope rejection of invalid senders on gatewayed domains, and if it was possible to modify the headers at the gateway to indicate the improper RCPT TO space, this could then be detected through a custom filter in Declude. I would like to avoid if possible a setup that would require a different server to handle the envelope rejection, though the cost of a box running Linux plus hosting may be worth it if I can preserve the RCPT TO and other header formatting. I've noted in the ORF logs that the space is shown when present, just not replicated. My plan was to place both MS SMTP with ORF on the same server as IMail with Declude, and I'm thinking that it may also be possible to parse the logs in real-time in order to retrieve this information through an external test in Declude. ORF supports syslogging or logging to a text file which isn't locked. If this is reasonable to do, i.e. not processor intensive or overly convoluted , I could get someone to program it for me. While this might solve the problem in the setup that I've been considering, it doesn't resolve the issue for others using gateways that strip this information. I'm hoping that maybe someone here has some ideas about which technologies could be used to do both envelope rejection of invalid gatewayed addresses, and either maintain the RCPT TO command formatting, modify the headers to indicate the condition, or possibly a way to approach this through a separate app that is exposed to the ORF logs. BTW Scott, I'd pay extra for an implementation of Declude not dependent upon IMail for functionality. Although there are many other spam blocking products out there that will do this, none of them are created and supported by someone with the degree of experience that you have nor the level of understanding. To people like myself this would represent a very nice savings on hardware, software and hosting costs. Thanks, Matt -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re: [Declude.JunkMail] Gateways and CMDSPACE conundrum
Matt, Let me do some research tonight; it's possible that I will have something tomorrow that will let you forward through the incoming envelope stuff in an x- header. (I shudder to think about the log parsing alternative.) --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Gateways and CMDSPACE conundrum
Sandy, I'm eagerly awaiting the results of your research :) Thanks, Matt Sanford Whiteman wrote: Matt, Let me do some research tonight; it's possible that I will have something tomorrow that will let you forward through the incoming envelope stuff in an x- header. (I shudder to think about the log parsing alternative.) --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
