Re: can I roll back to an earlier version of updates
Karsten Bräckelmann wrote: On Sun, 2010-02-28 at 18:44 -0500, Lee Dilkie wrote: For what ever reason, my sa-update to 3.30 has buggered itself. In my efforts to debug it's now at the situation that SA has no rules to run and I'm getting swamped. The first sentence is seriously confusing. You can not sa-update to 3.3.0. sa-update only updates the rules, for the already installed version. Yeah, sorry about that... As I've discovered, it's all tied to the version of SA and 3.2 rules won't run with 3.3 SA. How, if it's possible, can I tell SA and sa-update to use the 3.2 version of the ruleset? Simply deleting the tree and sa-compiling did not work. SA is still looking for 3.3 rules and as it finds none, is letting everything through. You cannot really and reliably make SA 3.3 use 3.2 rules. agreed ;) Anyway, what comes to mind: Did you run sa-update after the upgrade to 3.3.0 at all? If not, did you install the rules tarball alongside SA? I was originally running the 3.3 rules and that was fine, and as far as I know, I did even run sa-upgrade (can't tell you if it upgraded the rules over the base ones) but it's the latest sa-update that pulled in newer rules that didn't link. And it's my monkeying around, deleting rules directories, that has left me without rules from updates spamassassin_org. And boy! do they block a lot of spam or what! ;) How did you upgrade? Any chance both versions ended up living on your system? Running 3.3.0 with a broken sa-update for whatever reason, can be cured by removing the entire update dir, and installing the plain, stock 3.3.0 rules tarball, if not already done. I'm on freebsd, I'm going to try and find out where that's stored, it's likely in the ports tree somewhere. thanks for the help.. Unfortunately, my update to the latest ruleset fails lint... as I said in an earlier email... config: failed to parse line, skipping, in /tmp/.spamassassin545130JflrRtmp/72_active.cf: mimeheader __TVD_MIME_ATT_AOPDF Content-Type =~ /^application\/octet-stream.*\.pdf/i config: failed to parse line, skipping, in /tmp/.spamassassin545130JflrRtmp/72_active.cf: mimeheader __TVD_MIME_ATT_AP Content-Type =~ /^application\/pdf/i config: failed to parse line, skipping, in /tmp/.spamassassin545130JflrRtmp/72_active.cf: mimeheader __TVD_MIME_ATT_TP Content-Type =~ /^text\/plain/i Is there any way that I can force the system to download the ruleset so I can comment out the offending lines and carry on? (I'd at least like to see what they are, and why it doesn't parse, maybe it's something in my config). -lee -- Fuelly http://www.fuelly.com/driver/dilkie/golf
Re: can I roll back to an earlier version of updates
On Mon, 2010-03-01 at 06:45 -0500, Lee Dilkie wrote:
Karsten Bräckelmann wrote:
Anyway, what comes to mind: Did you run sa-update after the upgrade to
3.3.0 at all? If not, did you install the rules tarball alongside SA?
I was originally running the 3.3 rules and that was fine, and as far as
I know, I did even run sa-upgrade (can't tell you if it upgraded the
rules over the base ones) but it's the latest sa-update that pulled in
newer rules that didn't link. And it's my monkeying around, deleting
rules directories, that has left me without rules from updates
spamassassin_org. And boy! do they block a lot of spam or what! ;)
How did you upgrade? Any chance both versions ended up living on your
system?
Running 3.3.0 with a broken sa-update for whatever reason, can be cured
by removing the entire update dir, and installing the plain, stock 3.3.0
rules tarball, if not already done.
I'm on freebsd, I'm going to try and find out where that's stored, it's
likely in the ports tree somewhere.
man spamassassin
See the section Configuration Files. The first path mentioned for
Default Configuration Data should be the sa-update one. SA version is
embedded in that path, inside /var/lib here, IIRC /var/db or something
on FreeBSD.
The last one in that block of paths should be where SA expects the stock
rules. The first existing one from that list wins, anything else will be
ignored.
spamassassin -D can help in identifying bad rule sets being picked up,
and where SA ultimately looks for the cf files.
Is there any way that I can force the system to download the ruleset so
I can comment out the offending lines and carry on? (I'd at least like
to see what they are, and why it doesn't parse, maybe it's something in
my config).
Drop the bad update first, and revert to stock. Re-install it from
ports, if need be.
--
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Setting Blacklist_from and whitelist_to
On Sun, 2010-02-28 at 12:13 -0800, damuz wrote: Martin Gregorie-2 wrote: How is SA used by your hosted email MTA, IOW is Spamassasin called in pre-queue before the mail has been accepted or is it called later? How much control do you have over that server? Can you set up grey-listing for your domain on it? If you're getting much backscatter (remote MTAs sending 'unknown user' rejections for mail with a forged sender), consider setting up an SPF record for your domain. Well-behaved remote MTAs will use it to check for forged senders and not send the rejections if the sender was forged. It would be better to let the MTA reject unknown users as part of pre-queue processing because that puts less processing load on the main chain. Do you have enough access to do this? Martin I'll answer as best as I know. Our network runs SBS2003 and Exchange deals with the outgoing mail and collects it from the hosting company which also hold the website. SA is configurable from the hosting sites Cpanel (web login) and I'm assuming that as it 'tags' and 'scores' the mail that Exchange then downloads, that it is called in after the mail has already been accepted. I don't really know. My point was that rejecting mail for unknown users on your site is best done by the hosting MTA, not SA. This means that you need access to the MTA configuration, rather than the SA configuration. Once mail has been accepted by the hosting MTA you can only deliver it or throw it in a bit bucket. You should *not* reject it at this stage or you become part of the junk mail problem. Grey-listing: I'm using a similar set-up to you. My mail is accepted by my ISP's smarthost and passed to my private MTA for delivery. The only difference is that I run SA rather than my ISP. About a year ago my ISP turned on grey-listing and overnight spam dropped from 70% of incoming mail to less than 10%. IME its A Good Thing, but you need to persuade your email hosting organisation to implement it because its a front end for the hosting MTA you're using. HTH Martin
spammers targeting ironport quarantine now: phish
Imagine my surprise this am when I got a quarantine report from our ironport email server (when I don't have one!) Phishers targeting ironport users now. if anyone has ironport, can you look at this email to see if it looks like an ironport quarantine report? I do notice the lack of ironport headers in this email, and its base64 encoded (I think ironport quarantine reports just used nice html) http://pastebin.com/1YXY5rPq should be easy enough to write sigs. look for ironport headers, and if none, block it. -- Michael Scheidell, CTO Phone: 561-999-5000, x 1259 *| *SECNAP Network Security Corporation * Certified SNORT Integrator * 2008-9 Hot Company Award Winner, World Executive Alliance * Five-Star Partner Program 2009, VARBusiness * Best Anti-Spam Product 2008, Network Products Guide * King of Spam Filters, SC Magazine 2008 __ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.secnap.com/products/spammertrap/ __
Re: Finding URLs in html attachments
On Mon, 1 Mar 2010, Benny Pedersen wrote:
On man 01 mar 2010 02:37:37 CET, John Hardin wrote
I've suggested this before, but the current position appears to be if the
MUA doesn't display it automatically, why should we scan it?
same goes for just enter this url when the sender was tired of doing it
right, fuzzyocr solved this
Justin, I would respectfully suggest this may no longer be a reasonable
position, at least for plain text and HTML attachments...
+1
Please correct me if I've misunderstood.
http://www.mail-archive.com/users@spamassassin.apache.org/msg64449.html
That doesn't answer the question of whether I misunderstood the position
Justin was taking on attachments.
Jonas, what's the current status of that plugin? It looks pretty stable to
me. And, can it extract from basic text attachments? I assume so...
extracttext_externaltext{CS:UTF-8} /bin/cat -
extracttext_use text.txt .htm .html text/(?:plain|html)
?
--
John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
---
We have to realize that people who run the government can and do
change. Our society and laws must assume that bad people -
criminals even - will run the government, at least part of the
time. -- John Gilmore
---
13 days until Albert Einstein's 131st Birthday
Re: [sa] Setting Blacklist_from and whitelist_to
On Sun, 28 Feb 2010, damuz wrote: Secondly, it occurred to me that all the (legit) mail to us will only be to a handful of email addresses and much of the spam still getting through is sent to spurious recipie...@mydomain.com. So with this in mind, is it useful or advisable to setup those legit email addresses as whitelist_to and if so, what becomes of the 'rest' of the mail or do you have to define only receive to whitelist_to? You have to 'fine tune' this kind of test. Keep in mind that the visible 'To:' header is hardly more than a *comment* on the mail. It may contain a mailing list name, or another *valid* recipient on another domain, while the mail was sent to *your* domain as a 'Bcc' hidden recipient. At the first stage of the SMTP transaction your MTA (should have) already rejected any mail that was actually 'addressed' to an invalid address. So the issue you are dealing with can be described as 'mail to a legitimate recipient with a suspicious To: header'. So it quickly devolves to the fact that the *only* thing you can reject is mail that has a 'To:' address that is @your.domain but which is not a valid (now or at any time in the past!) recipient on your domain. You can't flag mail that is 'To:' another domain. That could be valid! Now you need to be careful that when you invoke a 'whitelist' you do so for the 'To:' header, and NOT for the envelope recipient, which, by definition will always be a 'hit'. Unfortunately, the standard 'whitelist_to' will 'hit' on any embedded headers that your MTA adds to show the envelope recipient. You could essentially end up whitelisting all mail. So you need to whitelist on the visible headers *manually* So, if your list of internal recipients is not overly large, you may want to try the following: header __VALID_MYDOMAIN ToCc =~ /(validuser1|validuser2|...)\...@yourdomain.com/i header __TO_MYDOMAIN ToCc =~ /\...@yourdomain.com/i meta LOC_INVALID_MYDOMAIN ( __TO_MYDOMAIN ( ! __VALID_MYDOMAIN ) ) describe LOC_INVALID_MYDOMAIN Address in To or Cc header to invalid address on our domain scoreLOC_INVALID_MYDOMAIN 1 Obivously, score modestly until we are sure there are no false positives. The big 'problem' with this scheme is that *any* change to the list of valid users requires the first rule to be updated. So I only recommend this approach if you have absolute control over your mail system. - Charles
Re: Rule QA: Completeness / Preflight?
On 03/01, Justin Mason wrote: it's based on who's reported their logs -- give it time to complete. Thanks. nope -- preflights have been stopped, as they're quite CPU-intensive and we don't have the hardware. How about hit-frequencies output from the corpora used for sa-update updates? -- Dance, dance, wherever you may be - Lord of the Dance http://www.ChaosReigns.com
Re: [sa] Re: Finding URLs in html attachments
On Sun, 28 Feb 2010, LuKreme wrote: Your best bet is to check if mail claiming to be from paypal is, in fact, from paypal. Actually, I think his problem is that the reference to paypal has been buried in an attachment, described as 'type' of 'octet/binary' so that SA won't think it is text and scan it, and thus he doesn't *have* any 'visible' cue that the mail claims to be from paypal. And yes, I think that is a pretty serious problem. Looks like he may have to use a 'full' test to look for the references to paypal - C
DNSWL --report plugin
You must create an account here to use this: http://www.dnswl.org/registerreporter.pl It is still experimental. I expect it to work flawlessly. If it doesn't, please email me details off-list. It causes the spamassassin --report (or -r) command to also report to the DNSWL.org whitelist, for the RCVD_IN_DNSWL_* tests. Only use it on _manually_verified_ spam. This command also does what sa-learn --spam does (train your bayesian filter). Download http://www.chaosreigns.com/dnswl/dl/DNSWLh.pm to /usr/share/perl5/Mail/SpamAssassin/Plugin/ (or your equivalent) In your spamassassin config, add the following three lines: loadplugin Mail::SpamAssassin::Plugin::DNSWLh dnswl_address u...@example.com dnswl_password yourpassword (The last two values come from the registration link at the top of this email.) We would appreciate you running spamassassin --report with this plugin on all manually verified spams which have been mis-classified as non-spam. (Feel free to report all manually verified spams.) Spams over 48 hours old are automatically not reported. Note: If you're not doing any bayesian training, --report can be grumpy if you don't use the spamassassin config option: bayes_learn_during_report 0 I'm anxious to also support --revoke (reporting non-spam). -- The whole aim of practical politics is to keep the populace alarmed -- and hence clamorous to be led to safety -- by menacing it with an endless series of hobgoblins, all of them imaginary. - H. L. Mencken http://www.ChaosReigns.com
Re: Custom Rules Question
Because your first option matches the style inside the brackets and your second option does take into account the forward slash before style? Todd Michael Dilworth wrote: OK, it's late and I'm tired, and this will probably end up being stupid regex issue, but: why does... rawbody STYLE_IN_BODY /\body\.*style/si match and: rawbody STYLE_IN_BODY /\body\.*\style\/si not match? given a message body: htmlhead ... /headbody ... style garbage... /style ... /body /html No multipart, no Mime, just straight html as a message TIA michael... -- Todd Adamson Network Partners, Inc. tadam...@routers.com (402)434-5395 x3001
Re: DNSWL --report plugin
On 3/1/10 10:31 AM, dar...@chaosreigns.com wrote: You must create an account here to use this: http://www.dnswl.org/registerreporter.pl I did, thanks, using the manual reported. you need some way to exclude the reporters ip address. (i just reported a spam from badoo. and instead of reporting THEM as spammers, it recorded ME as the spammer) spamcop has a couple of options so that the 'last trusted' ip is NOT reported. one option would include a checkbox before reporting: list all dnsbl listed hosts in received and allow the reporter to exclude themselves. but, what to do with automated reports? spamcop makes you validate it before it reports it. so, how will we exclude the senders (registered) ip and report the real spammer? will the SA pm exclude all SA headers, including dropping any ip in the trusted or local nets? -- Michael Scheidell, CTO Phone: 561-999-5000, x 1259 *| *SECNAP Network Security Corporation * Certified SNORT Integrator * 2008-9 Hot Company Award Winner, World Executive Alliance * Five-Star Partner Program 2009, VARBusiness * Best Anti-Spam Product 2008, Network Products Guide * King of Spam Filters, SC Magazine 2008 __ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.secnap.com/products/spammertrap/ __
Re: DNSWL --report plugin
Michael Scheidell wrote: On 3/1/10 10:31 AM, dar...@chaosreigns.com wrote: You must create an account here to use this: http://www.dnswl.org/registerreporter.pl I did, thanks, using the manual reported. you need some way to exclude the reporters ip address. (i just reported a spam from badoo. and instead of reporting THEM as spammers, it recorded ME as the spammer) spamcop has a couple of options so that the 'last trusted' ip is NOT reported. one option would include a checkbox before reporting: list all dnsbl listed hosts in received and allow the reporter to exclude themselves. but, what to do with automated reports? spamcop makes you validate it before it reports it. so, how will we exclude the senders (registered) ip and report the real spammer? will the SA pm exclude all SA headers, including dropping any ip in the trusted or local nets? I can think of two ways to do this cleanly: 1) Make this a part of the account setup. When you create your account, you specify your mailserver IPs and then those IPs will not be reported based on submissions from your account. 2) Make it an SA option that the plugin can use to provide this information when sending the report. (Or even have the plugin grab the trusted/internal_networks setting and use that.) -- Bowie
Re: DNSWL --report plugin
On 03/01, Michael Scheidell wrote: you need some way to exclude the reporters ip address. Yep. I knew there was one, but it's apparently only currently usable by admins. Terrible. I deleted your submission. The reports are currently including the list of trusted and untrusted relays, so the last untrusted relay should be the IP we want. That's based on spamassassin's trusted_networks settings. But we're not processing that info yet. I'll... harass the necessary person. spamcop has a couple of options so that the 'last trusted' ip is NOT reported. You have somebody listed in trusted_networks that you want to report for spamming? one option would include a checkbox before reporting: list all dnsbl listed hosts in received and allow the reporter to exclude themselves. Yup. but, what to do with automated reports? spamcop makes you validate it before it reports it. Yup. so, how will we exclude the senders (registered) ip and report the real spammer? Still being decided. will the SA pm exclude all SA headers, including dropping any ip in the trusted or local nets? It does exclude all SA headers, just as --remove-markup or -d does. Doesn't look like it strips trusted / internal network IPs. Should be identical to what gets sent to SpamCop, since this module is mostly a copy of the SpamCop module. -- Blessed are the cracked, for they shall let in the light. http://www.ChaosReigns.com
Re: DNSWL --report plugin
On 3/1/10 11:05 AM, dar...@chaosreigns.com wrote: It does exclude all SA headers, just as --remove-markup or -d does. Doesn't look like it strips trusted / internal network IPs. Should be identical to what gets sent to SpamCop, since this module is mostly a copy of the SpamCop module. spamcop has a 'quick reporter' option that allows you to send test emails in/out, and once registered like that, spamcop will exclude those ips. spamcop ALSO looks at the envelope to (or to), and if the mx matches, they exclude. example: email went to scheid...@secnap.net. and the mx's for secnap.net matched a couple of the received headers, so they exclude those. -- Michael Scheidell, CTO Phone: 561-999-5000, x 1259 *| *SECNAP Network Security Corporation * Certified SNORT Integrator * 2008-9 Hot Company Award Winner, World Executive Alliance * Five-Star Partner Program 2009, VARBusiness * Best Anti-Spam Product 2008, Network Products Guide * King of Spam Filters, SC Magazine 2008 __ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.secnap.com/products/spammertrap/ __
Re: Rule QA: Completeness / Preflight?
On Mon, Mar 1, 2010 at 15:01, dar...@chaosreigns.com wrote: On 03/01, Justin Mason wrote: it's based on who's reported their logs -- give it time to complete. Thanks. nope -- preflights have been stopped, as they're quite CPU-intensive and we don't have the hardware. How about hit-frequencies output from the corpora used for sa-update updates? that's the ruleqa.spamassassin.org UI.
Re: Rule QA: Completeness / Preflight?
On 03/01, Justin Mason wrote: that's the ruleqa.spamassassin.org UI. Which data is used for the sa-updates? Just the latest random weekly network mass-check? -- Life is but a walking shadow, a poor player that struts and frets his hour upon the stage--and then is heard no more. It is a tale told by an idiot, full of sound and fury, signifying nothing. - William Shakespeare http://www.ChaosReigns.com
Re: Block Spammers Spoofing My Domain
On Sun, Feb 28, 2010 at 4:09 PM, Bill Landry b...@inetmsg.com wrote: Move the back-slash \ before the dot . (\.org) as you currently have it after the dot (.\org) Bill Bill - I got my example from Ralph Hildebrandt's Postfix config directly from his site: http://www.arschkrebs.de/postfix/#chapter5 Respectfully it's 3 years old but he does have it the exact way I do: /^localhost$/ 550 Don't use my own domain (localhost) /^arschkrebs\.de$/ 550 Don't use my own domain /^postfixshrine.\org$/ 550 Don't use my own domain /^88\.198\.105\.204$/ 550 Spammer comes to me, Greets me with my own IP, His mail I shall not see. /^\[88\.198\.105\.204\]$/ 550 Spammer comes to me, Greets me with my own IP, His mail I shall not see. /^fatush\.arschkrebs\.de$/ 550 Don't use my own hostname /^mail\.postfixshrine.\org$/550 Don't use my own hostname /^[0-9.-]+$/550 Your software is not RFC 2821 compliant: EHLO/HELO must be a domain or an address-literal (IP enclosed in []) - not a naked IP. Are you saying that his example is incorrect and wont properly work? It's strange because he has it as you suggest for the '.de' TLD however not for the '.org' for whatever reason and I don't understand why. Thanks for any clarification!
Putting your dead domains to use
For what it's worth - if any of you have domains you don't use you can point them to my virus harvesting server for spam harvesting. That gets rid of the spam coming to you and it helps block spam for everyone using my blacklist. Set the MX to a single entry: tarbaby.junkemailfilter.com Good email going to this host won't be blacklisted. The sender has to do several other things in order to be blacklisted. But the virus traffic does help protect other people from getting email from the same spambots that are spamming your old domains. This is for the hostkarma blacklist. Thanks in advance.
Re: Block Spammers Spoofing My Domain
Carlos Williams wrote: Bill - I got my example from Ralph Hildebrandt's Postfix config directly from his site: http://www.arschkrebs.de/postfix/#chapter5 Respectfully it's 3 years old but he does have it the exact way I do: /^localhost$/ 550 Don't use my own domain (localhost) /^arschkrebs\.de$/550 Don't use my own domain /^postfixshrine.\org$/550 Don't use my own domain /^88\.198\.105\.204$/ 550 Spammer comes to me, Greets me with my own IP, His mail I shall not see. /^\[88\.198\.105\.204\]$/ 550 Spammer comes to me, Greets me with my own IP, His mail I shall not see. /^fatush\.arschkrebs\.de$/550 Don't use my own hostname /^mail\.postfixshrine.\org$/ 550 Don't use my own hostname /^[0-9.-]+$/550 Your software is not RFC 2821 compliant: EHLO/HELO must be a domain or an address-literal (IP enclosed in []) - not a naked IP. Are you saying that his example is incorrect and wont properly work? It's strange because he has it as you suggest for the '.de' TLD however not for the '.org' for whatever reason and I don't understand why. Thanks for any clarification! The point is that there are a couple of typos in those rules. /^postfixshrine.\org$/ The backslash should come before the period. It will actually work the other way, but it allows any character to be there rather than restricting it to a literal period. The same is true for the other '.org' domain as well. \.org = period followed by 'org' .\org = any character followed by 'org' (the backslash is ignored) -- Bowie
Re: new (small) shortener campaign suggestion for URLRedirect
I think I'm misunderstanding something, but I'm not sure what. Please tell me why I'm confused. :-) On 2010-02-24 11:30, Chip M. wrote: Jonas, do you have any performance and/or efficacy stats for your URLRedirect plugin? Unfortunately, no. I am logging info from it (to the general mail log), but I haven't put anything together to analyze the logs. My main concern with doing real-time HTTP HEADs is performance. Well... Since the URLRedirect plugin inserts the redirection targets into the messages metadata so that other plugins (such as URIBL for example) can see those targets, the HEAD requests must be done before other rules. Currently the plugin only caches the results in simple perl structure, and in a pr instance manner. Puttingthe cache in shared memory or a database is an obvious improvement that should be done. This could make a big difference at high volume sites. Two reasons I haven't done much with the plugin, except adding support for Marc Perkel's URL shortener DNS list. is that I have no idea wether anyone except me actually uses the plugin, and I haven't seen much URL shortener spam since I made it. Other than the lack of cache, the plugin shouldn't be much f a performance problem. It does the HEAD requests in paralell (when possible), and has a runtime definebale timeout (default is 10 seconds, I do nt recommend raising that). It also has limits on the amount of requests done for one message. I would like to do the requests in paralell with other processing as well, but that's hard. It has to insert the targets into metadata before any rules or plugins that uses URIs, and I don't know of a way to do that other than having it run it's course before the regular rules. Jonas, I've been thinking that if you embedded the SA spam score in the HTTP request's Agent, that would provide BitLy/et-al with extremely useful data, which should improve their detection rate (if they choose to use the extra data). This would require the plugin to know the score before SpamAssassin has calculated it, wich is kind of difficult. I have not found any good algorithm implementing that kind of prescience. :-) It could insert the score(s) (or score aggregates) of *previous* messages into the user agent, but I think a general score aggregate system not tied to URL shortener services would be more suited for that. I for example might be interested in aggregate scores for mailes refering our web sites even though they are not URL shorteners. You could also include the recipient's domain, which may help them to correlate data. I'm not sure what they would do wth that, but again I think that kind of thing fits more into a general report framework than in a very specialized spamassassin plugin such as URLRedirect. I'm also wondering about using UDP to send a quick real-time, no-response-needed message (instead of a high overhead HTTP request), then (mostly) auto-quarantine, and later in a separate, batch queue, do a proper HTTP Head. Anything that's clean after a certain amount of time, could be automatically re-injected back into the main queue. That would allow pooling of requests, and shift the load from the main email gateway. I really don't understand this idea. What would the UDP packet contain? What would it be good for? You can do a SpamAssassin run in a batch queue allready. The URLRedirecft plugin doesn't care if SpamAssassin runs in a queue job, at delivery, in the receiving MTA, or wherever. When to call SA, wich is what calls URLRedirects methods, is completely outside the plugins control. If you want to queue some mails for later SA checking you should do that withoput having to run the same SA as you want to avoid running. You could do that by having some faster, leaner thing check the mail and decide to either send it to SA directly or to the slow queue. You could also run two SA configs where one has just about everything (including default rules) disabled. That SA could have rules used to determine wether to send the mail oin to the normal SA or the slow queue. In any case, it's hard for an SA plugin to decide wether SA should have been called or not, so this has to be done with more than just a SA plugin. The only impact of the URLRedirect plugin I can see in this is that you could use it to just see if there is an identified redirector URL in the message and use that to decide in the first (lean and fast) SA run in the two-SA-scenario above. If this is what you meant, I could easily implement an option to just identify redirectors rather than actually testing them with HEAD requests. The UDP part would be pointless without cooperation from the shortener services. If they do embrace it, they can use the UDP data to more quickly identify most bad links, and have that all ready for when we send out HTTP requests. I don't get this either. How would the UDP requests help them find bad links? How it help
Re: Finding URLs in html attachments
On Sun, 28 Feb 2010, LuKreme wrote:
On 28-Feb-10 17:25, David B Funk wrote:
I'm seeing a spate of PayPal/bank phishes that use an html attachment
(base-64 encoded) as the vehicle for the payload.
SPF!
runs; ducking, shucking, and weaving
Actually I'm happy to utilize SPF when I can. But westernunion.com
doesn't publish SPF records.
And this -is- a case in point where SPF is relevant to SA. ;)
Is there any way to get SA to treat that attachment as text to feed to
the rule engine?
Your best bet is to check if mail claiming to be from paypal is, in
fact, from paypal. Without checking SPF, you can at least check if the
server sending the mail is a paypal server using just header checks.
If you search the archives for paypal ebay you should find a few
solutions on how to deal with these.
I've got a number of PayPal specific rules, but these phishes are all
over the financial spectrum (banks, paypal, ebay, western-union, etc).
The clear-text verbage is vague enough to make writing FP-proof but
effective rules difficult. That's why I was hoping to be able to hit
against the HTML payload.
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Re: [sa] Re: Finding URLs in html attachments
On Mon, 1 Mar 2010, Charles Gregory wrote:
On Sun, 28 Feb 2010, LuKreme wrote:
Your best bet is to check if mail claiming to be from paypal is, in fact,
from paypal.
Actually, I think his problem is that the reference to paypal has been
buried in an attachment, described as 'type' of 'octet/binary' so that SA
won't think it is text and scan it, and thus he doesn't *have* any
'visible' cue that the mail claims to be from paypal. And yes, I think
that is a pretty serious problem.
Looks like he may have to use a 'full' test to look for the references to
paypal
Been there, done that, doesn't work.
AFAIK SA ignores 'octet/binary' attachments for the rule engine. None of
the rules that I tried (uri, body, full, rawbody) saw anything that was
known to be in one of those attachments.
Thus my query.
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Re: [sa] Re: Finding URLs in html attachments
On Mon, 1 Mar 2010, David B Funk wrote: Looks like he may have to use a 'full' test to look for the references to paypal Been there, done that, doesn't work. AFAIK SA ignores 'octet/binary' attachments for the rule engine. None of the rules that I tried (uri, body, full, rawbody) saw anything that was known to be in one of those attachments. You may have to examine the 'raw' message and look for 'encoding' that disguises the URI's in the attachment. Ths whole thing might be encoded as base64 or something... A real mess to work with. You might have more success making a rule that looks for mime headers that are type 'octet' but named 'html'. You won't be able to score that too high on its own, but it might combine well in a meta rule with certain buzz phrases from the text portions of the e-mail. - C
Re: Rule QA: Completeness / Preflight?
On Mon, Mar 1, 2010 at 17:09, dar...@chaosreigns.com wrote: On 03/01, Justin Mason wrote: that's the ruleqa.spamassassin.org UI. Which data is used for the sa-updates? Just the latest random weekly network mass-check? Yep, exactly. (with additional checks to ensure the data is good enough beforehand.) -- --j.
Re: [sa] Re: Finding URLs in html attachments
On Mon, 1 Mar 2010, Charles Gregory wrote: On Mon, 1 Mar 2010, David B Funk wrote: Looks like he may have to use a 'full' test to look for the references to paypal Been there, done that, doesn't work. AFAIK SA ignores 'octet/binary' attachments for the rule engine. None of the rules that I tried (uri, body, full, rawbody) saw anything that was known to be in one of those attachments. You may have to examine the 'raw' message and look for 'encoding' that disguises the URI's in the attachment. Ths whole thing might be encoded as base64 or something... A real mess to work with. You might have more success making a rule that looks for mime headers that are type 'octet' but named 'html'. I already have some rules for that in my sandbox, but IIRC they aren't scoring too well on ruleqa. You won't be able to score that too high on its own, but it might combine well in a meta rule with certain buzz phrases from the text portions of the e-mail. ...or look into the TextExtract plugin as Benny suggested. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Where We Want You To Go Today 07/05/07: Microsoft patents in-OS adware architecture incorporating spyware, profiling, competitor suppression and delivery confirmation (U.S. Patent #20070157227) --- 13 days until Albert Einstein's 131st Birthday
Re: can I roll back to an earlier version of updates
no joy. doesn't look like the ports version of SA comes with any stock rules (nothing obvious in the ports dir tree, the work/ directory had en empty 72_active.cf file)... I deinstalled and then installed and it all went well but it tells me to run sa-update to get the rules, and that's my problem You may wish to run sa-update now to obtain the latest rules. NOTE: FREEBSD users: If you are updating from a version prior to 3.20. sa-update now places state files in /var/db/spamassassin and not /var/lib/spamassassin. This is to be consistant with Freebsd file directory conventions. If you run sa-compile, you will notice that files are in /var/db/spamassassin/compiled/perlversion/version instead of /var/db/spamassassin/compiled/version. No attempts have been made to move old versions over. You must recompile. === Installing rc.d startup script(s) === Compressing manual pages for p5-Mail-SpamAssassin-3.3.0_3 === Running ldconfig /sbin/ldconfig -m /usr/local/lib === Registering installation for p5-Mail-SpamAssassin-3.3.0_3 r...@spock: /usr/ports/mail/p5-Mail-SpamAssassin $ sa-update config: failed to parse line, skipping, in /tmp/.spamassassin92852PBQ5Yktmp/72_active.cf: mimeheader __TVD_MIME_ATT_AOPDF Content-Type =~ /^application\/octet-stream.*\.pdf/i config: failed to parse line, skipping, in /tmp/.spamassassin92852PBQ5Yktmp/72_active.cf: mimeheader __TVD_MIME_ATT_AP Content-Type =~ /^application\/pdf/i config: failed to parse line, skipping, in /tmp/.spamassassin92852PBQ5Yktmp/72_active.cf: mimeheader __TVD_MIME_ATT_TP Content-Type =~ /^text\/plain/i channel: lint check of update failed, channel failed So is there *any* way for me to get this ruleset and put it on my server and edit out the offending lines in 72_active.cf?? Is there an archive I can download? (I'm thinking of modifying sa-update to comment-out where it removes the tmp files) -lee Karsten Bräckelmann wrote: On Mon, 2010-03-01 at 06:45 -0500, Lee Dilkie wrote: Karsten Bräckelmann wrote: Anyway, what comes to mind: Did you run sa-update after the upgrade to 3.3.0 at all? If not, did you install the rules tarball alongside SA? I was originally running the 3.3 rules and that was fine, and as far as I know, I did even run sa-upgrade (can't tell you if it upgraded the rules over the base ones) but it's the latest sa-update that pulled in newer rules that didn't link. And it's my monkeying around, deleting rules directories, that has left me without rules from updates spamassassin_org. And boy! do they block a lot of spam or what! ;) How did you upgrade? Any chance both versions ended up living on your system? Running 3.3.0 with a broken sa-update for whatever reason, can be cured by removing the entire update dir, and installing the plain, stock 3.3.0 rules tarball, if not already done. I'm on freebsd, I'm going to try and find out where that's stored, it's likely in the ports tree somewhere. man spamassassin See the section Configuration Files. The first path mentioned for Default Configuration Data should be the sa-update one. SA version is embedded in that path, inside /var/lib here, IIRC /var/db or something on FreeBSD. The last one in that block of paths should be where SA expects the stock rules. The first existing one from that list wins, anything else will be ignored. spamassassin -D can help in identifying bad rule sets being picked up, and where SA ultimately looks for the cf files. Is there any way that I can force the system to download the ruleset so I can comment out the offending lines and carry on? (I'd at least like to see what they are, and why it doesn't parse, maybe it's something in my config). Drop the bad update first, and revert to stock. Re-install it from ports, if need be. -- Fuelly http://www.fuelly.com/driver/dilkie/golf
Re: can I roll back to an earlier version of updates
progress report.. commented out the place where the lint results were
checked and rules got installed.
looking at 72_active.cf I see a number of lines ending in CR (^M). Is
this intentional?
ie.
header __SUBJ_3DIGIT Subject =~ /\b\d{3}[^0-9]/^M
header __SUBJ_APPROVE Subject =~ /Approve/i^M
header __SUBJ_RE Subject =~ /^R[eE]:/^M
-lee
Lee Dilkie wrote:
no joy.
doesn't look like the ports version of SA comes with any stock rules
(nothing obvious in the ports dir tree, the work/ directory had en empty
72_active.cf file)... I deinstalled and then installed and it all went
well but it tells me to run sa-update to get the rules, and that's my
problem
You may wish to run sa-update now to obtain the latest rules.
NOTE: FREEBSD users: If you are updating from a version prior to 3.20.
sa-update now places state files in /var/db/spamassassin and not
/var/lib/spamassassin. This is to be consistant with Freebsd file
directory conventions.
If you run sa-compile, you will notice that files are in
/var/db/spamassassin/compiled/perlversion/version instead of
/var/db/spamassassin/compiled/version.
No attempts have been made to move old versions over. You must recompile.
=== Installing rc.d startup script(s)
=== Compressing manual pages for p5-Mail-SpamAssassin-3.3.0_3
=== Running ldconfig
/sbin/ldconfig -m /usr/local/lib
=== Registering installation for p5-Mail-SpamAssassin-3.3.0_3
r...@spock: /usr/ports/mail/p5-Mail-SpamAssassin
$ sa-update
config: failed to parse line, skipping, in
/tmp/.spamassassin92852PBQ5Yktmp/72_active.cf: mimeheader
__TVD_MIME_ATT_AOPDF Content-Type =~ /^application\/octet-stream.*\.pdf/i
config: failed to parse line, skipping, in
/tmp/.spamassassin92852PBQ5Yktmp/72_active.cf: mimeheader
__TVD_MIME_ATT_AP Content-Type =~ /^application\/pdf/i
config: failed to parse line, skipping, in
/tmp/.spamassassin92852PBQ5Yktmp/72_active.cf: mimeheader
__TVD_MIME_ATT_TP Content-Type =~ /^text\/plain/i
channel: lint check of update failed, channel failed
So is there *any* way for me to get this ruleset and put it on my server
and edit out the offending lines in 72_active.cf?? Is there an archive I
can download? (I'm thinking of modifying sa-update to comment-out where
it removes the tmp files)
-lee
Karsten Bräckelmann wrote:
On Mon, 2010-03-01 at 06:45 -0500, Lee Dilkie wrote:
Karsten Bräckelmann wrote:
Anyway, what comes to mind: Did you run sa-update after the upgrade to
3.3.0 at all? If not, did you install the rules tarball alongside SA?
I was originally running the 3.3 rules and that was fine, and as far as
I know, I did even run sa-upgrade (can't tell you if it upgraded the
rules over the base ones) but it's the latest sa-update that pulled in
newer rules that didn't link. And it's my monkeying around, deleting
rules directories, that has left me without rules from updates
spamassassin_org. And boy! do they block a lot of spam or what! ;)
How did you upgrade? Any chance both versions ended up living on your
system?
Running 3.3.0 with a broken sa-update for whatever reason, can be cured
by removing the entire update dir, and installing the plain, stock 3.3.0
rules tarball, if not already done.
I'm on freebsd, I'm going to try and find out where that's stored, it's
likely in the ports tree somewhere.
man spamassassin
See the section Configuration Files. The first path mentioned for
Default Configuration Data should be the sa-update one. SA version is
embedded in that path, inside /var/lib here, IIRC /var/db or something
on FreeBSD.
The last one in that block of paths should be where SA expects the stock
rules. The first existing one from that list wins, anything else will be
ignored.
spamassassin -D can help in identifying bad rule sets being picked up,
and where SA ultimately looks for the cf files.
Is there any way that I can force the system to download the ruleset so
I can comment out the offending lines and carry on? (I'd at least like
to see what they are, and why it doesn't parse, maybe it's something in
my config).
Drop the bad update first, and revert to stock. Re-install it from
ports, if need be.
--
Fuelly http://www.fuelly.com/driver/dilkie/golf
Re: can I roll back to an earlier version of updates
Final update folks, sorry for the noise if it's bothersome...
commented out the three offending lines in 72_active.cf and --lint
passed and I'm back up and running.
No idea what the issue is, those lines looked fine to me. I'm running
perl 5.8.9, could that be an issue?
-lee
details: ##lee is my handiwork
ifplugin Mail::SpamAssassin::Plugin::MIMEHeader
mimeheader __TVD_FW_GRAPHIC_ID1 Content-Id =~
/[0-9a-f]{12}(?:\$[0-9a-f]{8}){2}\@/
endif
ifplugin Mail::SpamAssassin::Plugin::MIMEEval
##lee mimeheader __TVD_MIME_ATT_AOPDF Content-Type =~
/^application\/octet-stream.*\.pdf/i
endif
ifplugin Mail::SpamAssassin::Plugin::MIMEEval
##lee mimeheader __TVD_MIME_ATT_AP Content-Type =~ /^application\/pdf/i
endif
ifplugin Mail::SpamAssassin::Plugin::MIMEEval
##lee mimeheader __TVD_MIME_ATT_TP Content-Type =~ /^text\/plain/i
endif
ifplugin Mail::SpamAssassin::Plugin::MIMEHeader
mimeheader __TVD_OUTLOOK_IMGContent-Id =~ /image\d+\.(?:gif|jpe?g|png)\@/
endif
Lee Dilkie wrote:
progress report.. commented out the place where the lint results were
checked and rules got installed.
looking at 72_active.cf I see a number of lines ending in CR (^M). Is
this intentional?
ie.
header __SUBJ_3DIGIT Subject =~ /\b\d{3}[^0-9]/^M
header __SUBJ_APPROVE Subject =~ /Approve/i^M
header __SUBJ_RE Subject =~ /^R[eE]:/^M
-lee
Lee Dilkie wrote:
no joy.
doesn't look like the ports version of SA comes with any stock rules
(nothing obvious in the ports dir tree, the work/ directory had en empty
72_active.cf file)... I deinstalled and then installed and it all went
well but it tells me to run sa-update to get the rules, and that's my
problem
You may wish to run sa-update now to obtain the latest rules.
NOTE: FREEBSD users: If you are updating from a version prior to 3.20.
sa-update now places state files in /var/db/spamassassin and not
/var/lib/spamassassin. This is to be consistant with Freebsd file
directory conventions.
If you run sa-compile, you will notice that files are in
/var/db/spamassassin/compiled/perlversion/version instead of
/var/db/spamassassin/compiled/version.
No attempts have been made to move old versions over. You must recompile.
=== Installing rc.d startup script(s)
=== Compressing manual pages for p5-Mail-SpamAssassin-3.3.0_3
=== Running ldconfig
/sbin/ldconfig -m /usr/local/lib
=== Registering installation for p5-Mail-SpamAssassin-3.3.0_3
r...@spock: /usr/ports/mail/p5-Mail-SpamAssassin
$ sa-update
config: failed to parse line, skipping, in
/tmp/.spamassassin92852PBQ5Yktmp/72_active.cf: mimeheader
__TVD_MIME_ATT_AOPDF Content-Type =~ /^application\/octet-stream.*\.pdf/i
config: failed to parse line, skipping, in
/tmp/.spamassassin92852PBQ5Yktmp/72_active.cf: mimeheader
__TVD_MIME_ATT_AP Content-Type =~ /^application\/pdf/i
config: failed to parse line, skipping, in
/tmp/.spamassassin92852PBQ5Yktmp/72_active.cf: mimeheader
__TVD_MIME_ATT_TP Content-Type =~ /^text\/plain/i
channel: lint check of update failed, channel failed
So is there *any* way for me to get this ruleset and put it on my server
and edit out the offending lines in 72_active.cf?? Is there an archive I
can download? (I'm thinking of modifying sa-update to comment-out where
it removes the tmp files)
-lee
Karsten Bräckelmann wrote:
On Mon, 2010-03-01 at 06:45 -0500, Lee Dilkie wrote:
Karsten Bräckelmann wrote:
Anyway, what comes to mind: Did you run sa-update after the upgrade to
3.3.0 at all? If not, did you install the rules tarball alongside SA?
I was originally running the 3.3 rules and that was fine, and as far as
I know, I did even run sa-upgrade (can't tell you if it upgraded the
rules over the base ones) but it's the latest sa-update that pulled in
newer rules that didn't link. And it's my monkeying around, deleting
rules directories, that has left me without rules from updates
spamassassin_org. And boy! do they block a lot of spam or what! ;)
How did you upgrade? Any chance both versions ended up living on your
system?
Running 3.3.0 with a broken sa-update for whatever reason, can be cured
by removing the entire update dir, and installing the plain, stock 3.3.0
rules tarball, if not already done.
I'm on freebsd, I'm going to try and find out where that's stored, it's
likely in the ports tree somewhere.
man spamassassin
See the section Configuration Files. The first path mentioned for
Default Configuration Data should be the sa-update one. SA version is
embedded in that path, inside /var/lib here, IIRC /var/db or something
on FreeBSD.
The last one in that block of paths should be where SA expects the stock
rules. The first existing one from that list wins, anything else will be
ignored.
Re: spammers targeting ironport quarantine now: phish
On Mon, Mar 1, 2010 at 5:56 AM, Michael Scheidell scheid...@secnap.netwrote: Imagine my surprise this am when I got a quarantine report from our ironport email server (when I don't have one!) Phishers targeting ironport users now. if anyone has ironport, can you look at this email to see if it looks like an ironport quarantine report? I do notice the lack of ironport headers in this email, and its base64 encoded (I think ironport quarantine reports just used nice html) http://pastebin.com/1YXY5rPq should be easy enough to write sigs. look for ironport headers, and if none, block it. Actually, all signs point to this being a misconfigured IronPort appliance. Someone at IronPort is attempting to contact the administrator. Daniel
Re: Finding URLs in html attachments
On 01-Mar-10 12:45, David B Funk wrote: AFAIK SA ignores 'octet/binary' attachments for the rule engine. None of the rules that I tried (uri, body, full, rawbody) saw anything that was known to be in one of those attachments. So there was no paypal info (spoofed) in the headers at all? But yeah, not being able to scan the 'binary' attachment sounds like a real issue. -- This is our music from the bachelor's den, the sound of loneliness turned up to ten. A harsh soundtrack from a stagnant waterbed and it sounds just like this. This is the sound of someone losing the plot making out that they're OK when they're not. You're gonna like it, but not a lot. And the chorus goes like this...
Spamhaus DBL
http://www.spamhaus.org/dbl/ I think sa-folks would have this already in some URIBL rule. What are the scores you assign for a dbl positive hit ? I assume my current datafeed would already extend to data access on the dbl list. I will have to setup my rbldnsd before trying this out.
