First: thanks for the followup/information/analysis. Most helpful. Second:
On Fri, May 17, 2013 at 10:10:24AM -0400, Jon Camfield wrote: > I'm doing some follow-up tests to see if it follows redirects, links > posted without http:// or https:// , links without www.* and so on. > This could inform the utility of (a) (I'm arguing as a devil's advocate > here). Given that MS might have an existing catalog of malware sites > and/or a separate method for finding new ones; this HEAD scanning may be > looking for new, unknown redirects to known malware sites. (However, > this wouldn't find in-page redirects or javascript redirects/additions, > and a number of other "popular" malware/adspam distribution tools). I agree. But in addition to these issues, this approach (if it's what they're using) is just about guaranteed to fail. Consider: to a decent first approximation, any page on any site may be hosting malware at any time. We see instances of this daily, sometimes because the site is compromised, other times because it includes content from another site (e.g., an advertising network) that's been compromised. And this is before we even get to the myriad sites that are hosting malware on purpose. My point being that examination of page P at time T1 tells you nothing about page P at time T2, until/unless you've accumulated a sufficient number of observations at (T1, T2, ..., Tn) that allow you to say something like "Hey...y'know, page P has been hosting malware for the last 289 days...it's probably hosting malware now, too." Unfortunately, this doesn't work the other way: the absence of malware on page P for 289 days doesn't provide much confidence it's not there now. Second, anyone hosting malware on purpose or who has managed to gain administrative control of the web server hosting the site/page can set it up to serve different content in response to HTTP requests from Microsoft (or Trend Micro or Kaspersky or whatever) networks than it would elsewhere. They can also vary content by user-agent (and that's probably useful when trying to serve up different exploits for different browsers). Both of these are old spammer tricks; surely Microsoft's security people have to be aware of them. My point here being that scanning from one's own network allocation is sometimes not very effective. Third, malware detection is, well, a joke. Test after test after test shows that even ridiculously expensive packages miss all kinds of stuff. (That includes Microsoft Essentials, by the way, although in their defense, ALL the products suck so badly that I can't really fault them for this.) And of course any malware author who's motivated can pre-test their work against any number of them and specifically craft it to avoid detection. To put it another way, given a sufficiently clueful and resourceful malware author, the initial detection rate across all products should be 0. Annnnd, sufficiently clueful and resourceful malware authors already exist and are getting better all the time. Bottom line: either Microsoft is telling the truth, in which case this was a hopelessly inept and ridiculously ineffective "malware scanning" exercise, or they're lying and just threw this fabricated story against the wall to see if it would stick. My money's on the latter: I think they're evil, not stupid. ---rsk -- Too many emails? Unsubscribe, change to digest, or change password by emailing moderator at [email protected] or changing your settings at https://mailman.stanford.edu/mailman/listinfo/liberationtech
