There are two kinds of AI/ML:1. the kind that recognizes what humans recognize
(faces, cars, etc.)2. the kind that recognizes things humans can't see (stock
market trends, etc.)
The first item is real, and is slowly changing the world. The second is bogus,
snake oil, emperors without clothes.
As long as I've been in the field of network intrusion detection (more than 2
decades), there have been a stream of papers every year promising machines can
see evil on the network that humans couldn't see. They've never worked in
practice.
That's not to say good things don't exist. Arbor Networks, for example, does
fine job at pointing out anomalies. But it's based on human ingenuity, not
machine learning, and it requires human effort to use.
On Wednesday, June 21, 2017 10:40 AM, dave aitel <[email protected]>
wrote:
Let's talk about the giant pile of wrong that is this reporting on Cisco's
new marketing campaign around detecting encrypted malware traffic. "This is a
seminal moment in networking" is the quote from their CEO that CNBC decided to
run. Let's revisit the basics of this "new" technology: do statistical analysis
on encrypted data to find malware traffic.
People have literally decoded conversations from encrypted data using that
same basic technique. Not even recently - that work is from 2008 and was not
surprising even then.
"The software, which will be offered as a subscription service, is currently
in field trials with 75 customers, and according to Robbins, is 99 percent
effective." 99% effective with the kind of traffic a normal network sees means
you are FLOODED AND OVERWHELMED WITH FALSE POSITIVES. Although they don't
specify what that number even means. Is it false positives? False negatives?
Both? Let's just say this: 99.99% is useless when doing a network-based IDS.
All that might get you is an indicator you can use to remotely load a more
sophisticated remote tool onto an endpoint for further detailed analysis. You
essentially, need BOTH if you have this level of network-based IDS, and the
endpoint people will probably say you don't need the network sniffer anymore,
because scaling good analysis at that level at anything near realtime is nearly
impossible (c.f. Alex Stamos's talk) to the point where they still try to sell
you stuff that has 1% false positive rates. :) I'm going to bug our big
customers to see if any of them are in this 75 field trial and what they think
in real life. And I'm going to be honest and say that if you are thinking of
investing in this sort of thing, but you haven't tested it against Cobalt
Strike and INNUENDO, then you are knowingly buying snake oil. A good percentage
of our consulting business right now is literally just that because these
anomaly detection products are so expensive and so hard to test. Anyways, maybe
I am wrong! If you are one of the privileged 75 and you love this and it is
amazing, let me/us know!
-dave
_______________________________________________
Dailydave mailing list
[email protected]
https://lists.immunityinc.com/mailman/listinfo/dailydave
_______________________________________________
Dailydave mailing list
[email protected]
https://lists.immunityinc.com/mailman/listinfo/dailydave
