This has been an area of interest for me for some time. It's very true the regexp based detection technologies can produce high rates of false positives and are easily evaded. It's not uncommon for data leaks to take place over vpns; a case study like this was presented at blackhat this year. Even without encryption, the number of possible obfuscation techniques is quite large (and we're assuming the data is ASCII; there are probably enough obscure back end applications with binary protocols to keep a good sized protocol dissector development team frustrated indefinitely).
I've seen some good success combining specification based techniques - like these regexps - with behavioral detection - such as using netflow or other flow data, for example, to detect unexpected large or long duration data streams headed for places that don't makes sense (e.g. foreign networks, foreign countries or external networks with which no business relationship exists). It seems to often be the case that systems containing high-value data have a predictable enough network behavioral repertoire that this kind of behavioral detection performs acceptably. This kind of behavioral detection, optionally corroborated with available specification based detection such as regexp detects, can have acceptably low false positive rates. Another advantage of flow data is that it is hard to evade detection of the fact that you're moving a lot of data; you can obfuscate and encrypt the traffic but you can't conceal the fact that a quantity of traffic (and presumably data, if the payload is not garbage) is being transmitted. Of course, if an obvious attack of some sort precedes all of this - with a resulting detect or detects from an IDS to corroborate - then confidence is again higher. Regards, Craig Chamberlain Principal Security Consultant [EMAIL PROTECTED] | www.q1labs.com -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Ofer Shezaf Sent: Tuesday, October 02, 2007 7:25 AM To: [EMAIL PROTECTED]; [email protected] Subject: RE: Using Snort to find creditcard data? All the answers where good but also partial as the subject is far from trivial. There are few aspects to detecting credit card numbers on the network, and I will try to address them: 1. Matching credit card numbers 2. Handling false positives 3. Evasion 4. Logging Matching Credit Card Numbers ============================ Valid card numbers: 1. Are 13-16 digits long. This is easy to detect using regular expressions but may result in a lot of false positives. A lot of IDs are in this range. 2. Conform to the LUAN checksum function. Being a checksum function it matches 1 out of 10 numbers in the range. Since many times applications that use numbers of this length use an entire range, there will still be false positives. LUAN cannot be verified using regular expressions and would require code. 3. Have certain prefixes which were assigned to issuers. A pretty good table of assigned prefixes can be found in Wikipedia, but I'm not sure it is comprehensive (http://en.wikipedia.org/wiki/Credit_card_number). Prefixes further reduce false positives and can be implemented using a (complex) regular expression. Using prefixes introduce a risk of false negatives due to omission of less common prefixes. For example we have not been aware until recently of Bankcard from Austria. This is especially a problem internationally. False positives =============== The problem is that the above rules generate a lot of false positives. Most false positives are related to normal application traffic using long ASCII numbers. Such an application would usually use a range and therefore hit some valid numbers. Since the PCI requirement is for "Encrypt transmission of cardholder data (only) across open, public networks", another source of false positives are applications that transmit credit card numbers intentionally and legally. The solution for such false positives would be exceptions, which I'm not sure Snort is the best solution for and would require an application layer IDS. A network layer exception would be limited to addresses and ports while a good exception would be by a specific property of the transaction such as URL and parameter (for HTTP traffic). For web traffic I would use for example something like ModSecurity. But I'm biased. Evasion ======= It is important to note that any such mechanism will detect only erroneous use of credit card numbers. Even the simplest transformation function on the numbers will enable them to bypass detection, so most malicious usage would not be detected. There is also an issue with leakage through encrypted channels, since PCI requires encryption, leakage would many times be encrypted. IDS limitations regarding encrypted traffic have been discussed extensively here (http://archives.neohapsis.com/archives/sf/ids/2007-q3/0084.html) and elsewhere. Logging: ======== Assuming that we did everything right and built a system for detecting credit card numbers on the network, we cannot keep the number as we would violate PCI in the detection system. Solutions are: (a) Encrypt all collected information (b) Mask the credit card number ~ Ofer Shezaf Ofer Shezaf [EMAIL PROTECTED], Phone:+972-9-9560036 #212, Cell: +972-54-4431119 CTO, Breach Security, www.breach.com Chair, OWASP Israel, www.owasp.org/index.php/israel Officer, Web Applicaiton Security Consortium, www.webappsec.org Leader, ModSecurity Core Rule Set Project, www.modsecurity.org/projects/rules/ Maintainer, Web Hacking Incidents Database, www.webappsec.org/projects/whid > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] > Sent: Wednesday, September 26, 2007 9:36 PM > To: [email protected] > Subject: Using Snort to find creditcard data? > > Would it be possible to write a Snort rule that triggers on possible > creditcard numbers and how would it look like? > > PCI standars says that all creditcard data should be encrypted, It > woild be nice to verify that no card data shows up where it > shouldn't... > > ----------------------------------------------------------------------- > - > Test Your IDS > > Is your IDS deployed correctly? > Find out quickly and easily by testing it with real-world attacks from > CORE IMPACT. > Go to > http://www.coresecurity.com/index.php5?module=Form&action=impact&campai > gn=intro_sfw > to learn more. > ----------------------------------------------------------------------- > - ------------------------------------------------------------------------ Test Your IDS Is your IDS deployed correctly? Find out quickly and easily by testing it with real-world attacks from CORE IMPACT. Go to http://www.coresecurity.com/index.php5?module=Form&action=impact&campaig n=intro_sfw to learn more. ------------------------------------------------------------------------ ------------------------------------------------------------------------ Test Your IDS Is your IDS deployed correctly? Find out quickly and easily by testing it with real-world attacks from CORE IMPACT. Go to http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw to learn more. ------------------------------------------------------------------------
