Rahul: It's true that the baseline set of criteria does focus on remotely exploitable, server-side vulnerabilities found in enterprise software. What is less well known is that there is an optional criteria module focused entirely on coverage protection for client-side vulnerabilities (also in enterprise software). As with the evolving server-side set of vulnerabilities, this client-side set is published online as well. You can find it here:
http://www.icsalabs.com/icsa/docs/html/communities/nips/criteria/Vulnera bilitySet_ClientSide_070703.xls So, when we do the research twice a year (approximately) to determine the vulnerability set - it's done for both the server-side set required by the baseline set of criteria as well as for the optional client-side set. You probably noticed that no one has been tested successfully against this optional module. Enterprise end users may have to demand that products be tested for client-side vulnerability coverage protection (if they are interested in ensuring proper protection for attacks targeting both sets) before developers will pursue such testing. I can and have recommended it, but it really does take a push from end users. As I said before, if folks have questions about ICSA Labs Network IPS testing, please feel free to get in touch. Take care, Jack Walsh Technology Programs Manager, Intrusion Detection & Prevention ICSA Labs 717.790.8126 [EMAIL PROTECTED] > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Rahul K > Sent: Tuesday, December 11, 2007 2:03 PM > To: Stefano Zanero > Cc: Focus-Ids Mailing List > Subject: Re: ICSA Labs Network IPS Testing > > Hi, > > Having some experience in developing and testing IPS, I have my two > bits to add. Most IPS tests, like Stefano said, are tricky at best and > pointless at worst. I don't want to take any potshots at ICSA or > anyone else, but it is not simple for anyone to do an exhaustive test > of an IPS and that too with the same test plan for every IPS. > > ICSA, to their credit, say that of all the vulnerabilities they will > only focus on remote server-side vulnerabilities and that too only > those that they (and other vendors) think will affect enterprises. > Fair enough. They don't care about client-side vulns, local vulns and > vulnerabilities in Shoutcast. > > They test a particular subset (however small it may be) and certify > the IPS. So even if one buys an IPS that blocks all server side > attacks launched by ICSA, it does not mean that the server behind the > IPS is secure from remote attacks. Vendors and buyers need such > certifications so that it is easier to make a sale and deploy an IPS > respectively - after all, not everyone subscribes to focus-ids. > > It would be reasonable to criticize ICSA if one finds out they are not > doing what they promise correctly. But if the criticism is for not > testing exhaustively, that seems excessive. > > Cheers, > Rahul > > On 12/5/07, Stefano Zanero <[EMAIL PROTECTED]> wrote: > > Hi, didn't mean to interfere in your ongoing flame, but: > > > > > IPS certification testing, I thought I ought to correct > some misleading > > > information > > > > Oh, good, let's see! You don't mind if instead of going through your > > whitepapers I just use your own email as a source, right? > > > > > IPS certification testing program. The truth is that we > do not "pick > > > specific attacks and say that you must block these." > > > > That's wonderful to hear. So, what do you do instead? > > > > > provides coverage protection for all attacks targeting an > evolving set > > > of medium-to-high severity vulnerabilities that we and a > consortium of > > > 15 network IPS vendors > > > > (http://www.icsalabs.com/icsa/topic.php?tid=6a87$5813f3e2-37b7 7ee3$3b4a- > > > f1d4a32d) believe are relevant to enterprise end users. > > > > So, you pick specific attacks (which are a snapshot of a set of > > vulnerabilities that you + the tested vendors believe are > relevant) and > > say "you must block these", right ? > > > > This seems exactly the same sentence that Joel posted, only > a bit more > > elaborate :) > > > > And just to shoot another shot in the dead horse of IDPS testing, > > testing MISUSE based detectors (as most IPS are) on > "detection rate" is > > pointless. Testing them on coverage is tricky at best, and does not > > really provide any useful insight at all on IPS where (as > Joel pointed > > out) having 60k signatures instead of 30k does not really > mean anything. > > > > Oh, and on a side note: > > > > > a) is in no position to speak authoritatively about ICSA > Labs network > > > IPS testing, > > > > The sheer fact that someone is "in no position to speak" about your > > tests means that your tests are lacking. If a test is properly > > documented and scientific, everybody is in a position to > speak about it. > > > > In the particular case of Joel Snyder, who has been doing excellent > > tests for a long time, I'd say he is in a particularly good > position to > > comment. > > > > If this email sounds harsh, well, it is. I just don't like people > > commenting AGAINST other people, instead than pointing out > the specific > > flaws in their posts. > > > > Best, > > Stefano > > > > > -------------------------------------------------------------- > ---------- > > Test Your IDS > > > > Is your IDS deployed correctly? > > Find out quickly and easily by testing it > > with real-world attacks from CORE IMPACT. > > Go to > http://www.coresecurity.com/index.php5?module=Form&action=impa ct&campaign=intro_sfw > > to learn more. > > > -------------------------------------------------------------- > ---------- > > > > > > -------------------------------------------------------------- > ---------- > Test Your IDS > > Is your IDS deployed correctly? > Find out quickly and easily by testing it > with real-world attacks from CORE IMPACT. > Go to > http://www.coresecurity.com/index.php5?module=Form&action=impa ct&campaign=intro_sfw > to learn more. > -------------------------------------------------------------- > ---------- > > ------------------------------------------------------------------------ Test Your IDS Is your IDS deployed correctly? Find out quickly and easily by testing it with real-world attacks from CORE IMPACT. Go to http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw to learn more. ------------------------------------------------------------------------
