I am a concerned about the simplistic approach of only testing the accuracy of this devices. Many other parameters should be measured, including the amount of bandwidth the device can handle, the time from attack detection to reporting, etc. Not to mention how the alerts are reported, etc
Now, if you want to only test the accuracy, then you should probably generate "normal" traffic and see if there are false positives, and then mix that traffic with attack traffic, collected from a honeypot, for example. You should also try typical evasion tools, like libwhisker, to check how the devices handle fragmentation and others evasion techniques. Or of course, use some commercial product, ias other posted suggested, if it has a good set of tests. Some good literature on that http://perso.rd.francetelecom.fr/debar/papers/DebMor02.pdf by Herve Debar et all http://csrc.nist.gov/publications/nistir/nistir-7007.pdf http://users.ece.gatech.edu/~owen/Research/Conference%20Publications/Athanasiades_IIAW2003.pdf Best, Jesus On 1/11/06, Aaron Turner <[EMAIL PROTECTED]> wrote: > On 1/7/06, Andres Riancho <[EMAIL PROTECTED]> wrote: > > You could use tcpsic for testing how well the appliance handles > > fragmented packets, you could use nikto and nessus to see how many > > attacks each one detects a > > Why would you use a vulnerability scanner to see how an UTM detects an > attack? Vuln scanners nowadays go to great lengths NOT to actually > exploit a vulnerability because people get pissed when their servers > crash. > > If you want to see how well an UTM detects someone running nessus > against your network, then by all means use nessus. But if you want > to see how well it detects an actual attack, then you had better use > traffic which actually looks like an actual exploit. > > With that said, the original poster should go look in the list > archives and google. Someone asks how to test an > IDS/IPS/Firewall/UTM/etc on this list every virtually every week and > it's pretty boring reading the same answers/vendors reminding us that > nothing significant has changed in the last year or so. > > > -- > Aaron Turner > http://synfin.net/ > > ------------------------------------------------------------------------ > Test Your IDS > > Is your IDS deployed correctly? > Find out quickly and easily by testing it > with real-world attacks from CORE IMPACT. > Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 > to learn more. > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------ Test Your IDS Is your IDS deployed correctly? Find out quickly and easily by testing it with real-world attacks from CORE IMPACT. Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 to learn more. ------------------------------------------------------------------------
