Andres, >> For unit testing, PyUnit (http://pyunit.sourceforge.net/) is the obvious >> choice for Python. > > Cool, it seemed like that to Javier and I too, but we needed the >community confirmation :)
I think we all agree on PyUnit :) . With PyUnit we may be already able to get 50% to 70% code (test) coverage. >> However, as soon as you start talking http server (or any server), by >> definition you aren't dealing with unit tests anymore. > > I'm not sure how deep you're into the codebase, but... how would >you test a function that spiders a website? The function takes one URL >as input, requests that resource, parses the result and then fetches >all the URLs that came out of the parser. If we don't have some type >of http server, how can we do it? I think we should distinguish between unit tests (the code does what we want) and performance tests. For performance tests (for the entire framework) I would still use a server like before (Moth [0] to be precise). > >> For integration and functional testing, most of the frameworks rely on >> driving a browser (e.g., Selenium) or simulating a browser. Since neither of >> those is the case here, I think your best bet is still to test with PyUnit, >> just not for unit tests. It probably needs some strong helper code around it >> to handle the pieces like starting the server, set up the pages, and help >> abstract the configuration of the module. > > Agreed, > >> Still with PyUnit, you could mock the server and pages (e.g., >> mockito-python, http://code.google.com/p/mockito-python/), but I'm not sure >> that is what you are looking for. That would do true unit testing and ensure >> that the code was doing what you intended, but would test to see if a plugin >> actually was effective at identifying a vulnerability, for example. > > My objective is to be able to write test cases that "run" the >plugin with 30 different inputs, and verify that I get the expected >responses. > If we talk about unit tests I think it is not a good idea to use a real server. The tests would take very long, because the HTTP requests have to be really sent over the network. Tests should be easy to execute. Everybody who does a SVN checkout should be able to run 1 command on the command line and see which tests fail (without the need of a web server). I think the best would be to mock out the methods of the xUrllib. Example for the xss plugin (see Gene's link to mockito [1]): responseHeaders = {"Content-Type":"text/html"} xUrllibMock = mock(xUrllib) payload = '</A/style="xss:exp/**/ression(alert(\'XSS\'))">' url = "http://example.com/a.php?abc="+payload #Now here's the real magic happening: when(xUrllibMock).GET(url).thenReturn(httpResponse(200, payload, responseHeaders, url, url)) #Now use the mock object basePlugin._urlOpener = xUrllibMock #Now we would start the scan and check that the XSS was found with asserts or whatever ... Of course this is a LOT of work. We have to rebuild real-world vulnerabilites in our test cases. >> Both types of testing have value, but the integration testing is what I >> think you are looking for. > > Interesting... I'll investigate and check if there is some kind of >integration testing methodology we could use apart from the unit >testing. > >Regards, > I think it is a good idea to start with PyUnit and easy methods (that don't need mock objects). Just what came to my mind. cheers floyd [0] http://www.bonsai-sec.com/en/research/moth.php [1] http://code.google.com/p/mockito-python/ >> >> On Fri, Oct 1, 2010 at 11:46 AM, Andres Riancho <andres.rian...@gmail.com> >> wrote: >>> >>> Guys, >>> >>> Anyone has experience with unit testing frameworks? Which one do >>> you recommend for integrating into w3af? We need something that will >>> allow us to run an http server, setup dynamic pages inside it, setup >>> the plugin before running it, and finally run the test themselves. I'm >>> attaching some pictures we took with Javier about how it would work. >>> >>> Regards, >>> -- >>> Andrés Riancho >>> Founder, Bonsai - Information Security >>> http://www.bonsai-sec.com/ >>> http://w3af.sf.net/ >>> >>> >>> >------------------------------------------------------------------------------ >>> Start uncovering the many advantages of virtual appliances >>> and start using them to simplify application deployment and >>> accelerate your shift to cloud computing. >>> http://p.sf.net/sfu/novell-sfdev2dev >>> _______________________________________________ >>> W3af-develop mailing list >>> W3af-develop@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/w3af-develop >>> >> >> >> ------------------------------------------------------------------------------ >> Start uncovering the many advantages of virtual appliances >> and start using them to simplify application deployment and >> accelerate your shift to cloud computing. >> http://p.sf.net/sfu/novell-sfdev2dev >> _______________________________________________ >> W3af-develop mailing list >> W3af-develop@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/w3af-develop >> >> > > > >-- >Andrés Riancho >Founder, Bonsai - Information Security >http://www.bonsai-sec.com/ >http://w3af.sf.net/ > >------------------------------------------------------------------------------ >Start uncovering the many advantages of virtual appliances >and start using them to simplify application deployment and >accelerate your shift to cloud computing. >http://p.sf.net/sfu/novell-sfdev2dev >_______________________________________________ >W3af-develop mailing list >W3af-develop@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/w3af-develop > ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
