Patrick, It looks good to me, I like the wild card option being added. The only thing I would add is mentioning in the documentation the limitations of the whitelist/blacklist with regards to the other discovery plugins such as google/archive.org/etc. I think option number 2 is the way to go if we want to get around that. My guess though, is that anyone who wants to issue requests to the passive discovery plugins will be able to define their whitelist/blacklist on the webspider plugin and allow the other plugins to function correctly. Again, I think that should be added to the documentation as a side effect of the global whitelist/blacklist.
My .02 cents. Zach On Sat, Aug 1, 2009 at 3:01 PM, Patrick Hof <patrick...@web.de> wrote: > > Hi, > > it's awfully quiet on the list. Is everyone in Vegas? ;) > > I've attached a patch which adds black- and whitelist functionality to w3af, > to > be able to restrict the scanning. This is a merge of my own code and Zach > Jansen's code, which he posted to the list on July 7th and which I didn't see > before I started coding... If you'd be so kind to review and test this, I > would > be glad. The tests I did seemed to work. > > My code (hopefully) improves his by adding support for more than one regex and > also the ability to switch to wildcard mode, which is easier to type but less > powerful. The first improvement comes with a problem though: As Python > interprets commas as list item separators, you currently can't use commas in > your regex. Maybe someone has an idea on how to solve this elegantly. > > There's still the problem remaining that this patch will "break" some of > the passive discovery plugins as Zach alredy wrote, like the ones using > Archive.org or Google. Before I go and implement it, I wanted to discuss what > I > have come up with. I can think of two options at the moment: > > 1. Add a static list to the whitelisting code with all the URLs we do not want > to restrict. Not a good solution IMHO, as the users' decision gets overriden > without their knowledge. Maybe they really want to restrict requests to > Google. > > 2. Add an "override whitelist" config item to all plugins affected. Allow the > user to decide if this plugin should be allowed to bypass the whitelist. > This > would save the user the hassle to add a whitelisting to the config, as he > only > has to set a tick. > > Are there any other solutions I'm missing? > > Patrick > > -- > The Plague: You wanted to know who I am, Zero Cool? Well, let me explain > the New World Order. Governments and corporations need people > like you and me. We are Samurai... the Keyboard Cowboys... and > all those other people who have no idea what's going on are > the cattle... Moooo. > (Hackers) > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > W3af-develop mailing list > W3af-develop@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/w3af-develop > ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
