Here is the patch by Kevin,
---------- Forwarded message ---------- From: Muffys Wump <muff...@hotmail.com> Date: Fri, Dec 5, 2008 at 2:04 PM Subject: RE: [W3af-develop] WebCrawler Question/Enhancement To: andres.rian...@gmail.com Hi Andres, I'm sending you the changes I've made. Let me know what you think about them. I've tested them today and it seems they work with both kinds of application. My name's Kevin by the way ;) Cheers, Kevin > Date: Fri, 28 Nov 2008 12:15:17 -0200 > From: andres.rian...@gmail.com > To: muff...@hotmail.com > Subject: Re: [W3af-develop] WebCrawler Question/Enhancement > > Muffys, > > On Fri, Nov 28, 2008 at 11:21 AM, Muffys Wump wrote: >> >> >> Hi Andres, >> >> I did some hacking and changed parts of the urlParser.py in order to support >> these session id's we've talked about. I send you the diff from the current >> beta 7 >> release and the complete file. > > I just read the patch, which looks good. Some comments/questions about it: > > - Could you please test it with your application, and make it work in > such a way that you don't find a loop in the web spider? Maybe this > also needs changes in the spider? > > - Once you are done with the urlParser, and you are confident that it > works ok in your environment, could you test it in a "normal" > environment? After that, I would test it and finally add the patch to > the trunk. > > - Could you please change the name of the function from *sessionId* to > *parameter* or some other, more generic, name? I think that the > "parameter" can be user for more things, maybe... login.jsp;foo=bar > ... and there you don't have a sessionid. > >> Having these session id's in the URL causes some trouble ;) I think I've >> encountered >> an endless loop. The Spider got redirected to the login page and on every >> visit the >> application generated a new session id for it. I'm not sure but I think w3af >> took these URL's >> as new because of the changing session id. >> >> You might be able to tell me if I'm wrong with my guess. >> >> Cheers, >> Kevin >> >> >>> From: muff...@hotmail.com >>> To: andres.rian...@gmail.com >>> Date: Thu, 27 Nov 2008 16:01:58 +0000 >>> CC: w3af-develop@lists.sourceforge.net >>> Subject: Re: [W3af-develop] WebCrawler Question/Enhancement >>> >>> >>> Hi Andres, >>> >>>> Date: Thu, 27 Nov 2008 12:38:32 -0200 >>>> From: andres.rian...@gmail.com >>>> To: muff...@hotmail.com >>>> Subject: Re: [W3af-develop] WebCrawler Question/Enhancement >>>> CC: w3af-develop@lists.sourceforge.net >>>> >>>> Muffys, >>>> >>>> On Thu, Nov 27, 2008 at 12:16 PM, Muffys Wump wrote: >>>>> >>>>> Hello, >>>>> >>>>> I'm trying to crawl a web application using the w3af WebSpider plugin. >>>>> The application uses a simple login form to authenticate users. After >>>>> successful >>>>> authentication the session id isn't stored inside a cookie, >>>>> the id is instead part of the URL like this: >>>>> >>>>> https://foobar.../listinbox_en.jsp;jsessionid=2fa5844582d42560e13c764c8a9e686a?login=myem...@address.com >>>>> https://foobar.../listcounterpart_en.jsp;jsessionid=2fa5844582d42560e13c764c8a9e686a?login=myem...@address.com >>>>> >>>>> In order to scan this application I set the target to the following >>>>> URL/Host >>>>> (using a valid session of course): >>>>> https://foobar.../listinbox_en.jsp;jsessionid=2fa5844582d42560e13c764c8a9e686a?login=myem...@address.com >>>>> >>>>> By looking at the URLs the WebSpider came up with, I see that the plugin >>>>> never >>>>> gets past the login form. I'm wondering if I'm doing something wrong or if >>>>> the WebSpider isn't able to use the parameters inside the target URL >>>>> correctly. >>>>> If that's the case, is there a chance that this could be easily fixed? >>>>> I'm also a developer and >>>>> would be able to fix this with a little help (never done python before). >>>>> >>>>> Any help or suggestions appreciated. >>>> >>>> Interesting problem... kind of hard to fix... look at this tests I've >>>> just performed: >>>> >>>> cyb...@toshi-noteb:~/w3af/trunk$ python >>>> Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) >>>> [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 >>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>> import core.data.parsers.urlParser as urlParser >>>>>>> qs = urlParser.getQueryString('http://localhost/abc?def=1;login=123') >>>>>>> qs >>>> {'login': '123', 'def': '1'} >>>>>>> str(qs) >>>> 'login=123&def=1' >>>>>>> qs = >>>>>>> urlParser.getQueryString('https://foobar/listinbox_en.jsp;jsessionid=2fa5844582d42560e13c764c8a9e686a?login=myem...@address.com') >>>>>>> qs >>>> {'login': 'myem...@address.com'} >>>>>>> str(qs) >>>> 'login=myemail%40address.com' >>>>>>> >>>> >>>> w3af is going to parse the login stuff correctly in the simplest case >>>> (first test I made) and incorrectly for the second test. I think that >>>> the problem is in the "getQueryString" function, and the way it parses >>>> the ";". I don't know if ";" is a valid separator just like "?". >>>> What's the RFC for URLs ? >>>> >>> >>> I did some research and using ";" in URLs is allowed. It's specified in the >>> Servlet 2.4 Documentation (SRV.7.13) >>> >>> URL rewriting is the lowest common denominator of session tracking. When a >>> client will not accept a cookie, URL rewriting may be used by the server as >>> the basis >>> for session tracking. URL rewriting involves adding data, a session ID, to >>> the URL >>> path that is interpreted by the container to associate the request with a >>> session. >>> The session ID must be encoded as a path parameter in the URL string. The >>> name of the parameter must be jsessionid. Here is an example of a URL >>> containing encoded path information: >>> >>> http://www.myserver.com/catalog/index.html;jsessionid=1234 >>> >>>> Let's suppose that you can make the parser work as you expect... the >>>> next step would be for the "str(qs)" to work also the way you expect, >>>> and separate name/value pairs with "?" instead from the expected "&". >>>> >>>> My opinion: This case is rare but has to be included in some way into >>>> the framework. If you can code a patch... I'll apply it to the trunk >>>> after review. >>>> >>> >>> I see what I can do. >>> >>> Thanks for your help! >>> Cheers, >>> Kevin >>> >>>> Cheers, >>>> >>>>> Thanks >>>>> >>>>> >>>>> _________________________________________________________________ >>>>> Explore the seven wonders of the world >>>>> http://search.msn.com/results.aspx?q=7+wonders+world&mkt=en-US&form=QBRE >>>>> ------------------------------------------------------------------------- >>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's >>>>> challenge >>>>> Build the coolest Linux based applications with Moblin SDK & win great >>>>> prizes >>>>> Grand prize is a trip for two to an Open Source event anywhere in the >>>>> world >>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>>>> _______________________________________________ >>>>> W3af-develop mailing list >>>>> W3af-develop@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/w3af-develop >>>>> >>>> >>>> >>>> >>>> -- >>>> Andres Riancho >>>> http://w3af.sourceforge.net/ >>>> Web Application Attack and Audit Framework >>> >>> _________________________________________________________________ >>> Connect to the next generation of MSN Messenger >>> http://imagine-msn.com/messenger/launch80/default.aspx?locale=en-us&source=wlmailtagline >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>> Build the coolest Linux based applications with Moblin SDK & win great >>> prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> W3af-develop mailing list >>> W3af-develop@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/w3af-develop >> >> _________________________________________________________________ >> Connect to the next generation of MSN Messenger >> http://imagine-msn.com/messenger/launch80/default.aspx?locale=en-us&source=wlmailtagline > > > > -- > Andres Riancho > http://w3af.sourceforge.net/ > Web Application Attack and Audit Framework _________________________________________________________________ Discover the new Windows Vista http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE -- Andres Riancho http://w3af.sourceforge.net/ Web Application Attack and Audit Framework
webSpider.diff
Description: Binary data
w3afCore.diff
Description: Binary data
w3af_console_script.w3af
Description: Binary data
urlParser.diff
Description: Binary data
------------------------------------------------------------------------------ SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
_______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
