Laurent,
On Wed, Jan 4, 2012 at 7:34 AM, <laurent.gu...@algosecure.fr> wrote:
> Hi,
>
> Thank you taking time to read my verbose mail ;)
A pleasure when the email has interesting content :)
>
>> Laurent,
>>
>> Please read comments inline,
>>
>> On Tue, Jan 3, 2012 at 7:51 AM, <laurent.gu...@algosecure.fr> wrote:
>> > Hi,
>> >
>> > I'm a new w3af user (using and testing it for 1 month) and I'd like to
> congratulate you for the work you already achieved :
>> > * different phases (discovery, audit, exploit...) separated (clever)
>> > * a plugin system
>> > * possibility to use profiles and scripts
>> > * spiderman !
>> > * written in python ^^
>>
>> :P
>>
>> > When scanning a sample RESTful application vulnerable to XSS, I noticed
>> > that
> w3af (svn) does not handle nor find vulnerabilities (XSS, SQLi) on RESTful
> URLs.
>> > So I started to look at the code to see how could this be achieved
>> > (discovery
> phase, audit phase...), but when looking at the mailing I saw that you were
> already working on it ;)
>> >
>> > Your idea to use fuzzing on url parts to do some "brute force" is very
> clever, but I spotted a little problem inside this "URL parts fuzzing" code
> (svn)
> that make it fail in my tests : it always return double-encoded mutants, that
> make :
>> > 1. the detection of allowed special chars in XSS audit plugin fail (receive
> single-encoded special chars instead of non-encoded special chars), causing
> reflected XSS detection be aborted
>> > 2. the detection of reflected/stored XSS fail because payload that is
> returned in result page is single-encoded and can't be executed by the browser
>> >
>> > I made a quick and dirty patch that make the _createUrlPartsMutants()
>> > method
> (fuzzer.py) return 2 versions of each mutant : one single-encoded, and one
> double-encoded :
>>
>> That's smart, I like the idea, not all web applications handle
>> URLs the same.
>
>
> By the way, I realize that when activating "URL parts fuzzing" functionnality,
> the createMutants() method does not return the original URL, only mutants.
> This implies that each pentest must be launched 2 times : 1 time without URL
> parts fuzzing, and 1 time with, does this behaviour is intentionnal ?
I'm not sure I understand what you're saying. Maybe Taras can help
me out since I didn't write that piece of the code!
>
>> > core/data/fuzzer/fuzzer.py
>> > 549c549
>> > < m.setDoubleEncoding(True)
>> > ---
>> >> m.setDoubleEncoding(False)
>> > 550a551,562
>> >>
>> >> # Same URLs but with different types of encoding!
>> >> m2 = m.copy()
>> >> m3 = m.copy()
>> >> # for mod_rewrite
>> >> m2.setSafeEncodeChars('/')
>> >> if m2.getURL() != m.getURL():
>> >> res.append(m2)
>> >> # double encoding
>> >> m3.setDoubleEncoding(True)
>> >> res.append(m3)
>> >
>> > This patch made the URL parts fuzzing code work fine for me, and now w3af
> detect XSS vulnerabilities on my RESTful webapp ^^
>>
>> And it will also work for so many other web applications.
>>
>> After applying the patch (manually) this is what I get:
>>
>> """
>> m.setDoubleEncoding(False)
>>
>> # Same URLs but with different types of encoding!
>> m2 = m.copy()
>> m3 = m.copy()
>> # for mod_rewrite
>> m2.setSafeEncodeChars('/')
>> if m2.getURL() != m.getURL():
>> res.append(m2)
>> # double encoding
>> m3.setDoubleEncoding(True)
>> res.append(m3)
>>
>> res.append(m)
>> """
>>
>> Which is a bit confusing for me:
>>
>> * if m2.getURL() != m.getURL(): Why do we need this?
>> * res.append(m) , do we really need that?
>>
>> If you work on this patch a little bit and make some changes,
>> please send the ".patch" file in the next email so that I can "patch
>> -p0 < laurent.patch" and apply it automagically.
>
>
> Sorry for that, I attached the proper cleaned patch. In fact :
>
> * m is single encoded version, so we need it
> * m3 is double-encoded version, so we need it
> * m2 : I re-use this code from _createFileNameMutants() method, this is
> perhaps
> not really useful in this case?
Taras, since you've written that piece of code, would you mind
reviewing / testing / commiting the change to SVN?
>> > That patch makes also w3af detect now 62% of sample XSS vulnerabilities on
> the wavsep 1.3 vulnrerable framework (http://code.google.com/p/wavsep/),
> instead
> of initially 27% before (remark : all the other vulnerabilities could be
> detected
>> > by adding more XSS injection patterns in w3af ;)
>>
>> Interesting, which XSS injection patterns?
>
>
> I've added for example this new pattern (xss.py), to treat case when
> expression
> is embedded in a javascript code bloc, as a variable value:
>
> # if we are inside a javascript bloc
> xss_tests.append(('''RANDOMIZE";alert('RANDOMIZE');RANDOMIZE="''',
> [browsers.ALL, ]))
Sorry but I think that the pattern will trigger false positives,
we can't add it to the xss.py file. The problem is that it works
perfectly for the case you're testing (user input echoed inside
<script> tags) but it will trigger a XSS vulnerability in other cases
(user input echoed inside tag text <a>text</a>). What we really need
to do is to verify WHERE IN THE HTML we're seeing our echoed response,
and then send the payloads that make sense for that situation.
> By the way, I envisage to make a patch for putting XSS injection patterns
> into an
> external config file rather than hardcode them in xss.py, this would make
> patterns managing easier.
Not really, I like the idea of keeping the payloads in the same
file, so you have "all you need to know" in the same place. Also,
having the files in a python script allows us to easily add "tags" to
it such as "send only of I'm inside script tags". If you want to write
a very useful patch, I would recommend you write the previously
defined idea.
> Such mechanic could also be used in the other parts of the application, what
> is
> your opinion ?
>
> And I'd like to add a special keyword ("ALL", or -1 if integer matters) for
> XSS
> audit plugin option "_number_of_stored_xss_checks", for selecting
> automatically
> all patterns (because if you add a new pattern you must edit all your scripts
> to
> increment the value fo this parameter).
Oh, that's correct, please fix if you've got time :)
>
>> > I don't yet understand the whole application, but I wonder if management of
> single and double-encoding could be done in another way, to make it more
> generic
> for all mutant generation methods and for all audit plugins : perhaps not by
>> > returning 2 versions of each mutant URL, but by returning only 1 mutant URL
> for each test, and test it firstly single-encoded then double-encoded if
> single-encoded fails ?
>>
>> Are you only talking about XSS here or all web vulnerabilities?
>> Any way, in most cases (99,9%) when w3af sends something to the remote
>> web application it does NOT get what it expects (an error). So the
>> idea of sending first single-encoded and then double-encoded if the
>> first test fails is almost the same as sending both; because you won't
>> be sending less requests (except from the case where the vulnerability
>> was actually found).
>
>
> I talked about all vulnerabilities, because encoding matters everywhere I
> think.
> So the single/double encoded sending could perhaps be managed by the sending
> code, not by the mutant creation code ?
I would do this in the fuzzer.py , by creating new mutants that in
some cases are double-encoded and some other cases are single-encoded
(based on user configuration, default would be single encoded only).
> You're right, it's better to send both variations, single and double-encoded.
>
>> > Another thing : when using URL parts fuzzing code to detect vulnerabilities
> in RESTful URLs, w3af returns several identical vulnerabilities for the same
> "true" URL (what is logic, each fuzzed URL that works report 1 separate
> vulnerability
>> > because it's a completely different URL). I didn't looked at it at the
> moment, but I think it could perhaps be interresting to do some aggregation of
> found vulnerabilities to make w3af return only 1 vulnerability for each "true"
> URL, specifying all
>> > variations that have worked (fuzzed URLs) ?
>>
>> That's a good idea. We're doing that in the _has_no_bug() method,
>> but it doesn't work with fuzzed URLs because that method compares the
>> URL to avoid reporting two very similar vulnerabilities. Maybe that
>> function requires a little bit of attention to help us in this case!
>
>
> Perhaps this can be done by keeping track of the original URL in each fuzzed
> mutant ? so agregation would be immediate at the end of the scan ?
Not a bad idea... but at the same time, we'll be finding applications like:
* http://host.tld/food/hot-dog/2/
* http://host.tld/food/burger/3/
* http://host.tld/food/sandwich/2/
And we'll create these mutants:
* http://host.tld/food/<script>...</script>/2/
* http://host.tld/food/<script>...</script>/3/
* http://host.tld/food/<script>...</script>/2/
If we compare the resulting URL for reporting unique
vulnerabilities, we'll get two different ones (
http://host.tld/food/<script>...</script>/2/ and
http://host.tld/food/<script>...</script>/3/ ). If we compare the
original URLs we're still going to get duplicates, in this case three
(hot-dog, burger, sandwich). What might help us is keeping track of
something like "http://host.tld/food/*/2/"; that means "injected into
the second path, the one after /food/", which would make it quite
unique (if we forget about the last path specification).
At this moment that's something with low priority compared with
the xss.py improvements, so if we do something, it should be the
xss.py stuff
Regards,
>
>> > (excuse me for my poor english, french people speaking ^^)
>>
>> Actually, your English was pretty good !
>>
>> Thanks for your very interesting comments :)
>
>
> Thank you ;)
>
>
>> > Regards,
>> >
>> > Laurent
>> >
>>
>> --
>> Andrés Riancho
>> Director of Web Security at Rapid7 LLC
>> Founder at Bonsai Information Security
>> Project Leader at w3af
>>
>
--
Andrés Riancho
Director of Web Security at Rapid7 LLC
Founder at Bonsai Information Security
Project Leader at w3af
------------------------------------------------------------------------------
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual
desktops for less than the cost of PCs and save 60% on VDI infrastructure
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
_______________________________________________
W3af-develop mailing list
W3af-develop@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/w3af-develop
