Sergio, On Tue, Aug 5, 2014 at 5:42 PM, Sergio A <foobarm...@gmail.com> wrote: > Hi guys, > > Yesterday, while "playing" with w3af I saw something (detailed below) > with the "allowed methods" plugin related to checking if the the http > CONNECT method is available in a server or not and I'd like to know if > you think it could be a bug or not. > > In the case you think it is a bug, I'd be very happy if I can fix it > and, if possible, would like some advice or guidelines on what next > steps you think I should do for implement the fix. > The problem is that I'm not a python guru like you at all, currently I > just know some python for my day to day stuff. > And if it is not a bug, I'd like as well to contribute. So guidelines, > advice, etc on how to do it is welcome :) > > Here is what I did: > > 1. git pull the last w3af version. > > 2. setup an apache server as a forward proxy > > Setup an apache web server that has enabled > the CONNECT method for proxying clients (forward proxy not reverse > proxy) so them can navigate setting the apache box as their proxy (the > only client I´m testing now is on the same host hence Allow from > 127.0.0.1). > This is the relevant apache config I have (apart of the enabled > mod_proxy modules, etc): > > <VirtualHost *:80> > ProxyRequests On > ProxyVia On > AllowCONNECT 80 443 563 > </VirtualHost> > > <Proxy *> > Order deny,allow > Deny from all > Allow from 127.0.0.1 > </Proxy> > > 3. Manually check CONNECT method is working > > (look at the format on the CONNECT line): > > bla@ubuntu:~$ telnet localhost 80 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > CONNECT google.com:80 HTTP/1.1 > HOST: google.com > > HTTP/1.0 200 Connection Established <--- it worked > Proxy-agent: Apache/2.4.7 (Ubuntu) > > blablablabla > > HTTP/1.0 400 Bad Request <---- this is normal because blablabla is a bad > request > Content-Type: text/html; charset=UTF-8 > Content-Length: 1419 > Date: Mon, 04 Aug 2014 18:13:32 GMT > Server: GFE/2.0 > <snip>
Ok, so far so good :) I understood the setup. > 4. Setup a w3af profile for checking CONNECT method > > I setup a profile enabling only "allowed methods" plugin > and this is what I see as the output of that plugin: > > """ The URL "http://localhost/"; has the following enabled HTTP methods: > CONNECT, GET, GET, HEAD, HEAD, OPTIONS, OPTIONS, POST, POST. This > information was found in the requests with ids 32, 39, 47, 52, 55 and > 71. """ Strange to see that there are duplicated methods! GET/GET, HEAD/HEAD, etc. ugly at least. > Note that, apart of all the methods but CONNECT been duplicated, what > I see when going into request/response navigator for the CONNECT > request (which is the one with id 55) is (look on the CONNECT line format): > > Request: > CONNECT http://localhost/ HTTP/1.1 > Host: localhost > Accept-encoding: gzip, deflate > Accept: */* > User-agent: w3af.org > > Response: > HTTP/1.1 400 Bad Request > date: Mon, 04 Aug 2014 16:57:55 GMT > content-length: 300 > content-type: text/html; charset=iso-8859-1 > connection: close > server: Apache/2.4.7 (Ubuntu) > > But, if I run again the same w3af profile and look with wireshark what > I see "on the wire" (look here as well in the CONNECT line) is: > > Request: > CONNECT / HTTP/1.1 > Host: localhost > Accept-encoding: gzip, deflate > Accept: */* > User-agent: w3af.org Ah, yes, that's a "bug" in the representation of the HTTP requests. It is "known" (and ugly) that w3af sends: CONNECT / HTTP/1.1 Host: localhost Accept-encoding: gzip, deflate Accept: */* User-agent: w3af.org And this is recorded in the log/shown in the GUI: CONNECT http://localhost/ HTTP/1.1 Host: localhost Accept-encoding: gzip, deflate Accept: */* User-agent: w3af.org This was done to avoid adding host/port/protocol fields in the GUI and text files. Does that make sense? Not sure. But now we just show: CONNECT http://localhost/ HTTP/1.1 And all the information is there. In an enhanced version we could have a different GUI design with more widgets showing: Protocol: http Port: 80 Host: localhost Request: CONNECT / HTTP/1.1 That would be the ideal case. For your tests with the CONNECT method this is specially confusing. > Response: > HTTP/1.1 400 Bad Request > Date: Mon, 04 Aug 2014 17:06:57 GMT > Server: Apache/2.4.7 (Ubuntu) > Content-Length: 300 > Connection: close > Content-Type: text/html; charset=iso-8859-1 > > So in summary: > > (A) Manual request (which works): > CONNECT google.com:80 HTTP/1.1 > > (B) w3af Request/Response navigator reported request: > CONNECT http://localhost/ HTTP/1.1 > > (C) w3af on the wire request: > CONNECT / HTTP/1.1 > > I think that according to the RFC it looks like a valid request should > have just a hostname, a colon and a port, like (A) and not like (B) > neither (C): > http://tools.ietf.org/html/rfc7231#section-4.3.6 > > Do you think this could be a bug or issue on how w3af generates a > CONNECT request ? * There is room for improvement in the way we show the request in the GUI and store it in the log files. * In the allowed_methods plugin [0] we don't treat CONNECT in any special way (it has a different format from the rest of the methods but the plugin doesn't "know" about that) [0] https://github.com/andresriancho/w3af/blob/master/w3af/plugins/infrastructure/allowed_methods.py > Also, how about the duplicates in the plugin output (I mean method > names appearing here twice on all but CONNECT): > "The URL "http://localhost/"; has the following enabled HTTP methods: > CONNECT, GET, GET, HEAD, HEAD, OPTIONS, OPTIONS, POST, POST. This > information was found in the requests with ids 32, 39, 47, 52, 55 and > 71." That's a bug > 5. Digging a bit > > I went through the code trying to read it, and I think the connect > request is generated in: > w3af.core.data.url.extended_urllib.AnyMethod The request is sent here [1] and then it passed through the AnyMethod (nice find there!). [1] https://github.com/andresriancho/w3af/blob/master/w3af/plugins/infrastructure/allowed_methods.py#L174 > But I'm not sure on where you think it could be a good place to fix it > and how (of course in the case you thinking there´s a bug). The best place to fix this might be the allowed_methods.py plugin + extended_urllib.py, but it might be *really* difficult to fix since we don't have support for sending "raw" text, and: CONNECT google.com:80 HTTP/1.1 HOST: google.com Might be "difficult" to send the "google.com:80". My advise is that you should experiment with extended_urllib.py and wireshark to see if you can make it send what you need, and then (if you got it) write a new method in allowed_methods.py. This can help get you started with your tests: pedro@laptop:~/pch/w3af$ python Python 2.7.3 (default, Feb 27 2014, 19:58:35) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from w3af.core.data.url.extended_urllib import ExtendedUrllib >>> from w3af.core.data.parsers.url import URL >>> from w3af.core.data.dc.headers import Headers >>> xu = ExtendedUrllib() >>> xu.CONNECT(URL('http://google.com:80/'), headers=Headers([('Host', >>> 'www.google.com')])) <HTTPResponse | 400 | http://google.com/ | id:1> >>> Regards, > Regards > > ------------------------------------------------------------------------------ > Infragistics Professional > Build stunning WinForms apps today! > Reboot your WinForms applications with our WinForms controls. > Build a bridge from your legacy apps to the future. > http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk > _______________________________________________ > W3af-develop mailing list > W3af-develop@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/w3af-develop -- Andrés Riancho Project Leader at w3af - http://w3af.org/ Web Application Attack and Audit Framework Twitter: @w3af GPG: 0x93C344F3 ------------------------------------------------------------------------------ Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk _______________________________________________ W3af-develop mailing list W3af-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-develop
