Re: [W3af-users] Authenticated spider issues and questions

[Andres Riancho]

Kukulkan,

The authentication plugins do not send the login / check URLs to
the core. So any URL you put in the configuration, or is a result of
requesting those URLs will not make it to other plugins / the crawler.

That was the original design and is working as expected. Might not
be ideal for cases (yours?)... we'll see!

When the user configures authentication plugins, those are run at
the beginning of the scan [0][1], before sending "almost any other
request" and before the crawling plugins. This means that you could
configure w3af like this:
* auth plugin logins the scanner
* the scanner will re-use cookies just like any browser (like
you mention above)
* crawl plugin will re-use cookies to follow the links you set
in the target. Remember that you can set the target to a comma
separated list of URLs, that might help.

Those steps will be run in that order, so the crawler should have
cookies when reaching the target.

The GUI is NOT maintained and I don't recommend using it. Use the
console or REST API.

w3af doesn't support javascript, so it won't be able to extract
"phpAccountSummary.php" from:

```

window.setTimeout("window.location.href = 'phpAccountSummary.php';", 0);

```

If you want me to help a little bit more, please do send me scan
logs with debugging information and HTTP requests (both files are
generated by text_file plugin)

[0] 
https://github.com/andresriancho/w3af/blob/39004228300e1eb38ae0cdb3946725e7a3adb8c8/w3af/core/controllers/core_helpers/strategy.py#L649
[1] 
https://github.com/andresriancho/w3af/blob/39004228300e1eb38ae0cdb3946725e7a3adb8c8/w3af/core/controllers/core_helpers/strategy.py#L111-L112



On Thu, Apr 26, 2018 at 7:31 AM, Volker Schmid  wrote:
> Hello Andres,
>
> I created a cookie file and tried again. Now it seems to use the cookie, but
> spider is still not successful. I can see that it spidered several pages but
> it does not follow the links inside. Looks like it does not even try to
> spider the page that was found in login page result like this:
>
> 
> window.setTimeout("window.location.href = 'phpAccountSummary.php';", 0);
> 
>
> It just inspects the few pages linked on the start and login page. But it
> does not spider the pages behind. I thought it would also use the page I set
> for login verification (phpAccountSummary.php). It opens it, even successful
> after login, but it does not spider the links inside there.
>
> Again, if I set the spider target directly to
> https://vsprovider2.de.mysystem.com/phpAccountSummary.php, the
> "Results"->"URLs" stays completely empty.
>
> I also have to restart w3af GUI each time I scanned because any further
> action leads to crashes, strange GUI behaviour (missing values in scan
> config fields) or missing logs and URL's in "Results" view occasionally. The
> GUI seems very buggy to me.
>  Is there some other, more stable version available? And is there a more
> sophisticated authentication/spider PlugIn available?
>
> Thanks,
>
> Kukulkan



-- 
Andrés Riancho
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
Twitter: @w3af
GPG: 0x93C344F3

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
W3af-users mailing list
W3af-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/w3af-users

Re: [W3af-users] Authenticated spider issues and questions

[Volker Schmid]

Hello Andres,

I created a cookie file and tried again. Now it seems to use the cookie, but 
spider is still not successful. I can see that it spidered several pages but it 
does not follow the links inside. Looks like it does not even try to spider the 
page that was found in login page result like this:


window.setTimeout("window.location.href = 'phpAccountSummary.php';", 0);


It just inspects the few pages linked on the start and login page. But it does 
not spider the pages behind. I thought it would also use the page I set for 
login verification (phpAccountSummary.php). It opens it, even successful after 
login, but it does not spider the links inside there.

Again, if I set the spider target directly to 
https://vsprovider2.de.mysystem.com/phpAccountSummary.php, the 
"Results"->"URLs" stays completely empty.

I also have to restart w3af GUI each time I scanned because any further action leads to 
crashes, strange GUI behaviour (missing values in scan config fields) or missing logs and 
URL's in "Results" view occasionally. The GUI seems very buggy to me.
 
Is there some other, more stable version available? And is there a more sophisticated authentication/spider PlugIn available?


Thanks,

Kukulkan

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
W3af-users mailing list
W3af-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/w3af-users

Re: [W3af-users] Authenticated spider issues and questions

[Volker Schmid]

Hi,

thanks for the quick response.


Maybe the web_spider is following the logout link, which is
invalidating the session?

You should ignore logout urls when doing auth scans


I already did by using ignore_regex: "(phpLogout\.php|phpNewPassword\.php)"

By this I hope he does not call any URL with these two scripts inside. Both 
would be bad for the scan.


Yeah, that could be because of the javascript redirect. Maybe try to
set phpAccontSummary.php in the w3af target configuration?


I changed the target to be phpAccountSummary.php. But now it logs in successfully and 
unsuccessfully a few times. It does not spider any other URL's now. It does not find a 
single URL but in "Results", and if I look for this request, I can see that the 
successful requests returned several links but they are not followed at all.


No, lets try with the things I recommended above, if that doesn't work
we'll try giving w3af a cookie via config/http/cookies


Hm. The cookie with the Session-ID is returned by the first call to phpLogin.php. I 
assumed that w3af is using the cookies like a webbrowser does (eg after receiving one, 
always send the content with every further request). But due to the requests in the 
"Results", it does not send the cookie it received before with the next 
requests. It simply does not respect the session cookie. Interestingly, another cookie is 
always used (but there the content is static and no session ID).

Any other idea?

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
W3af-users mailing list
W3af-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/w3af-users

Re: [W3af-users] Authenticated spider issues and questions

[Andres Riancho]

Kukulkan,

Answers inline,

On Wed, Apr 25, 2018 at 4:33 AM, Volker Schmid  wrote:
> Hi,
>
> I'm new to w3af and start to get deeper into authentication. I use only two
> PlugIns: crawl->web_spider and auth->detailed. The current site is using a
> form in phpLogin.php. This is doing a JS redirect so I use
> phpAccontSummary.php to verify if user was logged in successfully (searching
> there for "Log out").
>
> This is the config for auth-detailed:
> [auth.detailed]
> username = pente...@mysystem.com
> password = EGjv4gmj
> username_field = txtUsername
> password_field = txtPassword
> auth_url = https://vsprovider2.de.mysystem.com/phpLogin.php?action=login
> check_url = https://vsprovider2.de.mysystem.com/phpAccountSummary.php
> check_string = Log out
> data_format = %u=%U&%p=%P
> follow_redirects = False
> method = POST
> url_encode_params = True
>
>
> Due to the website logs, login for user "Pentest Pentest" (ID 3) was
> successful several times:
>
> 2018-04-25 09:12:25 USER_LOGIN_SUCCESS  Pentest Pentest (3)
> 2018-04-25 09:12:20 USER_LOGIN_SUCCESS  Pentest Pentest (3)
> 2018-04-25 09:12:15 USER_LOGIN_SUCCESS  Pentest Pentest (3)
>
>
> In the GUI log I get this:
>
> [Mi 25 Apr 2018 09:12:25 CEST] Can't login into web application as
> pente...@mysystem.com/EGjv4gmj
.
>
> In the console output (using GUI) of w3af I can find such entries:
>
> GET https://vsprovider2.de.mysystem.com/phpAccountSummary.php returned HTTP
> code "200" (id=19,from_cache=0,grep=0,rtt=0.01,did=None)
> User "pente...@mysystem.com" is NOT logged into the application
> POST https://vsprovider2.de.mysystem.com/phpLogin.php?action=login with
> data: "txtUsername=pente...@mysystem.com=EGjv4gmj" returned HTTP
> code "200" (id=20,from_cache=0,grep=1,rtt=0.06,did=None)
> GET https://vsprovider2.de.mysystem.com/phpAccountSummary.php returned HTTP
> code "200" (id=21,from_cache=0,grep=0,rtt=0.03,did=None)
> User "pente...@mysystem.com" is currently logged into the application
> Login success for pente...@mysystem.com/EGjv4gmj
> detailed._login() took 0.11s to run
>
> (...many other spider entries...)
>
> GET https://vsprovider2.de.mysystem.com/phpAccountSummary.php returned HTTP
> code "200" (id=74,from_cache=0,grep=0,rtt=0.04,did=None)
> User "pente...@mysystem.com" is NOT logged into the application

Maybe the web_spider is following the logout link, which is
invalidating the session?

You should ignore logout urls when doing auth scans

> (...a few other spider entries...)
>
> ET https://vsprovider2.de.mysystem.com/phpAccountSummary.php returned HTTP
> code "200" (id=78,from_cache=0,grep=0,rtt=0.04,did=None)
> User "pente...@mysystem.com" is currently logged into the application
> Login success for pente...@mysystem.com/EGjv4gmj
> detailed._login() took 0.18s to run
>
> (...many other spider entries...)
>
> GET https://vsprovider2.de.mysystem.com/phpAccountSummary.php returned HTTP
> code "200" (id=111,from_cache=0,grep=0,rtt=0.01,did=None)
> User "pente...@mysystem.com" is NOT logged into the application
> web_spider.discover(https://vsprovider2.de.mysystem.com/phpCreateRegifyLocal.php)
> web_spider is testing
> "https://vsprovider2.de.mysystem.com/phpCreateRegifyLocal.php;
> [web_spider] Crawling
> "https://vsprovider2.de.mysystem.com/phpCreateRegifyLocal.php;
> GET https://vsprovider2.de.mysystem.com/phpCreateRegifyLocal.php returned
> HTTP code "302" (id=112,from_cache=0,grep=1,rtt=0.01,did=None)
> web_spider.discover(uri="https://vsprovider2.de.mysystem.com/phpCreateRegifyLocal.php;)
> took 0.02s to run
> POST https://vsprovider2.de.mysystem.com/phpLogin.php?action=login with
> data: "txtUsername=pente...@mysystem.com=EGjv4gmj" returned HTTP
> code "200" (id=113,from_cache=0,grep=1,rtt=0.07,did=None)
> GET https://vsprovider2.de.mysystem.com/phpAccountSummary.php returned HTTP
> code "200" (id=114,from_cache=0,grep=0,rtt=0.01,did=None)
> User "pente...@mysystem.com" is NOT logged into the application
> Can't login into web application as pente...@mysystem.com/EGjv4gmj
>
> So this are very mixed results (sometimes success sometimes not) and I do
> not know why it sometimes reports successful login and sometimes it does
> not?
>
> Due to the request navigator and the results to phpLogin.php there, login
> was always successful if w3af sent the correct login data by POST. I can see
> that phpAccontSummary.php delivered positive results sometimes.
>
> Also, even if it was successful, it seems it does not spider the links found
> in phpAccontSummary.php. All the new links inside there are not listed in
> the URL's found.

Yeah, that could be because of the javascript redirect. Maybe try to
set phpAccontSummary.php in the w3af target configuration?

> I can see that w3af does not send the session cookie received during the
> first phpLogin.php all the time. It seems to forget sometimes. If not set,
> the webpage creates a new sessionid and returns it. So the logged in session
> is