Sheriff,
Sorry for the late reply, I've been really busy with work. Please
read inline,
On Tue, Apr 27, 2010 at 10:43 AM, Sheriff Texer <sheriff.te...@gmx.de> wrote:
> Hey,
>
> I'm using w3af for 2 weeks now, so I'm still at the beginning right now.
> There was one big problem I got nearly everytime when I wanted to scan a
> private app of mine. Even if I wanted to do only spidering (webSpider,
> spiderMan) this error occurs:
>
> ----------
> **IMPORTANT** The following error was detected by w3af and couldn't be
> resolved: The xUrllib found too much consecutive errors. The remote
> webserver doesn't seem to be reachable anymore; please verify
> manually
> ----------
>
> After this message occured, w3af stopped scanning (or spidering) without any
> result. After so many reading and searching for the answer, that led to
> nowhere, I decided to search a little bit in the code of
> /core/data/url/xUrllib.py.
>
> Here I found something intresting. The source of the error-message was
>
> if self._consecutiveErrorCount >= 10:
> msg = 'The xUrllib found too much consecutive errors. The remote webserver
> doesn\'t'
> msg += ' seem to be reachable anymore; please verify manually.'
> raise w3afMustStopException( msg )
>
> This code is found in the funtion _incrementGlobalErrorCount( self ), which
> is called in the urllib2.URLError-exception. The problem is, that every
> 400-response raise the urllib2.URLError-
> exception and increment the _consecutiveErrorCount. After the 10th
> 400-response w3af will raise the w3afMustStopException.
hmmm, I think you're mistaken. Not by far I must say... but
mistaken at the end. Maybe the source code comments are misleading?
# I get to this section of the code if a 400 error is returned
# also possible when a proxy is configured and not available
# also possible when auth credentials are wrong for the URI
if hasattr(e, 'reason'):
self._incrementGlobalErrorCount()
The thing is the I get to the point where the comments are, when I
find a 4xx error, the proxy is unreachable, or the server is
unreachable. Once there, I ask if the exception "e" has a reason
attribute. If the reason attribute is set, it means that I got there
because of the proxy is unreachable, or the server is unreachable. The
rest of the cases are handled here:
elif hasattr(e, 'code'):
# We usually get here when the response has codes 404,
403, 401, etc...
msg = req.get_method() + ' ' + original_url +'
returned HTTP code "'
msg += str(e.code) + '" - id: ' + str(e.id)
Just to be sure... have you tested if you get to line 440 of
xUrllib.py when a 404 occurs?
> I think this happens because of the function ignore_errors( self, yes_no ) or
> better, because of the attribute _ignore_errors_conf. This attribute was set
> to 'False' by default.
And it should be the default. This parameter is there for
"advanced users" that don't care about errors, just want the framework
to go on scanning, and take the responsability of ignoring errors.
Common users should have those defaults.
> self._ignore_errors_conf = False
>
> The problem is that neither this attribute is set to be 'True' nor the
> function ignore_errors( self, yes_no ) is called on any point in the code!
> But this will be necessary to ignore the
> 400-response like its commented in the code:
Here is where the "self._ignore_errors_conf" attribute is used:
def _incrementGlobalErrorCount( self ):
'''
Increment the error count, and if we got a lot of failures...
raise a "afMustStopException"
'''
if self._ignore_errors_conf:
return
And the function is a "setter" that's called from
xUrllibSettings.py (if I don't recall wrongly).
> def ignore_errors( self, yes_no ):
> '''
> Let the library know if errors should be ignored or not. Basically,
> ignore all calls to "_incrementGlobalErrorCount" and don't raise the
> w3afMustStopException.
>
> �...@parameter yes_no: True to ignore errors.
> '''
> self._ignore_errors_conf = yes_no
>
> Since I set _ignore_errors_conf manually to 'True' the error-message occur no
> more. Will this "fix" the bug or am I completly wrong?
Actually, this hides the errors. They still occur and the root
cause should be fixed.
> Many thanks for any help...
Thank YOU very much for taking the time to read the source code,
and try to find the error.
Regards,
> Yeehaw
>
> --
> GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
> Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
>
> ------------------------------------------------------------------------------
> _______________________________________________
> W3af-develop mailing list
> W3af-develop@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/w3af-develop
>
--
Andrés Riancho
Founder, Bonsai - Information Security
http://www.bonsai-sec.com/
http://w3af.sf.net/
------------------------------------------------------------------------------
_______________________________________________
W3af-develop mailing list
W3af-develop@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/w3af-develop
