Ali, You were right, there was a bug in the way w3af parsed the HTML documents! This is the commit that fixes it [0], for now only available in the develop branch, but I'll try to release a new w3af version for Kali in a couple of weeks and it should get there. Thanks for your bug report.
[0] https://github.com/andresriancho/w3af/commit/936f97c11c0cbdae2e9c0e768ee99646b441c3d1 On Sun, May 25, 2014 at 2:12 PM, Ali Khalfan <ali.khal...@gmail.com> wrote: > Hi Andres, > > The page is really not that hard to read...a simple grep on href would > show all the links. > > Here is what the page looks like (more or less)--only links in the > <link> tags are being read: > > > > > > > > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> > > > > <html> > > > > <head> > > <title> - JBB</title> > > <link href="css/IISJBB_Screen.css" rel="stylesheet" type="text/css" > media="screen" /> > > <link href="css/IISJBB_Print.css" rel="stylesheet" type="text/css" > media="print" /> > > <script language="JavaScript" type="text/javascript" > src="js/commonScripts.js"></script> > > </head> > > > > <body> > > <table border="0" cellpadding="0" cellspacing="0" class="tableBorderColor"> > > <tr> > > <td> > > > > > > > > > > > > <head> > > <script> > > > > function HandleOnClose() { > > if (event.clientY < 0 && event.clientX < 0) { > > if (!self.closed){ > > > document.location.href='LogoutAction.do'; > > } > > } > > } > > > > function callHelp(){ > > //window.open('HelpAction.do?method=retrieveHelp','Help', > 'width=720,height=500,resizable=1,scrollbars=1'); > > if (typeof(screenId) == 'undefined' || screenId == ''){ > > alert('No Help text provided'); > > }else { > > //alert(screenId); > > window.open('Help/'+screenId,'Help', > 'width=720,height=500,resizable=1,scrollbars=1'); > > } > > } > > > > function callUserManual(){ > > if (typeof(screenId) == 'undefined' || screenId == ''){ > > alert('No UserManual Provided'); > > }else { > > if(screenId=42){ > > //alert(screenId); > > > window.open('attachment/IISUserManual_Licensee.doc','Help', > 'width=720,height=500,resizable=1,scrollbars=1'); > > } > > } > > } > > > > </script> > > <LINK REL="SHORTCUT ICON" HREF="images/JBBlogo.ico"> > > </head> > > <body onunload="HandleOnClose()"> > > > > <table border="0" align="center" cellpadding="0" cellspacing="0" > > class="mainTable"> > > <tr> > > <td valign="top"> > > <div id="header"> > > <table width="1000" border="0" > cellpadding="0" cellspacing="0"> > > <tr valign="top"> > > <td > width="357" valign="top"><img src="images/JBB-Logo.jpg" > > > title="Institutional Information System - JBB" width="221" > > > height="70" /></td> > > > > <td > width="260" align="center" valign="top"> > > > <font > > > color="white"> <img src="images/IIS.JPG" > > > title="Institutional Information System - JBB" /> </font><br /> > > </td> > > > > <td > width="383" nowrap="nowrap" class="gNav" valign="Top" > > > align="right"> > > > > > > > > > <!-- <a href="javascript:callUserManual()">User Manual</a> | --> > > > > > > <a href="javascript:callHelp()">Help</a> | > > > > > > <a href="LogoutAction.do"> > > > > > > > > > > > > > > > </a> </td> > > </tr> > > <tr> > > <td > colspan="3"><img src="images/hdrStrip.jpg" width="1000" > > > height="33" title="" border="0" /></td> > > </tr> > > </table> > > </div> > > <table width="1000px" border="0" > cellspacing="0" cellpadding="0"> > > <tr valign="top"> > > <td > width="200" class="lNavSection"> > > <div > id="lNav"></td> > > </tr> > > </table> > > > > > > > > > > > > > > > > > > > > <head> > > <script> > > javascript:window.history.forward(1); > > var changesDone=""; > > function callFun(url,funName) { > > var flag = "true"; > > if (changesDone == 'Y') { > > > if(confirm('You have some unsaved changes. Do you wish to continue ? ')){ > > > flag="true"; > > }else { > > > flag="false"; > > } > > } > > if (flag == "true") { > > > document.GreetingForm.action=url; > > > document.GreetingForm.functionName.value=funName; > > > document.GreetingForm.submit(); > > } > > } > > </script> > > </head> > > <table width="899px" border="0" cellspacing="0" cellpadding="0"> > > <tr valign="top"> > > <td width="200" class="lNavSection"> > > <div id="lNav"> > > <table width="200" border="0" > cellspacing="0" cellpadding="0" height="193"> > > > > <form action="" > name="GreetingForm" method="post"> > > > > > > <input type="hidden" > name="functionName" value="View Reports"> > > <tr> > > > > <td > valign="top" class="loginName"><strong>John Jackson</strong><br /> > > > > - > PENETRATION TEST <br /> > > > > [Role > for Penetration Test] <br /> > > > > <br /> > > <strong> > Last Login </strong> <br /> > > - > 22-05-2014 11:13:16 <br /> > > > > > > </td> > > </tr> > > > > <tr> > > > > <td > valign="top"><!--Create Banking Service Profile--> > > > > > > > > > > > > > > > > > > > > > > > > > > <a > href="javascript:callFun('ReportHomePageAction.do?method=display','View > Reports');" > > > class="Lnav enable" title="Select and View > reports "> > > > View Reports > > > <!-- JBB.iis.business.common.Function@35e76724--> > > > > > > > > > > > > > </td> > > </form> > > </tr> > > > > </table> > > > > <br /> > > <img src="images/NavBottom.jpg" title="" > width="200" height="200" /> > > <!-- > > </div> > > </td> > > <td > valign="top" width="699"> > > <table > border="0" width="500" cellspacing="0" cellpadding="0"> > > > <tr> > > > > <td class="brdCrumb"> > > > > </td> > > > </tr> > > > </table> > > > <table width="100%" border="0" > cellspacing="0" cellpadding="0"> > > > <tr> > > > > <td>--></div> > > </td> > > <td valign="top" width="699"> > > <table border="0" width="500" > cellspacing="0" cellpadding="0"> > > <tr> > > <td > class="brdCrumb"><!-- InstanceBeginEditable name="Breadcrumb" --><!-- > InstanceEndEditable --></td> > > </tr> > > </table> > > > > > > > > > > > > > > > > > > > > > > <head> > > <!-- InstanceBeginEditable name="doctitle" --> > > <title>Institutional Information System - JBB</title> > > > > </head> > > <body> > > <form name="ReportHomePageForm" method="post" > action="/IIS/ReportHomePageAction.do"> > > <table border="0" align="center" cellpadding="0" > cellspacing="0" > > class="mainTable"> > > <tr> > > <td valign="top" > width="699"> > > <table border="0" > width="500" cellspacing="0" cellpadding="0"> > > <tr> > > > <td class="brdCrumb"><!-- InstanceBeginEditable > name="Breadcrumb" --><a > > > > href="IISDashBoardAction.do?method=viewDashBoard">Dashboard </a> > > View Reports<!-- InstanceEndEditable --></td> > > </tr> > > </table> > > <table width="100%" > border="0" cellspacing="0" cellpadding="0"> > > <tr> > > > <td> > > > <h1>View Reports <!-- InstanceEndEditable --></h1> > > > </td> > > </tr> > > <tr> > > > <td class="errMsg"></td> > > </tr> > > <tr> > > > <td valign="top" class="PD22"> > > > <table width="560" border="0" cellpadding="0" cellspacing="0"> > > > <tr> > > > <td width="280" valign="top"></td> > > > <td valign="top" width="280"></td> > > > </tr> > > > <tr> > > > <td valign="top"></td> > > > <td align="center" > valign="top"></td> > > > </tr> > > > </table> > > > <!-- Alerts and Reminders --> <!-- End Alerts and Reminders --> > > > > > <table width="689" border="0" cellspacing="0" cellpadding="0"> > > > <tr> > > > <td valign="top" class="bdrTRBL"> > > > <table width="100%" border="0" > cellpadding="4" cellspacing="1" > > > class="tblTxt" > id="report"> > > > <tr> > > > > <td width="15%" align="center" class="tblHeader">Report > > > > No</td> > > > > <td width="85%" class="tblHeader">Report Name</td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">1</td> > > > > <td class="tblTD"><a > > > > > href="ReportNoOfInstitutions.do?method=displayNoOfInstitutions">All > Institutions </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">2</td> > > > > <td class="tblTD"><a > > > > > href="ReportJBBRegister.do?method=displayJBBRegister">JBB Register for > All Institutions </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">3</td> > > > > <td class="tblTD"><a > > > > > href="ReportCategoryInstitutions.do?method=displayCategoryListOfInstitutions">Category-wise > List of Institutions </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">4</td> > > > > <td class="tblTD"><a > > > > > href="ReportListOfInsOnGivenDate.do?method=displayListOfInsOnGivenDate">List > of Institutions as on a Given Date </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">5</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionsAction5.do?method=displayContactDetails">Contact > Details for all Institutions</a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">6</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionsAction6.do?method=displayAddMgmtDetails">Address > And Management Details </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">7</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionsAction7.do?method=displayExternalAuditors">External > Auditors </a></td> > > > </tr><!-- > > > <tr> > > > > <td align="center" class="tblTD">8</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionsAction8.do?method=displayAddCeoDetails">Address > And CEO Manager Details </a></td> > > > </tr> > > > --><tr> > > > > <td align="center" class="tblTD">8</td> > > > > <td class="tblTD"><a > > > > > href="ReportActuaries.do?method=displayactuaries">Actuaries </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">9</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionsAction10.do?method=displayListOverseasBranch">List > of Foreign Branches for Local Banks </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">10</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionsAction11.do?method=displayNoOfBranches">Institution-wise > Number of Branches </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">11</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionCommonAction12.do?method=displayOnsiteAtms">Institution-wise > Onsite and Off-site ATM details </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">12</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionCommonAction13.do?method=displaylistofsubsidiaries">List > of Subsidiaries </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">13</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionCommonAction14.do?method=displaylistofstaff">Number > of Staff in jamblan and outside jamblan in All Institutions </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">14</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionCommonAction15.do?method=displaylicenseinfo">License > Information </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">15</td> > > > > <td class="tblTD"><a > > > > > href="ReportListOfInstitutionsBODDirector.do?method=displayListOfInstutionBOD">List > of Institutions where a given person is in the Board of Director or > Executive </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">16</td> > > > > <td class="tblTD"><a > > > > > href="ReportManagementHistory.do?method=displayManHistory">Management > History </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">17</td> > > > > <td class="tblTD"><a > > > > > href="ReportAllInstitutionsAction18.do?method=displayListShareholders">List > of majority shareholders for all institutions </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">18</td> > > > > <td class="tblTD"><a > > > > > href="ReportAllInstitutionsAction19.do?method=displayListOfDirectors">List > of Board of Directors </a></td> > > > </tr><!-- > > > <tr> > > > > <td align="center" class="tblTD">20</td> > > > > <td class="tblTD"><a > > > > > href="ReportAllInstitutionsAction20.do?method=displayExeMgmtDetails">Senior > Management details </a></td> > > > </tr> > > > > > --><tr> > > > > <td align="center" class="tblTD">19</td> > > > > <td class="tblTD"><a > > > > > href="ReportChangesToBOD.do?method=displayChangeOfBOD">Changes made to > an Institution Board of Directors </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">20</td> > > > > <td class="tblTD"><a > > > > > href="ReportLastInspectionVisit.do?method=displayLastInspection">Last > Inspection Visit Details </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">21</td> > > > > <td class="tblTD"><a > > > > > href="ReportLastPrudentialMeeting.do?method=displayLastPrudMeet">Last > Prudential Meeting Details </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">22</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionsExemption.do?method=displayInstitutionExemption">List > of Institutions with Exemptions provided </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">23</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstUnderLiquidation.do?method=displayInstUnderLiqudation">List > of Institutions under Liquidation</a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">24</td> > > > > <td class="tblTD"><a > > > > > href="ReportListOfSharesSuperBoard.do?method=displayShares">List of > Shares Supervisory Board </a></td> > > > </tr> > > > > > <tr> > > > > <td align="center" class="tblTD">25</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionCommonAction27.do?method=displayCapitalShares">Capital > and Share Details </a></td> > > > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">26</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionCommonAction28.do?method=displayOwnership">Aggregate > Exclusive List of Ownership in Other Companies Listed in BSE </a></td> > > > > > </tr> > > > > > <tr> > > > > <td align="center" class="tblTD">27</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionCommonAction29.do?method=displayCommittee">Board > and Management Committee Details </a></td> > > > </tr> > > > <tr> > > > > <td align="center" class="tblTD">28</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionsAction32.do?method=displayTopMgmtContactDetails">Top > Management Contact Details (Only Top In Hierarchy) </a></td> > > > </tr> > > > <!-- <tr> > > > > <td align="center" class="tblTD">28</td> > > > > <td class="tblTD"><a > > > > > href="ReportInstitutionsAction35.do?method=displayNewOrCancelled">List > of new or Cancelled Institutions </a></td> > > > </tr>--> > > > <!-- <tr> > > > > <td align="center" class="tblTD">31</td> > > > > <td class="tblTD"><a > > > > > href="ReportSpecialPurposeVehicles.do?method=displaySpecialPurposeVehicles">Special > Purpose Vehicles</a></td> > > > </tr>--> > > > </table> > > > <!-- InstanceEndEditable --></td> > > > </tr> > > > </table> > > > </td> > > </tr> > > </table> > > </td> > > </tr> > > </table> > > </td> > > </tr> > > > > </form> > > </body> > > <!-- InstanceEnd --> > > </html> > > > > > > > > > > > > > > > > > > > > > > <table width="100%" border="0" align="center" cellpadding="0" > cellspacing="0" > > > > > <table width="100%" border="0" cellspacing="0" cellpadding="0"> > > > > </table> > > > > </body> > > </html> > > > > > > > > > > -------- Original Message -------- > Subject: Re: [W3af-users] web_spider not crawling proprely > From: Andres Riancho <andres.rian...@gmail.com> > To: Ali Khalfan <ali.khal...@gmail.com> > CC: "w3af-users@lists.sourceforge.net" <w3af-users@lists.sourceforge.net> > Date: Tue May 20 2014 22:44:22 GMT+0300 (AST) > >> Maybe the site is rather complex (a lot of JavaScript), and can't be >> understood by w3af's HTML parser? >> >> If so, try this out: >> http://docs.w3af.org/en/latest/complex-web-apps.html >> >> On Tue, May 20, 2014 at 1:50 AM, Ali Khalfan <ali.khal...@gmail.com> wrote: >>> hi andres, >>> >>> i noticed when scanning a few of my applications that href" links are not >>> being detected by the web_spider plugin.. It seems that the only links >>> detected are images and stylesheets. >>> >>> i've taken a quick glance at the plugin code and it doesn't seem that anchor >>> links are being parsed. >>> >>> Is this the case? >>> >>> >>> ------------------------------------------------------------------------------ >>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE >>> Instantly run your Selenium tests across 300+ browser/OS combos. >>> Get unparalleled scalability from the best Selenium testing platform >>> available >>> Simple to use. Nothing to install. Get started now for free." >>> http://p.sf.net/sfu/SauceLabs >>> _______________________________________________ >>> W3af-users mailing list >>> W3af-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/w3af-users >>> >> >> >> -- Andrés Riancho Project Leader at w3af - http://w3af.org/ Web Application Attack and Audit Framework Twitter: @w3af GPG: 0x93C344F3 ------------------------------------------------------------------------------ The best possible search technologies are now affordable for all companies. Download your FREE open source Enterprise Search Engine today! Our experts will assist you in its installation for $59/mo, no commitment. Test it for FREE on our Cloud platform anytime! http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk _______________________________________________ W3af-users mailing list W3af-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/w3af-users