On Mon, 6 Dec 2004, Rob Ramsey wrote:
I am building a database of external urls as shown below. But when I start digging the limit_urls_to does not seem to be working it appears to keep adding New servers for all links on the pages (see below). This simple search is taking way to long can someone and the results are not as expected. Can someone point me in the right direction?
start_url: http://www.bid4assets.com/storefront/index.cfm?fuseaction=govtassets&sfi d=21 /
http://www.bidshares.com/auctions/index.cfm?fuseaction=act_search&auctio ntypeid=1 /
http://bottrealtyauction.com/listing.php3 /
The line continuation character is a backslash (\), not a slash. Using a slash is probably creating all sorts of problems in terms of parsing the config file. That is likely the reason why limit_urls_to is not behaving as expected.
Jim
-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/
_______________________________________________
ht://Dig general mailing list: <[EMAIL PROTECTED]>
ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html
List information (subscribe/unsubscribe, etc.)
https://lists.sourceforge.net/lists/listinfo/htdig-general
