From: [email protected] To: [email protected] Subject: RE: [Bug-wget] Wget export URL list Date: Wed, 3 Sep 2014 16:12:07 -0400
> Can anyone tell me how to do this with wget ? > I want it to spider a given website and return the list of full urls in > that website. You haven't provided any details of your requirements beyond obtaining the result and using wget. wget alone will not do what you want. If it were me, I would: 1. wget the html for the site (only download the html) 2. run the html through a simple bash script to rip out all of the URLs - and grep -in and -out, as appropriate The simple script can be found at http://www.comp.eonworks.com/scripts/isolate_url_link-20020716.html This assumes a real OS (not win*) and have a small working knowledge of wget, CL (bash), and can dl and extract the script.
