Re: [Bug-wget] -e robots=off problem
On 06/12/2011 06:34 AM, david chou wrote: If there is a robots file , how could I retrieve file from that site? I 've used the -e robots=off instruction , but it still can not mirror a web site. Use the --debug flag to diagnose, and compare the output with
[Bug-wget] -e robots=off problem
If there is a robots file , how could I retrieve file from that site? I 've used the -e robots=off instruction , but it still can not mirror a web site.