> But, the problem with this was that wget honors > the new robots.txt file which means wget refuses to pull it down.
wget should ignore the robots.txt with the option
-erobots=off
(or add the line robots=off in .wgetrc)
Piero
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
