Hi Tomas! > Thanks a lot, but unfortunately that didn't work... > I just do a simple: > wget -r http://localhost > And my unwanted file is included all the time... Hm :(
> Have you had it to work with 1.7 or are you using the CVS-version? CVS? No, I am just a stupid Windows user with some binaries from Heiko ;) Are you sure your wgetrc is recognized, that it is in the right directory? >From my experience I also noted that some servers block wGet as it is not a Browser. That is why I incorporated in my wGetGUI that the User can choose to have wGet ignore robots.txt and identify as a Mozilla browser. That normally should work. Maybe you also have to try both at the same time for your problem? Right now I am a bit puzzled what you meant by "I can't get wget 1.7 react on the following:" I thought you wanted wGet to ignore robots?! Correct? Good luck Jens http://www.jensroesner.de/wgetgui > > -----Original Message----- > From: Jens Roesner [mailto:[EMAIL PROTECTED]] > Sent: 29 October 2001 23:21 > To: Tomas Hjelmberg > Subject: Re: meta noindex > > Hi Tomas! > > Put > robots = off > in your wgetrc > You cannot use it in the command line if I am not mistaken. > I think it was introduced in 1.7 so you should have no problems. > > Good luck > Jens > http://www.jensroesner.de/wgetgui > > Tomas Hjelmberg schrieb: > > > > Hi, > > I can't get wget 1.7 react on the following: > > <html> > > <head> > > <META name="robots" content="noindex, nofollow"> > > </head> > > ... > > </html> > > > > Cheers /Tomas
