I want to exclude /var/www/html/tomas.html from being indexed.
It looks like:
<html>
<head>
<meta name="robots" content="noindex">
<title>Tomas</title>
</head>
<body>
<a href="http://www.blowfish:8080";>blowfish</a>
bu
</body>
</html>
I invoke wget with:
wget -r http://localhost
And tomas.html is unfourtnuately downloaded anyway...
-----Original Message-----
From: Daniel Stenberg [mailto:[EMAIL PROTECTED]]
Sent: 22 November 2001 13:15
To: Tomas Hjelmberg
Cc: Wget List
Subject: Re: bug?
On Thu, 22 Nov 2001, Tomas Hjelmberg wrote:
> Sorry, but can't anybody say at least that I'm wrong when I state that the
> <meta name="robots"> tag doesn't work?
> Has anyone got it to work under any circumstances?
Yes:
- State what you want to do.
- Describe how you do it.
- Describe what happens.
- Describe what you expected to happen.
Provide/show the HTML in the above.
--
Daniel Stenberg - http://daniel.haxx.se - +46-705-44 31 77
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
