Are the pages in correct HTML syntax? I once had a the problem
that someone over here wrote bad HTML; netscape presented it
"correctly" whereas htdig failed to index the (bottom of the)
page where all the links were (the last ... lines where seen
as comment because of a bad comment start).
Also use htdig -vvv to see more what htdig is doing.....
[EMAIL PROTECTED] wrote:
>
> I've been getting, from several different sites, messages of this type;
> manual inspection of the site reveals that there are in fact pages present.
> (and, there's no evidence of any robots.txt existing) . .
>
> Does anyone have any insights?
--jesse
----------------------------------------------------------------------
J. op den brouw Johanna Westerdijkplein 75
Haagse Hogeschool 2521 EN DEN HAAG
Faculty of Engineering The Netherlands
Electrical Engineering +31-70-4458936
----------------------------------------------------------------------
Linux - because reboots are for hardware changes
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
