Re: htdig: Pages get indexed, but no results: BUG?

[Netsolution Internet Consulting]

Gilles Detillieux wrote:

>
> Who says it's deleting anything?  Does an htdig -vvv seem to suggest that?
>
> What I'm suggesting is that htdig sees the href to PRODUCTS.HTM before any
> href to products.htm, and so it queues up the upper-case URL, but marks
> the lower-case URL as visited (because all visits are recorded in lower
> case).  So, it tries to get PRODUCTS.HTM, and fails, so it never sees the
> real file.  Whenever it sees any of the good hrefs to products.htm, it
> thinks the file was already visited, so it doesn't queue it up again.
>
> Do you have any hard evidence that htdig is indeed fetching products.htm
> from the server, and deleting its hrefs?

I actually did run htdig -sivvv and I did see that the pages which are linked from
products.htm were defenitely indexed - there are at least 100 pages linked from
products.htm so its certain that they have been indexed.

But when I search with keywords from these pages, htsearch does not find any results - 
I
made several tests, so Im sure.

htsearc finds pages from that site which do not start from products.htm - no problem
there.

That is why Im assuming that the pages get indexed and then deleted again.

That is also why I think that it does not help when I take a start URL starting 
directly
from products.htm.

Also I dig several sites, so would it make sense to use limit urls to?

Andriu



----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED] containing the single word "unsubscribe" in
the body of the message.