Hi ...
Thanx for your answer ...
> It does realize it saw a page. However, it's criteria is based on the URL.
> So if you have several URLs pointing to the same document, you're going to
> get duplicates. More powerful duplicate elimination code is in the works.
On our Webserver System we have some MHonArc Archives, all with
the same url !
The whole Sever is about 3.500 Documents large and the last
run told me, that htDig has about 55.000 Documents in its
Database .. I think this is way to much ;)
We had one "cross link" with different directories and only
Filesystem Symlinks, but we eliminated this one (so there should
be 100 files less ;))
Actually i don't see a message like "already habe this document
scipping everything inside ..."
Do you have any idea about this ?
Thanx
Michael
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED] containing the single word "unsubscribe" in
the SUBJECT of the message.
