According to Abdul Jabbar:
> Is there anyway to program Htdig to only search for Metadata and not index
> whole websites?. Some of the websites that I index Htdig absoultely
> cripples them is there anyway to slow down the relentless pressures? i.e.
> limit the number of hops?, please help. Thanks in advance.
See http://www.htdig.org/attrs.html#max_hop_count
http://www.htdig.org/attrs.html#server_wait_time
and http://www.htdig.org/attrs.html#server_max_docs
You could set factors other than meta_description_factor and
keywords_factor to 0 so that only the meta data is used in searches,
but that will not affect in any way the performance impact htdig has
on the servers you're indexing, whereas the attributes mentioned above,
particularly server_wait_time, should help. You can also reduce load
on the servers by avoiding indexing dynamic content such as CGI scripts
(use exclude_urls), and by indexing via the local filesystem wherever
possible (local_urls).
--
Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]>
Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil
Dept. Physiology, U. of Manitoba Phone: (204)789-3766
Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
