Geoff,
Thank you for the input, I think I will try adding the list
as "hidden" links so other spiders can see it as well. I have found
htDig to be a very nice package and since I provide free access to
it on our Web servers for our clients they have been very satisfied.
Bruce
Geoff Hutchison wrote:
>
> At 11:55 PM -0400 4/18/99, Bruce R. Nevins wrote:
> >I would like to have htdig follow those links as well as the standard
> >ones. Is there a way to do this, even if I have to include special
> >tags in comments to force it to follow. I would like to avoid having
> >to create a list of URLs that would have to be added to the
> >configuration file.
>
> I'm afraid not. The fact is, there are just *way* too many ways people try
> to have "links" on pages. I don't know of any spider that follows what
> you've mentioned.
>
> Your choices are to either compile a list of links as Torsten suggested, or
> to have text links on the page. Remember that the config file can include
> other files, e.g.:
>
> start_url: `${common_dir}/my.urls`
>
> -Geoff Hutchison
> Williams Students Online
> http://wso.williams.edu/
--
Bruce R. Nevins, [EMAIL PROTECTED]
Vice President, Nexilis Communications Corp.
2527 E. 3rd St., Tucson, AZ, 85716-4114
(520) 795-6000, (520) 795-0158 (Fax)
http://www.nexiliscom.com/
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED] containing the single word "unsubscribe" in
the SUBJECT of the message.
