Hello list a few simple things:
1)
If I have some .cgi scripts that output html will htdig index those to?
(Assuming I remove .cgi from the "don't search these" list)
IE http://domain.com/dynamic.cgi
2)
Is so, will it index urls I give it in a .url file that have query strings?
IE
http://domain.com/dynamic.cgi?name=fred
http://domain.com/dynamic.cgi?name=mary
Will return on a search for fred the entire url as the link?:
Fred's Page
... Hi I'm Fred...
http://domain.com/dynamic.cgi?name=fred 01/01/02, 1200 Bytes
3)
Do the url's I put in a .url file for rundig to index need to be encoded or unencoded?
IE ?name=joe mama&[EMAIL PROTECTED]
or ?name=joe%20mama&email=me%40me.me
4)
Will it follow links in html it is indexing that have query strings?
IE <a href="dynamic.cgi?name=monkey">Monkey</a> will it index
dynamic.cgi?name=monkey ?
5)
If my main robots.txt file says to not scan /member_area since it's a top secret
area and I set up a separate htdig database for /member_area with the authentication
in the config file, will htdig on trying to index /member_area see the robot.txt file
in it's parent directory and forbid indexing of it?
If 'Yes' how can I forbid a directory in the robots.txt file and still have a
separate db for that directory?
Thanks a bunch for your expertise and time!!!
Dan
-------------------------------------------------------
This SF.Net email is sponsored by: INetU
Attention Web Developers & Consultants: Become An INetU Hosting Partner.
Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
