Running the htdig database-creating command fails for me.  Amongst the
diagnostic output lines I got by using a few "v"'s were:

Parsing robots.txt file using myname = htdig

Robots.txt line: User-agent: *

Found 'user-agent' line: *

Pattern: 
 pushed
   Rejected: forbidden by server robots.txt!


The site which includes the pages I wish to index has a robots.txt file with
the line

User-agent: *

This should permit indexing by htdig.  So what's going wrong?  TIA.

Douglas

========
Douglas Kline
[EMAIL PROTECTED]




-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
ht://Dig general mailing list: <[EMAIL PROTECTED]>
ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html
List information (subscribe/unsubscribe, etc.)
https://lists.sourceforge.net/lists/listinfo/htdig-general

Reply via email to