I found this article: http://www.mail-archive.com/[EMAIL PROTECTED]/msg07789.html which actually applies to 3.2.0b5, which seems to be the version you are using. I downloaded 3.2.0b5 and compared:
/mypath/to/htdig/source/code/Server.cc (in 3.2.0b4) with /mypath/to/htdig/source/code/Server.cc (in 3.2.0b5)
with the patch suggested by Gilles in the msg07789 message.
I don't know C++, but I know C. I compared the three until I thought I understood the source code, and basically bodged the three together, recompiled 3.2.0b4, and it worked.
You may be able to use the latest snapshot of 3.2.0b5, without the compare-and-bodge. :-)
Jane
Douglas Kline wrote:
Hello,
I posted a similar problem, and had to find the results for myself. The problem wasn't really the robots.txt file - it was part of HtDig that needed patching. We are using HtDig 3.2.0b4. If that's your version, I can send you the patch I used.
Jane Dudley, Webmaster
Douglas Kline wrote:h
Running the htdig database-creating command fails for me. Amongst the diagnostic output lines I got by using a few "v"'s were:
Parsing robots.txt file using myname = htdig
Robots.txt line: User-agent: *
Found 'user-agent' line: *
Pattern: pushed
Rejected: forbidden by server robots.txt!
The site which includes the pages I wish to index has a robots.txt file wit
the line
User-agent: *
This should permit indexing by htdig. So what's going wrong? TIA.
Thanks for your response. I are working on version 3.2.0b5. Would the patch you have work on that version too? Where did you find the patch?
Douglas
======== Douglas Kline [EMAIL PROTECTED]
------------------------------------------------------- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click _______________________________________________ ht://Dig general mailing list: <[EMAIL PROTECTED]> ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-general
