HTDig 3_1_3 Sun Solaris 7 I have installed the above binary and am able to index and search my website OK apart from one area. I have a page in the website's document root - i.e. at the same level as the home page (which in my case - the homepage is an shtml file - I don't think this makes any difference though). I have a page in the same directory which is not linked directly from the home page but is referenced from a page 'lower' in the site's directory structure. However when I search for text I know is contained in this page the page is not returned by the search. The page being missed has no META noindex tag and I perfomed the following index build to see the list of URL's produced: htdig -vvv - the first lines of output are listed below. I have no robots.txt file and do not understand the 'retrieval command' line. Can I turn the robots.txt request off? New server: www.sghms.ac.uk, 80 Retrieval command for http://www.sghms.ac.uk/robots.txt: GET /robots.txt HTTP/1. 0 User-Agent: htdig/3.1.3 ([EMAIL PROTECTED]) Host: www.sghms.ac.uk Header line: HTTP/1.1 404 Not Found Header line: Date: Tue, 04 Jan 2000 14:18:41 GMT Header line: Server: Apache/1.3.9 (Unix) Header line: Connection: close Header line: Content-Type: text/html Header line: returnStatus = 1 pushed Any ideas where I am going wrong and how I can include the missing page in my search? Thanks Paul ----------------------------------- ------------------------------------ To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this.
