On Mon, 1 Oct 2001, Dan Langille wrote: > > Are you sure you've made the max_doc_size is large enough? etc. > > I can boost that easily. BTW: the documentation at
The question is not the missing document. The question is whether all URLs in the redirected document were indexed. The other question is what happened to the URL after htdig received the redirect--why are you not seeing the redirect followed. > Header line: Location: http://www.unixathome.org/adsl/archives/2001_05/ > redirect: http://www.unixathome.org/adsl/archives/2001_05/ OK... So why did it stop before it followed the redirect? There should be a few more lines below what you showed that would illustrate whether the redirected URL was properly added to the queue or rejected. > etc. I suspect the lack of a trailing / is causing the redirect > as mentioned above and then htdig never retries the > redirected URL. Does that sound plausible? Sure, this is probably why those URLs are being added incorrectly. But the fundamental question remains--redirects should be followed, but you're not seeing this. So what happened after the redirect? -- -Geoff Hutchison Williams Students Online http://wso.williams.edu/ _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
