> On Thu, 2003-02-13 at 04:25, olinux wrote: > > this may sound silly, but make sure that you *have* > > links to all of your pages. htdig travels thru links > > similar to a visitor who clicks every link on your > > pages. Make sure that links are not javascript links > > as most search engines (including htdig?) are > > unable/do not follow them. If this is the case, you > > could setup a link page with links to all articles for > > htdig to use. > > The site is a traditional structure. The Javascript navigation is > seconded by plain text links on every page. These links lead to an index > page for each subject. And each and every page is linked from the > subject index. > > That is why I am worried...
Do an indexing run with very verbose output (htdig -vvv). Then you'll see what links htdig is finding in each page, what links are pushed and thus followed, and what links are rejected (and for what reason). E.g. for my own homepage pick: www.pdv-systeme.de, # servers = 1 0:0:0:http://www.pdv-systeme.de/users/martinv/: ... Read a total of 10582 bytes title: Martin's Homepage href: mailto:[EMAIL PROTECTED] () Rejected: Not an http or relative link! url rejected: (level 1)mailto:[EMAIL PROTECTED] image: http://www.pdv-systeme.de/users/martinv/anydamn.gif href: http://www.anybrowser.org/campaign/ ([Viewable With Any Damn Browser] ) Rejected: URL not in the limits! url rejected: (level 1)http://www.anybrowser.org/campaign/ ... resolving 'http://www.pdv-systeme.de/users/martinv/formmail_pod.html' pushing http://www.pdv-systeme.de/users/martinv/formmail_pod.html + ... 2:2:1:http://www.pdv-systeme.de/users/martinv/formmail_pod.html: etc. cu, Martin -- One OS to rule them all | Martin Vorlaender | VMS & WNT programmer One OS to find them | work: [EMAIL PROTECTED] One OS to bring them all | http://www.pdv-systeme.de/users/martinv/ And in the Darkness bind them.| home: [EMAIL PROTECTED] ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
