Sounds like you might be reaching your max_doc_size - how does that compare with the size of this file?
Mike > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf > Of Janine Sisk > Sent: 03 February 2005 02:08 > To: htdig-general@lists.sourceforge.net > Subject: [htdig] Is there a maximum number of links htdig > will follow per page? > > > Sorry, I know the answer to this is in the archives (found it by > searching) but geocrawler.com isn't responding so I can't get to it. > > I have set up htdig to index a page that's dynamically generated and > contains approximately 15,000 links to articles within the > site. It is > only processing the first 1944 of them, leading me to believe > I must be > violating some rule of nature. Is there actually a maximum, > and if so > can I change it? If not, is there a strategy others use in this > situation? I'm thinking of putting out 1900 (or so) links on > a page, > followed by a link to the next page, and so on. > > thanks, > > janine > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: IntelliVIEW -- Interactive > Reporting Tool for open source databases. Create drag-&-drop > reports. Save time by over 75%! Publish reports on the web. > Export to DOC, XLS, RTF, etc. Download a FREE copy at > http://www.intelliview.com/go/osdn_nl > > _______________________________________________ > ht://Dig general mailing list: <htdig-general@lists.sourceforge.net> > ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html > List information (subscribe/unsubscribe, etc.) > https://lists.sourceforge.net/lists/listinfo/htdig-general > ******************************************************************** This email may contain information which is privileged or confidential. If you are not the intended recipient of this email, please notify the sender immediately and delete it without reading, copying, storing, forwarding or disclosing its contents to any other person Thank you Check us out at http://www.bt.com/consulting ******************************************************************** ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ ht://Dig general mailing list: <htdig-general@lists.sourceforge.net> ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-general
