Lachlan, Question:
What is your wordlist_page_size set to? The htdig default is zero, and the BDB default of 8K (in most situations) is then used. Altough the BDB max page size is 64K, we can't use that yet as a result of a multiplication bug in mp_cmpr I haven't tracked down yet. I use this as my default: wordlist_page_size: 32768 Larger pages are usually more efficient, especially since here we pay the overhead of deflating each page individually before returning the data. If your bug is caused by page overflow as I suspect, then this change will at least push the bug 'away' so that you may have to index several orders of magnitude more than 50,000 pages to see the bug. We've got all kinds of problems if we want to try and index 5 Million+ pages. I could be wrong, but I'd be interested to see if it makes the problem go away. Thanks! On Fri, 14 Feb 2003, Lachlan Andrew wrote: > An error occurs during an htdump straight after htdig. However, I > haven't yet got it to occur *within* htdig. > > Interestingly, the error first reported by htdump is similar to the > one I last reported, > > WordDB: CDB___memp_cmpr_read: unable to uncompress page at pgno = 23 > WordDB: PANIC: Input/output error > WordDBCursor::Get(17) failed DB_RUNRECOVERY: Fatal error, run > database recovery > > but the one by htpurge (and subsequent htdumps) is > > WordDB: CDB___memp_cmpr_read: unexpected compression flag value 0x8 > at pgno = 26613 > WordDB: PANIC: Successful return: 0 > WordDBCursor::Get(17) failed DB_RUNRECOVERY: Fatal error, run > database recovery > > I'll keep looking... > > On Friday 14 February 2003 05:05, Neal Richter wrote: > > Please attempt to reproduce the error using ONLY htdig next. > > > > If the error is still present, the the error is in htdig. If the > > error is not present then the bug is happening during htpurge. > > > > ------------------------------------------------------- > This SF.NET email is sponsored by: FREE SSL Guide from Thawte > are you planning your Web Server Security? Click here to get a FREE > Thawte SSL guide and find the answers to all your SSL security issues. > http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en > _______________________________________________ > htdig-dev mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/htdig-dev > Neal Richter Knowledgebase Developer RightNow Technologies, Inc. Customer Service for Every Web Site Office: 406-522-1485 ------------------------------------------------------- This SF.NET email is sponsored by: FREE SSL Guide from Thawte are you planning your Web Server Security? Click here to get a FREE Thawte SSL guide and find the answers to all your SSL security issues. http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en _______________________________________________ htdig-dev mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/htdig-dev
