On Wednesday, February 19, 2003, at 11:57 AM, Abbie Greene wrote:
I just tried running HTDig 3.2.0b4 on a new set of files and received what seemed like millions of the following error message:If you search the mailing list archives for 'wordkey compare', you will find that this problem has been reported in the past. However I am not aware of any final resolution regarding the problem. First, you might want to just delete your current databases and start indexing from scratch. Past reports seem to indicate that this problem is often intermittent. Reindexing will also cover the possibility that the databases were somehow corrupted due to something external to ht://Dig. If the problem is repeatable, that might be of interest to some of the developers.
�
WordKey::Compare: key length for a or b < info.num_length
Aside from reindexing, you might want to ensure that you have the most recent snapshot (or at least a relatively recent one). If you are encountering this problem repeatedly with a recent snapshot, that would also likely be of interest.
HTdig was working perfectly yesterday for me, however it was also on a set of 4000 files�I now have 26000 files it�s running against.�I realize this is MOST likely the problem as I�veThe number of documents you are dealing with should not itself be a problem. ht://Dig is regularly used for much larger collections with nothing particularly exotic in the way of hardware.
reached the outer limits of this system.�Now my question is this��Would it make a difference to set up separate databases to reduce the size of each database (however each database would still be an index of at least 5000 files).�I�m just curious how other people of dealt with this issue in the past.Unless it is advantageous for you to build multiple databases for organizational purposes, a single database is most likely sufficient for the amount of data you appear to be dealing with.
Jim
-------------------------------------------------------
This SF.net email is sponsored by: SlickEdit Inc. Develop an edge.
The most comprehensive and flexible code editor you can use.
Code faster. C/C++, C#, Java, HTML, XML, many more. FREE 30-Day Trial.
www.slickedit.com/sourceforge
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
