> Is it the old 2048 MB limit on filesize? Linux supports longs > for fseek in the like, is htdig limited to fseeking with int's? > > Is this a berkeley db limitation?
In some sense, yes, in some sense no. For one, keep in mind that it's not *Linux* per-se that has any problems with 2GB files as Linux on 64-bit systems have no problems with this. (Though as I've pointed out in the past, FreeBSD and Solaris/x86 have no problem with it either.) IIRC, it's also more than just an x86-Linux kernel issue, but a bit more complicated (glibc is also involved). The Berkeley DB code when compiled tries to figure out whether the OS has the 2GB limitation. You should look at the configure output from the db about "bigfile" and you may wish to try compiling it separately from ht://Dig with ./configure --enable-bigfile I'm not sure if the configure flag will force it to try bigfile support if the configure script feels your OS doesn't properly support them. > Does Htdig 3.2.X have the same limitation? Sometimes, as above. Though it's harder to break the limit since the word database is never written as an ASCII file, always compressed, and the document database has the db.excerpts split out. > Are there any guidelines (I checked the faq) for the number of url's > that are reasonable to index? Not really. Depends on how much of an excerpt you store, how big your URLs tend to be, whether you're running on a system with a 2GB limitation... Some users even back in the 3.1.0 days had installations of 10 million URLs on Solaris systems. -- -Geoff Hutchison Williams Students Online http://wso.williams.edu/ _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
