Yes, 2GB files are a common problem. I think the answer to your question is negative. However, in case you are using Linux please note that the newer kernels have support for very large files (i.e. files over 2GB). Also, FreeBSD and 64-bit OSs don't have that limit either.
This is an OS, not Java limitation. Some people have reported that they have successfully created indices over 2GB in size, but they were probably running one of the operating systems that don't suffer from this limitation. Otis --- Winton Davies <[EMAIL PROTECTED]> wrote: > Hi, > > I ran into a problem earlier this week, where by an index of 8 > million small documents resulted in an index file of 2GB. It turns > out this is a common file system limit (some say it might be a java > limit as well). > > Anyway, I have no idea which index file it was, but it seems that I > > need to be able to control the distribution of the index ? > > Each document looks like this: > > exact_keywords: my_phrase_with_underscores , indexed, but not > tokenized. > body: my phrase with underscores (repeated N times for weight) > <EOM> title words <EOM> body <EOM> > add_info1: > add_info2: (just stored, short info) > add_info3: > > The body is pretty short (say 120 words or so). > > > As it turns out, the Index I had created was actually usable (thank > > heaven, after 24 hours of indexing :)). > > *** Anyway, is there anyway to control how big the indexes grow ? > **** > > > Cheers, > Winton > > > p.s. A big thank you to Doug -- I was able to deliver my experimental > > results on time :) My boss is V.Happy :) > > > -- > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > __________________________________________________ Do You Yahoo!? Find a job, post your resume. http://careers.yahoo.com -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
