Samarendra Pratap [samarz...@gmail.com] wrote: > 1. Our default option is sort by score, however almost 8% of searches use > sorting on a field (yyyymmddHHMMSS). This field is indexed as string (not as > NumericField or DateField).
Guessing that the timestamp is practically unique for each document, sorting by String takes up a bit more than 18M * (40 bytes + 2 * "yyyymmddHHMMSS".length() bytes) ~= 1.2 GB of RAM as the Strings are cached. Coupled with the normal overhead of just opening an index of your size (500MB by your measurements?), I would have guessed that 3600MB would definitely be enough to open the index and do sorted searches. I realize that fiddling with production servers is dangerous, but connecting with JConsole and forcing a garbage collection might be acceptable? That should enable you to determine whether you're leaking memory or if it's just the JVM being greedy. I'd guess you leaking though, as HotSpot does not normally allocate up to the limit if it does not need to. Anyway, changing to one of the optimized fields for sorting dates should shave 1 GB off the memory requirement, so I'll recommend doing that no matter what the main cause of your memory problems is. Regards, Toke Eskildsen --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
