Otis, My app does run within Tomcat. But when I started getting these OutOfMemoryErrors I wrote a little unit test to watch the memory usage without Tomcat in the middle and I still see the memory usage.
Thanks, Jim --- Otis Gospodnetic <[EMAIL PROTECTED]> wrote: > Sorry if I'm stating the obvious. Is this happening > in some > stand-alone unit tests, or are you running things > from some application > and in some environment, like Tomcat, Jetty or in > some non-web app? > > Your queries are pretty big (although I recall some > people using even > bigger ones... but it all depends on the hardware > they had), but are > you sure running out of memory is due to Lucene, or > could it be a leak > in the app from which you are running queries? > > Otis > > > --- James Dunn <[EMAIL PROTECTED]> wrote: > > Doug, > > > > We only search on analyzed text fields. There are > a > > couple of additional fields in the index like > > OBJECT_ID that are keywords but we don't search > > against those, we only use them once we get a > result > > back to find the thing that document represents. > > > > Thanks, > > > > Jim > > > > --- Doug Cutting <[EMAIL PROTECTED]> wrote: > > > It is cached by the IndexReader and lives until > the > > > index reader is > > > garbage collected. 50-70 searchable fields is a > > > *lot*. How many are > > > analyzed text, and how many are simply keywords? > > > > > > Doug > > > > > > James Dunn wrote: > > > > Doug, > > > > > > > > Thanks! > > > > > > > > I just asked a question regarding how to > calculate > > > the > > > > memory requirements for a search. Does this > > > memory > > > > only get used only during the search operation > > > itself, > > > > or is it referenced by the Hits object or > anything > > > > else after the actual search completes? > > > > > > > > Thanks again, > > > > > > > > Jim > > > > > > > > > > > > --- Doug Cutting <[EMAIL PROTECTED]> wrote: > > > > > > > >>James Dunn wrote: > > > >> > > > >>>Also I search across about 50 fields but I > don't > > > >> > > > >>use > > > >> > > > >>>wildcard or range queries. > > > >> > > > >>Lucene uses one byte of RAM per document per > > > >>searched field, to hold the > > > >>normalization values. So if you search a 10M > > > >>document collection with > > > >>50 fields, then you'll end up using 500MB of > RAM. > > > >> > > > >>If you're using unanalyzed fields, then an > easy > > > >>workaround to reduce the > > > >>number of fields is to combine many in a > single > > > >>field. So, instead of, > > > >>e.g., using an "f1" field with value "abc", > and an > > > >>"f2" field with value > > > >>"efg", use a single field named "f" with > values > > > >>"1_abc" and "2_efg". > > > >> > > > >>We could optimize this in Lucene. If no > values of > > > >>an indexed field are > > > >>analyzed, then we could store no norms for the > > > field > > > >>and hence read none > > > >>into memory. This wouldn't be too hard to > > > >>implement... > > > >> > > > >>Doug > > > >> > > > >> > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > >>To unsubscribe, e-mail: > > > >>[EMAIL PROTECTED] > > > >>For additional commands, e-mail: > > > >>[EMAIL PROTECTED] > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________________________________ > > > > Do you Yahoo!? > > > > Friends. Fun. Try the all-new Yahoo! > Messenger. > > > > http://messenger.yahoo.com/ > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: > > > [EMAIL PROTECTED] > > > > For additional commands, e-mail: > > > [EMAIL PROTECTED] > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: > > > [EMAIL PROTECTED] > > > For additional commands, e-mail: > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > __________________________________ > > Do you Yahoo!? > > Friends. Fun. Try the all-new Yahoo! Messenger. > > http://messenger.yahoo.com/ > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > __________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
