Thanks for the help! Yes, it works fine without a wildcard search, which I believe at this point will be ok for our app.
Thanks again,
Cory
At 11:50 AM 5/28/2003 -0400, you wrote:
Cory,
When performing wildcard queries, the bulk of the memory is used during wildcard term expansion. The memory requirement is proportional to the number of matching terms, not the number of hits.
You should make sure you are using the latest Lucene. There was a fix in 1.3 to reduce the memory requirements of all all queries.
But, wildcard queries that expand to many terms are allways going to be memory intensive in Lucene. We ran into this problem and decided to put a check on the number of expanded terms and abort the query if the number got too high. If you're ambitious, you could modify the Lucene source to serialize the query process for queries with a large number of terms, but that would be a bit of work. If you absolutely require these huge wildcard queries, then you may have to look into it, though.
Non-wildcard queries that return a large result set should not be a memory problem, though.
Dave
Cory Albright
<[EMAIL PROTECTED] To: "Lucene Users List"
com> <[EMAIL PROTECTED]>
cc:
05/28/03 11:16 AM Subject: Re: too many hits - OutOfMemoryError
Please respond to
"Lucene Users
List"
Yes. Is that the problem?
At 05:13 PM 5/28/2003 +0200, you wrote: > > When I search with a query I know will hit most of the 1.8 million > > records, the "collect" print > > does not even print, it eats up the 700+MB I allocated and then > > throws an OutOfMemoryError. > >Are you using wildcard queries? > >-- >Eric Jain > > >--------------------------------------------------------------------- >To unsubscribe, e-mail: [EMAIL PROTECTED] >For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
