Hello.
I use PyLucene, python port of Lucene.
I have problem about using big index (50Gb) with IndexSearcher
from many threads.
I use IndexSearcher from PyLucene's PythonThread. It's really a wrapper
around a Java/libgcj thread that python is tricked into thinking
it's one of its own.
The core of problem:
When I have many threads (more than 5) I receive this exception:
File "/usr/lib/python2.4/site-packages/PyLucene.py", line 2241, in search
def search(*args): return _PyLucene.Searcher_search(*args)
ValueError: java.lang.OutOfMemoryError
<<No stacktrace available>>
When I decrease number of threads to 3 or even 1 then search works.
How do many threads can affect to this exception?..
I have 2 Gb of memory. So with one thread the process takes like
1200-1300Mb.
Andi Vajda suggested that "There may be overhead involved in having
multiple threads against a given index."
Does anyone here have experience in handling big indexes with many
threads?
Any ideas are appreciated.
Yura Smolsky.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
