Hello, Erik. EH> Are you using multiple IndexSearcher instances? Or only one and EH> sharing it across multiple threads?
EH> If using a single shared IndexSearcher instance doesn't help, it may be EH> beneficial to port your code to Java and try it there. I have single instance of IndexSearcher and I pass reference of it to each thread. I will port code to Java if no other ideas will come my mind... EH> On Feb 16, 2005, at 3:04 PM, Yura Smolsky wrote: >> Hello. >> >> I use PyLucene, python port of Lucene. >> >> I have problem about using big index (50Gb) with IndexSearcher >> from many threads. >> I use IndexSearcher from PyLucene's PythonThread. It's really a wrapper >> around a Java/libgcj thread that python is tricked into thinking >> it's one of its own. >> >> The core of problem: >> When I have many threads (more than 5) I receive this exception: >> File "/usr/lib/python2.4/site-packages/PyLucene.py", line 2241, in >> search >> def search(*args): return _PyLucene.Searcher_search(*args) >> ValueError: java.lang.OutOfMemoryError >> <<No stacktrace available>> >> >> When I decrease number of threads to 3 or even 1 then search works. >> How do many threads can affect to this exception?.. >> >> I have 2 Gb of memory. So with one thread the process takes like >> 1200-1300Mb. >> >> Andi Vajda suggested that "There may be overhead involved in having >> multiple threads against a given index." >> >> Does anyone here have experience in handling big indexes with many >> threads? Yura Smolsky. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
