Thanks a lot for your replies, Andi and Christian. This is exactly what I saw... about 11 threads per python process that was making lucene queries...
I have a follow-up question now (may be even naive): Andi, about your suggestion of using multithreading and that PyLucene will release GIL when it calls into JVM... So, when it does that, another python thread will acquire GIL, and then call into JVM, and then another thread... so on and so forth. Will this again not lead to too many Java threads running in parallel?? Did I misunderstand what you were suggesting? The staff here who run parallel/multithreaded java code use the cmd "java -XX:ParallelGCThreads=16 JavaApp". I tried to pass the same argument as "initVM(vmargs="-XX:ParallelGCThreads=4", but it doesn't obey. Am I missing something here, or misusing it, may be?? Himanshu On Fri, Jul 15, 2011 at 9:47 AM, Christian Heimes <li...@cheimes.de> wrote: > Am 15.07.2011 10:10, schrieb Andi Vajda: > > PyLucene embeds a Java VM. Thus, with each subprocess, a new JVM is > created with all its threads. This can get insane pretty quickly. > > The Java VM starts a lot of threads. On my Linux box eleven threads > additional threads are running after initVM() has been called. > > >>> import lucene, os, psutil > >>> psutil.Process(os.getpid()).get_num_threads() > 1 > >>> lucene.initVM() > <jcc.JCCEnv object at 0x7f23a66f31e0> > >>> psutil.Process(os.getpid()).get_num_threads() > 12 > > Christian >
