Jean-Marc Orliaguet wrote: > Julien Anguenot wrote: >>> apart from that, the NXLucene server died a day ago, I'm not sure if >>> this is related to the persistent connection bug on the client, or if it >>> is due to a log rotation that failed, or an invalid query. I'm still >>> trying to figure out. But since the lucene server was restarted it has >>> worked without any problem. >>> >> >> Did you get a core dump ? >> >> > > no, it looks more and more like a memory leak to me. the size of the > twistd process increases too fast, it never stabilizes or decreases again. > I tried with pylucene-2.1.0, it looks much better, but still... memory > figures go up when re-indexing a site.
yes we see the same thing here. You can switch the NXLucene server to DEBUG mode and see the Python GC status. The Python objects are constants there thus I can only suppose the memory leak is at GCJ GC level... Which is definitely hard to track... I would suggest you to restart your server once per day to flush the RAM... > >>> the lucene server setup is: >>> >>> 4 CPU 2GHz 64bits >>> Red Hat Enterprise Linux AS release 4 >>> gcc-3.4.6-3 (included in RedHat) >>> pylucene2 :-) >>> >> >> Of course, Red Hat enterprise :) You shouldn't get that much gcj bugs as >> with others distros. >> >> Are you aware about the gcc 3.4.6 bug related to the 2Go PyLucene store >> limitation ? [...] > yes and no :-) > > the main issue I believe is that the server is a 64bit architecture and > it has 4 CPUs (gcj 3.4.6 is too old for that type of architecture) oh ok. I was talking about the gcc version embedded with RHEL :) > So I have just installed the ubuntu 64bit binary > (http://downloads.osafoundation.org/PyLucene/linux/ubuntu64/) and the > memory usage now stays at: > > VIRT RES > 185m 68m > > even when reindexing an entire site (before it jumps at 600m !) Really amazing. Thanks for the info. Same gcc version over there ? > > also to bind twistd to a same cpu I start nxlucene with: > > $ taskset 1 bin/runnxlucene & > > so I believe it is one more problem solved, but all this is really edgy... Yes. GCJ use is definitely really egdy... > By the way, Julien are you planning to port nxlucene to JBoss, with a > similar XML-RPC service? We just hooked up the new search engine component within Nuxeo5 which later on can be used as a standalone server exposing let's say a SOAP interface instead of an XML-RPC interface. I would love to take time in the future to do this. Furthermore, instead of reimplementing my own XML queries and returning results as RSS streams I would implement Open Search nowadays (See : http://www.opensearch.org/Home) You'll have to wait a bit, though, because we got still couple of things to finish before starting to work out on this ;) J. -- Julien Anguenot | Nuxeo R&D (Paris, France) Open Source ECM - http://www.nuxeo.com Nuxeo 5 : http://www.nuxeo.org Mobile: +33 (0) 6 72 57 57 66
signature.asc
Description: OpenPGP digital signature
_______________________________________________ cps-devel mailing list http://lists.nuxeo.com/mailman/listinfo/cps-devel
