On Tuesday November 20 2007 10:39:00 am Bill Peverill wrote: > Java Lucene route. The GCJ version has served us well in a single thread > app, and we believe a JCC version will work great, but sorting out the > JCC/windows build process may be more risky for us than simply implementing > Java Lucene, with all its horrors. [Simply? Did I just say that?]
Look, building JCC from source is just not that hard. Get the build depdencies (binaries s.b. available for your platform), do a svn checkout, `python jcc/setup.py`, edit the Makefile (basically just set paths, there are examples), `make`, `make test` `make install`. Take a day or two, if it works, it'll be worth your time. As I already mentioned and should have been indicated from the ML/google, the solution to the "too many heap sections" is using LARGE_CONFIG, which is a build-time option to GCJ. If you're not comforable building PyLucene-JCC, don't even bother going this route, it's way harder. If you want to go the Java route, try Solr. > But First we will build a periodic restart mechanism for our GCJ > implementation. Except for the leak, this system is VERY fast and This is not as simple as it sounds, especially if you don't want to lose in-process requests or requests coming over the wire. > functionally complete, so we want to see it through [as do our paid > subscribers!] For that, I'll just read "as does your boss". ;-) > The secret will be to find a metric which can reliably > predict an impending failure.. -time- is the most basic, but we would We never found anything that was a particularly good indicator and settled for timed rotation. > Pete, it looks like you used this solution for a while as well.. did you > just restart on a schedule, or were you able to monitor for a predictive We restart on a schedule, but that's driven by our update mechanism, which mirrors the Solr index distribution approach. As I said, we were pretty much going to do restarts anyway. The difference from Solr is that while they swap indexes in-process, we swap processes. We recently threw things behind a load-balancer, which simplifies things somewhat, but it's still ugly. > event? Also, you restarted the process/service, but did you have to restart > the machine periodically as well, or was your solution stable without > machine restarts? [if restarting a service every hour can be called > stability.] If you need to restart your box for a problem like this, you need a new OS. I think this is starting to get a bit OT for this list. Best of luck. -- Peter Fein || 773-575-0694 || [EMAIL PROTECTED] http://www.pobox.com/~pfein/ || PGP: 0xCCF6AE6B irc: [EMAIL PROTECTED] || jabber: [EMAIL PROTECTED] _______________________________________________ pylucene-dev mailing list [email protected] http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
