Re: Lucene directory copy - master copy to local index

2009-06-22 Thread Ian Lea
Or if you don't want to use Solr, rsync is very good for this. Only takes changes, takes care of deletions, very robust. That is what Solr uses. See http://wiki.apache.org/solr/CollectionDistribution for details or ideas. -- Ian. On Sun, Jun 21, 2009 at 4:32 AM, Otis Gospodnetic < otis_gospod

Re: Lucene directory copy - master copy to local index

2009-06-22 Thread Michael McCandless
Solr (as of 1.4) is moving to Java only implementation for replication (http://wiki.apache.org/solr/SolrReplication). If you roll your own, make sure to only run rsync at "proper" times according to the master, eg, it's probably simplest to close the writer on the master before running rsync. If

Re: windows locking file problem

2009-06-22 Thread Malo Pichot
Hello, Thank you for help, Micheal. It works fine now. Next time I will trust Lucene a little bit more ;-) Malo Michael McCandless a écrit : > It's best to let IndexWriter manage the deletion of files (for exactly > this reason). > > It turns out, it's perfectly fine to open an IndexWriter wi

SegmentReader retaining memory

2009-06-22 Thread Groose, Brian
In the application I'm working on, I'm opening a new index every 15-20 minutes. This is done by opening the new index and then closing the old index. Opening one of these indexes, about 58GB on disk, appears to use about 700MB of memory based on some simple testing. I am passing -Xmx3072m to the

Re: Query rewriting/optimization

2009-06-22 Thread Luis Alves
look at http://issues.apache.org/jira/browse/LUCENE-1567 (New flexible query parser) This new parser allows for internally rewrites/optimizes Query, and it is backward compatible. Preetham Kajekar wrote: Hi, I am wondering if Lucene internally rewrites/optimizes Query. I am programatically