Re: solr different sizes on master and slave

2011-03-02 Thread Mike Franon
Right now I have the slave polling every 10 seconds, becuase we want to make sure they stay in sync. I have users who will do post directly from a web application. But I do notice it syncs very quick, becuase usually the update is only one or two records at a time. I am thinking maybe 10

Re: solr different sizes on master and slave

2011-03-02 Thread Mike Franon
Is it ok if I just delete the old copies manually? or maybe run a script that does it? On Tue, Mar 1, 2011 at 7:47 PM, Markus Jelsma markus.jel...@openindex.io wrote: Indeed, the slave should not have useless copies but it does, at least in 1.4.0, i haven't seen it in 3.x, but that was just a

Re: solr different sizes on master and slave

2011-03-02 Thread Markus Jelsma
Yes. But keep in mind that Solr may be actually using an index.TIMESTAMP directory for its live search. See either the replication.properties file or consult the replication page to see what index directory it uses. If it uses an index.TIMESTAMP directory you can safely move it to index and

Re: solr different sizes on master and slave

2011-03-02 Thread Jayendra Patil
Hi Mike, There was an issue with the Snappuller wherein it fails to clean up the old index directories on the slave side. https://issues.apache.org/jira/browse/SOLR-2156 The patch can be applied to fix the issue. You can also delete the old index directories, except for the current one which is

Re: solr different sizes on master and slave

2011-03-02 Thread Mike Franon
Thanks you very much for this info, that helps a lot! On Wed, Mar 2, 2011 at 10:05 AM, Jayendra Patil jayendra.patil@gmail.com wrote: Hi Mike, There was an issue with the Snappuller wherein it fails to clean up the old index directories on the slave side.

solr different sizes on master and slave

2011-03-01 Thread Mike Franon
I was curious why would the size be dramatically different even though the index versions are the same? One is 1.2 Gb, and on the slave it is 512 MB I would think they should both be the same size no? Thanks

Re: solr different sizes on master and slave

2011-03-01 Thread Markus Jelsma
Are there pending commits on the master? I was curious why would the size be dramatically different even though the index versions are the same? One is 1.2 Gb, and on the slave it is 512 MB I would think they should both be the same size no? Thanks

Re: solr different sizes on master and slave

2011-03-01 Thread Mike Franon
No pending commits, what it looks like is there are almost two copies of the index on the master, not sure how that happened. On Tue, Mar 1, 2011 at 3:08 PM, Markus Jelsma markus.jel...@openindex.io wrote: Are there pending commits on the master? I was curious why would the size be

Re: solr different sizes on master and slave

2011-03-01 Thread Mike Franon
ok doing some more research I noticed, on the slave it has multiple folders where it keeps them for example index index.20110204010900 index.20110204013355 index.20110218125400 and then there is an index.properties that shows which index it is using. I am just curious why does it keep multiple

Re: solr different sizes on master and slave

2011-03-01 Thread Jonathan Rochkind
The slave should not keep multiple copies _permanently_, but might temporarily after it's fetched the new files from master, but before it's committed them and fully wamred the new index searchers in the slave. Could that be what's going on, is your slave just still working on committing and

Re: solr different sizes on master and slave

2011-03-01 Thread Markus Jelsma
Indeed, the slave should not have useless copies but it does, at least in 1.4.0, i haven't seen it in 3.x, but that was just a small test that did not exactly meet my other production installs. In 1.4.0 Solr does not remove old copies at startup and it does not cleanly abort running