Hi Lior, Sorry for the late reply. I am able to successfully reproduce this problem on Solr 5.3.1 and on Solr 6.0.0 as well.
The interesting thing is that if the master is restarted before the poll interval (1 minute in your case) then the index is not downloaded again. But if the slave makes even one failed poll attempt then on restart it downloads the entire index again from the master. I have opened https://issues.apache.org/jira/browse/SOLR-9036 On Wed, Apr 13, 2016 at 12:01 AM, Lior Sapir <lior.sa...@gmail.com> wrote: > So what do you say: > Is it a problem in my environment + confs > OR > That's how the replication is working > > (if a slave fails to locate the master when polling then next time the > master is available it will replicate the entire index even if no document > was added to the master and no optimization was performed) > > ? > > > > On Sat, Apr 9, 2016 at 9:24 PM, Lior Sapir <lior.sa...@gmail.com> wrote: > > > Thanks for the reply. > > > > <str name="pollInterval">00:00:60</str> - Is valid > > But I tried 00:01:00 anyway. > > I also checked the clocks and they are synced: > > ntpdate -q solr01-isrl01 > > > > server 192.168.103.112, stratum 11, offset 0.003648, delay 0.02589 > > 9 Apr 18:09:20 ntpdate[23921]: adjust time server 192.168.103.112 offset > > 0.003648 sec > > > > So these are not the reasons for the full replication. In addition the > > replication is working perfectly until I restart the master > > Regarding the issue of 60 seconds being too fast, I can consider raising > > it to 5 minutes even though my configuration is based on the data-driven > > example contained in the solr package. > > > > But still, this will just make the probability of full replication lower. > > I don't want to rely on that in production. if I have any network issue > or > > the master server will restart from any reason. All of his slaves will > > start replicating when the master will be available again and the service > > will be harmed dramatically or even be down. > > > > Anyway, > > > > Can anyone with solr version 5.3.1 or above test this scenario? I want to > > understand if its something specific in my environment or that's just how > > the replication is behaving. > > > > I added another step to be more clear: > > > > 1. Setup a master > > 2. Setup a slave in a different server > > 3. The slave replicated the master index > > 4. From now on not even a single document is added. No optimization or > > what so ever is done on the master or slave > > 5. I stop the master > > 6. wait for the slave to replicate or initiate a replication via the UI > or > > script > > 7. I start the master > > 8. I see the slave is replicating/copying the entire index > > > > > > Lior. > > > > > > > > > > On Sat, Apr 9, 2016 at 6:15 PM, Walter Underwood <wun...@wunderwood.org> > > wrote: > > > >> I’m not sure this is a legal polling interval: > >> > >> <str name="pollInterval">00:00:60</str> > >> > >> Try: > >> > >> <str name="pollInterval">00:01:00</str> > >> > >> Also, polling every minute is very fast. Try a longer period. > >> > >> Check the clocks on the two systems. If the clocks are not synchronized, > >> that could cause problem. > >> > >> wunder > >> Walter Underwood > >> wun...@wunderwood.org > >> http://observer.wunderwood.org/ (my blog) > >> > >> > >> > On Apr 9, 2016, at 8:10 AM, Lior Sapir <lior.sa...@gmail.com> wrote: > >> > > >> > Anyone can tell me what was I doing wrong ? > >> > Is that the expected behavior (slave replicate entire index if on > >> previous replication attempt the master was not available ) ? > >> > > >> > > >> > > >> > > >> > On Thu, Apr 7, 2016 at 9:12 PM, Lior Sapir <lior.sa...@gmail.com > >> <mailto:lior.sa...@gmail.com>> wrote: > >> > Thanks for the reply. > >> > > >> > I easily re produced it in my "sandbox" env. Steps to re produce > >> > 1. Setup a master > >> > 2. Setup a slave in a different server > >> > 3. The slave replicated the master index > >> > 4. From now on not even a single document is added. No optimization or > >> what so ever is done on the master or slave > >> > 5. I stop the master > >> > 6. I start the master > >> > 7. I see the slave is replicating/copying the entire index > >> > > >> > This is exactly what happened in production when I restarted the > >> master. > >> > > >> > I attached the configurations files. > >> > > >> > Replication section: > >> > > >> > Master: > >> > > >> > <requestHandler name="/replication" class="solr.ReplicationHandler"> > >> > <lst name="master"> > >> > <str name="replicateAfter">commit</str> > >> > </lst> > >> > </requestHandler> > >> > > >> > Slave: > >> > > >> > <requestHandler name="/replication" class="solr.ReplicationHandler" > > > >> > <lst name="slave"> > >> > <str name="masterUrl"> > >> http://solr01-isrl01.flr.local:8983/solr/replication-master/replication > < > >> http://solr01-isrl01.flr.local:8983/solr/replication-master/replication > >> ></str> > >> > <str name="pollInterval">00:00:60</str> > >> > </lst> > >> > </requestHandler> > >> > > >> > > >> > > >> > Best, > >> > Lior > >> > > >> > On Thu, Apr 7, 2016 at 6:56 PM, Erick Erickson < > erickerick...@gmail.com > >> <mailto:erickerick...@gmail.com>> wrote: > >> > What does your configuration file look like for the replication > >> > handler? Does this happen whenever you restart a slave even if > >> > _nothing_ has changed on the master? > >> > > >> > And this will certainly happen if you're optimizing the master before > >> > you restart, although that doesn't sound likely. > >> > > >> > Best, > >> > Erick > >> > > >> > On Thu, Apr 7, 2016 at 6:54 AM, Lior Sapir <lior.sa...@gmail.com > >> <mailto:lior.sa...@gmail.com>> wrote: > >> > > Solr slave is doing full replication (entire index) of index after > >> master > >> > > restart > >> > > Using solr 5.3.1 not cloud (using maser slave architecture ) I see > >> that > >> > > slave replicates entire index after master restart even though the > >> index > >> > > version is the same > >> > > > >> > > This is bad for me since the slave which is doing serving replicates > >> 80gb > >> > > if I restart the server and our service is down > >> > > > >> > > I attached a file with some snippets of the slave log before and > >> after the > >> > > master restart. > >> > > > >> > > Is there some default configuration issue causing this problem? > >> > > Both indexes master and slave were not updated for sure before and > >> after the > >> > > master restart. > >> > > The index version stayed exactly the same. > >> > > > >> > > > >> > > > >> > > >> > > >> > >> > > > -- Regards, Shalin Shekhar Mangar.
