Hello, Pardon me if this has been already answered somewhere and I apologize for a lengthy post. I was wondering if anybody could help me understand Replication internals a bit more. We have a single master-slave setup (solr 1.4.1) with the configurations as shown below. Our environment is quite commit heavy (almost 100s of docs every 5 minutes), and all indexing is done on Master and all searches go to the Slave. We are seeing that the slave replication performance gradually decreases and the speed decreases < 1kbps and ultimately gets backed up. Once we reload the core on slave it will be work fine for sometime and then it again gets backed up. We have mergeFactor set to 10 and ramBufferSizeMB is set to 32MB and solr itself is running with 2GB memory and locktype is simple on both master and slave.
I am hoping that the following questions might help me understand the replication performance issue better (Replication Configuration is given at the end of the email) 1. Does the Slave get the whole index every time during replication or just the delta since the last replication happened ? 2. If there are huge number of queries being done on slave will it affect the replication ? How can I improve the performance ? (see the replications details at he bottom of the page) 3. Will the segment names be same be same on master and slave after replication ? I see that they are different. Is this correct ? If it is correct how does the slave know what to fetch the next time i.e. the delta. 4. When and why does the index.<TIMESTAMP> folder get created ? I see this type of folder getting created only on slave and the slave instance is pointing to it. 5. Does replication process copy both the index and index.<TIMESTAMP> folder ? 6. what happens if the replication kicks off even before the previous invocation has not completed ? will the 2nd invocation block or will it go through causing more confusion ? 7. If I have to prep a new master-slave combination is it OK to copy the respective contents into the new master-slave and start solr ? or do I have have to wipe the new slave and let it replicate from its new master ? 8. Doing an 'ls | wc -l' on index folder of master and slave gave 194 and 17968 respectively...I slave has lot of segments_xxx files. Is this normal ? MASTER <requestHandler name="/replication" class="solr. ReplicationHandler" > <lst name="master"> <str name="replicateAfter">startup</str> <str name="replicateAfter">commit</str> <str name="replicateAfter">optimize</str> <str name="confFiles">schema.xml,stopwords.txt</str> <str name="commitReserveDuration">00:00:10</str> </lst> </requestHandler> SLAVE <requestHandler name="/replication" class="solr.ReplicationHandler" > <lst name="slave"> <str name="masterUrl">master core url</str> <str name="pollInterval">00:03:00</str> <str name="compression">internal</str> <str name="httpConnTimeout">5000</str> <str name="httpReadTimeout">10000</str> </lst> </requestHandler> REPLICATION DETAILS FROM PAGE Master master core url Poll Interval 00:03:00 Local Index Index Version: 1296217104577, Generation: 20190 Location: /data/solr/core/search-data/index.20110429042508 Size: 2.1 GB Times Replicated Since Startup: 672 Previous Replication Done At: Fri May 06 15:41:01 EDT 2011 Config Files Replicated At: null Config Files Replicated: null Times Config Files Replicated Since Startup: null Next Replication Cycle At: Fri May 06 15:44:00 EDT 2011 Current Replication Status Start Time: Fri May 06 15:41:00 EDT 2011 Files Downloaded: 43 / 197 Downloaded: 477.08 KB / 588.82 MB [0.0%] Downloading File: _hdm.prx, Downloaded: 9.3 KB / 9.3 KB [100.0%] Time Elapsed: 967s, Estimated Time Remaining: 1221166s, Speed: 505 bytes/s Ravi Kiran Bhaskar
