Are you using Solr 4.1?
- Mark
On Mar 11, 2013, at 1:53 PM, Victor Ruiz <bik1...@gmail.com> wrote:
> Hi guys,
>
> I have a problem with Solr replication. I have 2 solr servers (Solr 4.0.0) 1
> master and 1 slave (8 processors,16GB RAM ,Ubuntu 11, ext3, each). In
> every server, there are 2 independent instances of solr running (I tried
> also multicore config, but having independent instances has for me better
> performance), every instance having a differente collection. So, we have 2
> masters in server 1, and 2 slaves in server 2.
>
> Index size is currently (for the biggest collection) around 17 million
> documents, with a total size near 12 GB. The files transferred every
> replication cycle are typically not more than 100, with a total size not
> bigger than 50MB. The other collection is not that big, just around 1
> million docs and not bigger than 2 GB and not a high update ratio. The big
> collection has a load around 200 queries per second (MoreLikeThis,
> RealTimeGetHandler , TermVectorComponent mainly), and for the small one it
> is below 50 queries per second
>
> Replication has been working for long time with any problem, but in the last
> weeks the replication cycles started to take long and long time for the big
> collection, even more than 2 minutes, some times even more. During that
> time, slaves are so overloaded, that many queries are timing out, despite
> the timeout in my clients is 30 seconds.
>
> The servers are in same LAN, gigabit ethernet, so the broadband should not
> be the bottleneck.
>
> Since the index is receiving frequents updates and deletes (update handler
> receives more than 200 request per second for the big collection, but not
> more than 5 per second for the small one), I tried to use the
> maxCommitsToKeep attribute, to ensure that no file was deleted during
> replication, but it has no effect.
>
> My solrconfig.xml in the big collection is like that:
>
> <?xml version="1.0" encoding="UTF-8" ?>
>
> <config>
>
> <luceneMatchVersion>LUCENE_40</luceneMatchVersion>
>
> <directoryFactory name="DirectoryFactory"
>
> class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>
>
> <indexConfig>
> <mergeFactor>3</mergeFactor>
>
> <deletionPolicy class="solr.SolrDeletionPolicy">
>
> <str name="maxCommitsToKeep">10</str>
> <str name="maxOptimizedCommitsToKeep">1</str>
>
> <str name="maxCommitAge">6HOUR</str>
>
> </deletionPolicy>
>
> </indexConfig>
>
> <jmx/>
>
> <updateHandler class="solr.DirectUpdateHandler2">
>
> <autoCommit>
> <maxDocs>2000</maxDocs>
> <maxTime>30000</maxTime>
> </autoCommit>
>
> <autoSoftCommit>
> <maxTime>500</maxTime>
> </autoSoftCommit>
>
> <updateLog>
> <str name="dir">${solr.data.dir:}</str>
> </updateLog>
>
> </updateHandler>
>
> <query>
> <maxBooleanClauses>2048</maxBooleanClauses>
>
> <filterCache
> class="solr.FastLRUCache"
> size="2048"
> initialSize="1024"
> autowarmCount="1024"/>
>
> <queryResultCache
> class="solr.LRUCache"
> size="2048"
> initialSize="1024"
> autowarmCount="1024"/>
>
>
> <documentCache
> class="solr.LRUCache"
> size="2048"
> initialSize="1024"
> autowarmCount="1024"/>
>
> <enableLazyFieldLoading>true</enableLazyFieldLoading>
>
> <queryResultWindowSize>50</queryResultWindowSize>
>
> <queryResultMaxDocsCached>50</queryResultMaxDocsCached>
>
> <listener event="newSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst>
> <str name="q">*:*</str>
> <str name="fq">date:[NOW/DAY-7DAY TO
> NOW/DAY+1DAY]</str>
> <str name="rows">1000</str>
> </lst>
> </arr>
> </listener>
> <listener event="firstSearcher"
> class="solr.QuerySenderListener">
> <arr name="queries">
> <lst>
> <str name="q">*:*</str>
> <str name="fq">date:[NOW/DAY-7DAY TO
> NOW/DAY+1DAY]</str>
> <str name="rows">1000</str>
> </lst>
> </arr>
> </listener>
>
> <useColdSearcher>true</useColdSearcher>
>
> <maxWarmingSearchers>4</maxWarmingSearchers>
> </query>
>
>
>
> <requestHandler name="/replication" class="solr.ReplicationHandler">
> <lst name="master">
> <str name="enable">${enable.master:false}</str>
> <str name="replicateAfter">commit</str>
> <str name="replicateAfter">startup</str>
> <str name="backupAfter">startup</str>
> <str
> name="confFiles">schema.xml,solrconfig.xml,stopwords_de.txt,stopwords_en.txt,mapping-FoldToASCII.txt,mapping-FoldToASCII_de.txt</str>
> </lst>
> <lst name="slave">
> <str name="enable">${enable.slave:false}</str>
> <str
> name="masterUrl">http://${MASTER_HOST}:${MASTER_PORT}/solr/${MASTER_CORE}</str>
> <str name="pollInterval">05:00</str>
> <str name="httpBasicAuthUser">${MASTER_HTTP_USER}</str>
> <str
> name="httpBasicAuthPassword">${MASTER_HTTP_PWD}</str>
> </lst>
> </requestHandler>
>
> <admin>
> <defaultQuery>*:*</defaultQuery>
> </admin>
>
> </config>
>
> Poll interval is now set to 5 min, I tried to reduce it to 2 min and to
> increase it up to 10, with no effect, the replication is always taking so
> long., even with a poll time of 2 minutes, when there are only a few megas
> to replicate.
>
> Any idea suggestion about what could be the problem?
>
> Thanks in advance,
> Victor
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-replication-takes-long-time-tp4046388.html
> Sent from the Solr - User mailing list archive at Nabble.com.
