Okay - yes, 4.0 is a better choice for replication than 4.1.
It almost sounds like you may be replicating the full index rather than just
changes or something. 4.0 had a couple issues as well - a couple things that
were discovered while writing stronger tests for 4.2.
4.2 is spreading onto mirrors now.
- Mark
On Mar 11, 2013, at 2:00 PM, Victor Ruiz <bik1...@gmail.com> wrote:
> no, Solr 4.0.0, I wanted to update to Solr 4.1 but I read that there was an
> issue with the replication, so I decided not to try it for now
>
>
> Mark Miller-3 wrote
>> Are you using Solr 4.1?
>>
>> - Mark
>>
>> On Mar 11, 2013, at 1:53 PM, Victor Ruiz <
>
>> bik1979@
>
>> > wrote:
>>
>>> Hi guys,
>>>
>>> I have a problem with Solr replication. I have 2 solr servers (Solr
>>> 4.0.0) 1
>>> master and 1 slave (8 processors,16GB RAM ,Ubuntu 11, ext3, each). In
>>> every server, there are 2 independent instances of solr running (I tried
>>> also multicore config, but having independent instances has for me better
>>> performance), every instance having a differente collection. So, we have
>>> 2
>>> masters in server 1, and 2 slaves in server 2.
>>>
>>> Index size is currently (for the biggest collection) around 17 million
>>> documents, with a total size near 12 GB. The files transferred every
>>> replication cycle are typically not more than 100, with a total size not
>>> bigger than 50MB. The other collection is not that big, just around 1
>>> million docs and not bigger than 2 GB and not a high update ratio. The
>>> big
>>> collection has a load around 200 queries per second (MoreLikeThis,
>>> RealTimeGetHandler , TermVectorComponent mainly), and for the small one
>>> it
>>> is below 50 queries per second
>>>
>>> Replication has been working for long time with any problem, but in the
>>> last
>>> weeks the replication cycles started to take long and long time for the
>>> big
>>> collection, even more than 2 minutes, some times even more. During that
>>> time, slaves are so overloaded, that many queries are timing out, despite
>>> the timeout in my clients is 30 seconds.
>>>
>>> The servers are in same LAN, gigabit ethernet, so the broadband should
>>> not
>>> be the bottleneck.
>>>
>>> Since the index is receiving frequents updates and deletes (update
>>> handler
>>> receives more than 200 request per second for the big collection, but not
>>> more than 5 per second for the small one), I tried to use the
>>> maxCommitsToKeep attribute, to ensure that no file was deleted during
>>> replication, but it has no effect.
>>>
>>> My solrconfig.xml in the big collection is like that:
>>>
>>> <?xml version="1.0" encoding="UTF-8" ?>
>>>
>>>
>> <config>
>>>
>>>
>> <luceneMatchVersion>
>> LUCENE_40
>> </luceneMatchVersion>
>>>
>>>
>> <directoryFactory name="DirectoryFactory"
>>>
>>
>> class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>>>
>>>
>>>
>> <indexConfig>
>>>
>> <mergeFactor>
>> 3
>> </mergeFactor>
>>>
>>>
>> <deletionPolicy class="solr.SolrDeletionPolicy">
>>>
>>>
>> <str name="maxCommitsToKeep">
>> 10
>> </str>
>>>
>> <str name="maxOptimizedCommitsToKeep">
>> 1
>> </str>
>>>
>>>
>> <str name="maxCommitAge">
>> 6HOUR
>> </str>
>>>
>>>
>> </deletionPolicy>
>>>
>>>
>> </indexConfig>
>>>
>>>
>> <jmx/>
>>>
>>>
>> <updateHandler class="solr.DirectUpdateHandler2">
>>>
>>>
>> <autoCommit>
>>>
>> <maxDocs>
>> 2000
>> </maxDocs>
>>>
>> <maxTime>
>> 30000
>> </maxTime>
>>>
>> </autoCommit>
>>>
>>>
>> <autoSoftCommit>
>>>
>> <maxTime>
>> 500
>> </maxTime>
>>>
>> </autoSoftCommit>
>>>
>>>
>> <updateLog>
>>>
>> <str name="dir">
>> ${solr.data.dir:}
>> </str>
>>>
>> </updateLog>
>>>
>>>
>> </updateHandler>
>>>
>>>
>> <query>
>>>
>> <maxBooleanClauses>
>> 2048
>> </maxBooleanClauses>
>>>
>>>
>> <filterCache
>>>
>> class="solr.FastLRUCache"
>>> size="2048"
>>> initialSize="1024"
>>> autowarmCount="1024"/>
>>>
>>>
>> <queryResultCache
>>>
>> class="solr.LRUCache"
>>> size="2048"
>>> initialSize="1024"
>>> autowarmCount="1024"/>
>>>
>>>
>>>
>> <documentCache
>>>
>> class="solr.LRUCache"
>>> size="2048"
>>> initialSize="1024"
>>> autowarmCount="1024"/>
>>>
>>>
>> <enableLazyFieldLoading>
>> true
>> </enableLazyFieldLoading>
>>>
>>>
>> <queryResultWindowSize>
>> 50
>> </queryResultWindowSize>
>>>
>>>
>> <queryResultMaxDocsCached>
>> 50
>> </queryResultMaxDocsCached>
>>>
>>>
>> <listener event="newSearcher" class="solr.QuerySenderListener">
>>>
>> <arr name="queries">
>>>
>> <lst>
>>>
>> <str name="q">
>> *:*
>> </str>
>>>
>> <str name="fq">
>> date:[NOW/DAY-7DAY TO NOW/DAY+1DAY]
>> </str>
>>>
>> <str name="rows">
>> 1000
>> </str>
>>>
>> </lst>
>>>
>> </arr>
>>>
>> </listener>
>>>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>>>
>> <arr name="queries">
>>>
>> <lst>
>>>
>> <str name="q">
>> *:*
>> </str>
>>>
>> <str name="fq">
>> date:[NOW/DAY-7DAY TO NOW/DAY+1DAY]
>> </str>
>>>
>> <str name="rows">
>> 1000
>> </str>
>>>
>> </lst>
>>>
>> </arr>
>>>
>> </listener>
>>>
>>>
>> <useColdSearcher>
>> true
>> </useColdSearcher>
>>>
>>>
>> <maxWarmingSearchers>
>> 4
>> </maxWarmingSearchers>
>>>
>> </query>
>>>
>>>
>>>
>>>
>> <requestHandler name="/replication" class="solr.ReplicationHandler">
>>>
>> <lst name="master">
>>>
>> <str name="enable">
>> ${enable.master:false}
>> </str>
>>>
>> <str name="replicateAfter">
>> commit
>> </str>
>>>
>> <str name="replicateAfter">
>> startup
>> </str>
>>>
>> <str name="backupAfter">
>> startup
>> </str>
>>>
>> <str
>>>
>>
>> name="confFiles">schema.xml,solrconfig.xml,stopwords_de.txt,stopwords_en.txt,mapping-FoldToASCII.txt,mapping-FoldToASCII_de.txt
>> </str>
>>>
>> </lst>
>>>
>> <lst name="slave">
>>>
>> <str name="enable">
>> ${enable.slave:false}
>> </str>
>>>
>> <str
>>>
>> name="masterUrl">http://${MASTER_HOST}:${MASTER_PORT}/solr/${MASTER_CORE}
>> </str>
>>>
>> <str name="pollInterval">
>> 05:00
>> </str>
>>>
>> <str name="httpBasicAuthUser">
>> ${MASTER_HTTP_USER}
>> </str>
>>>
>> <str name="httpBasicAuthPassword">
>> ${MASTER_HTTP_PWD}
>> </str>
>>>
>> </lst>
>>>
>> </requestHandler>
>>>
>>>
>> <admin>
>>>
>> <defaultQuery>
>> *:*
>> </defaultQuery>
>>>
>> </admin>
>>>
>>>
>> </config>
>>>
>>> Poll interval is now set to 5 min, I tried to reduce it to 2 min and to
>>> increase it up to 10, with no effect, the replication is always taking so
>>> long., even with a poll time of 2 minutes, when there are only a few
>>> megas
>>> to replicate.
>>>
>>> Any idea suggestion about what could be the problem?
>>>
>>> Thanks in advance,
>>> Victor
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://lucene.472066.n3.nabble.com/Solr-replication-takes-long-time-tp4046388.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-replication-takes-long-time-tp4046388p4046392.html
> Sent from the Solr - User mailing list archive at Nabble.com.
