Btw, confirmed that this doesn't happen on our development stage with 3.6.
On Wed, May 16, 2012 at 3:59 PM, Jasper Floor <jasper.fl...@m4n.nl> wrote:
> The slave index does indeed grow over a period of time regardless of
> restarts. We do run on 1.4 however. We will be updating to 3.6 very
> soon however so I will see how that works out. Actually we should be
> able to see this on our staging platform.
>
> thanks everyone.
>
> mvg,
> Jasper
>
> On Mon, May 14, 2012 at 4:40 PM, Bill Bell <billnb...@gmail.com> wrote:
>> This is a known issue in 1.4 especially in Windows. Some of it was resolved
>> in 3x.
>>
>> Bill Bell
>> Sent from mobile
>>
>>
>> On May 14, 2012, at 5:54 AM, Erick Erickson <erickerick...@gmail.com> wrote:
>>
>>> Hmmm, replication will require up to twice the space of the
>>> index _temporarily_, just checking if that's what you're seeing....
>>> But that should go away reasonably soon. Out of curiosity, what
>>> happens if you restart your server, do the extra files go away?
>>>
>>> But it sounds like your index is growing over a longer period of time
>>> than just a single replication, is that true?
>>>
>>> Best
>>> Erick
>>>
>>> On Fri, May 11, 2012 at 6:03 AM, Jasper Floor <jasper.fl...@m4n.nl> wrote:
>>>> Hi,
>>>>
>>>> On Thu, May 10, 2012 at 5:59 PM, Otis Gospodnetic
>>>> <otis_gospodne...@yahoo.com> wrote:
>>>>> Hi Jasper,
>>>>
>>>> Sorry, I should've added more technical info wihtout being prompted.
>>>>
>>>>> Solr does handle that for you. Some more stuff to share:
>>>>>
>>>>> * Solr version?
>>>>
>>>> 1.4
>>>>
>>>>> * JVM version?
>>>> 1.7 update 2
>>>>
>>>>> * OS?
>>>> Debian (2.6.32-5-xen-amd64)
>>>>
>>>>> * Java replication?
>>>> yes
>>>>
>>>>> * Errors in Solr logs?
>>>> no
>>>>
>>>>> * deletion policy section in solrconfig.xml?
>>>> missing I would say, but I don't see this on the replication wiki page.
>>>>
>>>> This is what we have configured for replication:
>>>>
>>>> <requestHandler name="/replication" class="solr.ReplicationHandler" >
>>>> <lst name="slave">
>>>>
>>>> <str
>>>> name="masterUrl">${solr.master.url}/df-stream-store/replication</str>
>>>>
>>>> <str name="pollInterval">00:20:00</str>
>>>> <str name="compression">internal</str>
>>>> <str name="httpConnTimeout">5000</str>
>>>> <str name="httpReadTimeout">10000</str>
>>>>
>>>> </lst>
>>>> </requestHandler>
>>>>
>>>> We will be updating to 3.6 fairly soon however. To be honest, from
>>>> what I've read, the Solr cloud is what we really want in the future
>>>> but we will have to be patient for that.
>>>>
>>>> thanks in advance
>>>>
>>>> mvg,
>>>> Jasper
>>>>
>>>>> You may also want to look at your Index report in SPM
>>>>> (http://sematext.com/spm) before/during/after replication and share what
>>>>> you see.
>>>>>
>>>>> Otis
>>>>> ----
>>>>> Performance Monitoring for Solr / ElasticSearch / HBase -
>>>>> http://sematext.com/spm
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: Jasper Floor <jasper.fl...@m4n.nl>
>>>>>> To: solr-user@lucene.apache.org
>>>>>> Cc:
>>>>>> Sent: Thursday, May 10, 2012 9:08 AM
>>>>>> Subject: slave index not cleaned
>>>>>>
>>>>>> Perhaps I am missing the obvious but our slaves tend to run out of
>>>>>> disk space. The index sizes grow to multiple times the size of the
>>>>>> master. So I just toss all the data and trigger a replication.
>>>>>> However, can't solr handle this for me?
>>>>>>
>>>>>> I'm sorry if I've missed a simple setting which does this for me, but
>>>>>> if its there then I have missed it.
>>>>>>
>>>>>> mvg
>>>>>> Jasper
>>>>>>
