Hi Erik,

Thanks for your quick reply. It's given me a few things to research and work on 
tomorrow.

In the meantime, in case it triggers any other thoughts,  just to say that our 
AutoCommit settings are
<autoCommit>
            <maxDocs>1800000</maxDocs>
            <maxTime>300000</maxTime>
        </autoCommit>
        <autoSoftCommit>
            <maxTime>10000</maxTime>
        </autoSoftCommit>

When I ingest data I don't see data appearing on the follower in the log.

It really seems like the data isn't being sent from the leader. As I said it 
could easily be something stupid that I've done along the way but I can't see 
what it is.

Thanks again,
Peter.

-----Original Message-----
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: 08 August 2017 18:23
To: solr-user <solr-user@lucene.apache.org>
Subject: Re: SolrCloud - leader updates not updating followers

This _better_ be a problem with your configuration or all my assumptions are 
false ;)

What are you autocommit settings? The documents should be forwarded to each 
replica from the leader during ingestion. However, they are not visible on the 
follower until a hard commit(openSearcher=true) or soft commit is triggered. 
Long blog on all this here:

https://lucidworks.com/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

An easy way to check that docs are being sent to the follower is to tail 
solr.log and send a doc to the leader. You should see the doc arrive at the 
follower. If you do see that then it's your autocommit settings.

NOTE: Solr promises eventual consistency. Due to the fact that autocommits will 
execute at different wall-clock times on the leaders and followers you can be 
out of sync by up to your autocommit interval.

You can force a commit by a URL like
"http://solr:port/solr/collection/update?commit=true"; that will commit on all 
replicas in a collection that may also be useful for seeing if it's an 
autocommit issue.

Best,
Erick

On Tue, Aug 8, 2017 at 9:49 AM, lancasp22 <peter.lancas...@findmypast.com> 
wrote:
> Hi,
>
> I've recently created a solr cloud on solr 5.5.2 with a separate
> zookeeper cluster. I write to the cloud by posting to update/json and
> the documents appear fine in the leader.
>
> The problem I have is that new documents added to the cloud aren't
> then being automatically applied to the followers of each leader. If I
> query a core then I can get different counts depending on which
> version of the core the query ran over and the solr admin statistics
> page confirms that the followers have fewer documents and are behind the 
> leader.
>
> If I restart the solr core for a follower, it does recover quickly and
> brings itself up-to-date with the leader. Looking at the logs for the
> follower you see that on re-start it identifies the leader and gets
> the changes from that leader.
>
> I think when documents are added to the leader these should be pushed
> to the followers so maybe it's this push process that isn't being triggered.
>
> It’s quite possible that I’ve made a simple error in the set-up but
> I’m baffled as to what it is. Please can anyone advise on any
> configuration that I need to check that might be causing these symptoms.
>
> Thanks in anticipation,
> Peter Lancaster.
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-leader-updates-not-updati
> ng-followers-tp4349618.html Sent from the Solr - User mailing list
> archive at Nabble.com.
________________________________

This message is confidential and may contain privileged information. You should 
not disclose its contents to any other person. If you are not the intended 
recipient, please notify the sender named above immediately. It is expressly 
declared that this e-mail does not constitute nor form part of a contract or 
unilateral obligation. Opinions, conclusions and other information in this 
message that do not relate to the official business of findmypast shall be 
understood as neither given nor endorsed by it.
________________________________

__________________________________________________________________________

This email has been checked for virus and other malicious content prior to 
leaving our network.
__________________________________________________________________________

Reply via email to