Re: Add me in solr mailing list

[Pritam Kute]

Hi Santosh Kumar,

You need to subscribe yourself to Solr mailing list. For more details refer
"Mailing List and IRC" section here:
http://lucene.apache.org/solr/community.html

Hope this helps you.

Regards
--
Pritam Kute

On Tue, Aug 1, 2017 at 5:38 PM, Santosh Kumar 
wrote:

> Hi,
> Please add me in solr mailing list.
>
>
> Thanks,
> Santosh
>

Solr 4.10.4 export handler NPE

[Lasitha Wattaladeniya]

Hi devs,

I was exploring the /export handler in solr and got an exception. When I
research online I found this open jira case : SOLR-8860

https://issues.apache.org/jira/plugins/servlet/mobile#issue/SOLR-8806

is this a valid jira case? Any workarounds?

Jira says affect version is 5.5 but I'm getting this in 4.10.4 also


Regards,
Lasitha

Re: Move index directory to another partition

[Dave]

To add to this, not sure of solr cloud uses it, but you're going to want to 
destroy the wrote.lock file as well

> On Aug 1, 2017, at 9:31 PM, Shawn Heisey  wrote:
> 
>> On 8/1/2017 7:09 PM, Erick Erickson wrote:
>> WARNING: what I currently understand about the limitations of AWS
>> could fill volumes so I might be completely out to lunch.
>> 
>> If you ADDREPLICA with the new replica's  data residing on the new EBS
>> volume, then wait for it to sync (which it'll do all by itself) then
>> DELETEREPLICA on the original you'll be all set.
>> 
>> In recent Solr's, theres also the MOVENODE collections API call.
> 
> I did consider mentioning that as a possible way forward, but I hate to
> rely on special configurations with core.properties, particularly if the
> newly built replica core instanceDirs aren't in the solr home (or
> coreRootDirectory) at all.  I didn't want to try and explain the precise
> steps required to get that plan to work.  I would expect to need some
> arcane Collections API work or manual ZK modification to reach a correct
> state -- steps that would be prone to error.
> 
> The idea I mentioned seemed to me to be the way forward that would
> require the least specialized knowledge.  Here's a simplified stating of
> the steps:
> 
> * Mount the new volume somewhere.
> * Use multiple rsync passes to get the data copied.
> * Stop Solr.
> * Do a final rsync pass.
> * Unmount the original volume.
> * Remount the new volume in the original location.
> * Start Solr.
> 
> Thanks,
> Shawn
> 

Re: Move index directory to another partition

[Shawn Heisey]

On 8/1/2017 7:09 PM, Erick Erickson wrote:
> WARNING: what I currently understand about the limitations of AWS
> could fill volumes so I might be completely out to lunch.
>
> If you ADDREPLICA with the new replica's  data residing on the new EBS
> volume, then wait for it to sync (which it'll do all by itself) then
> DELETEREPLICA on the original you'll be all set.
>
> In recent Solr's, theres also the MOVENODE collections API call.

I did consider mentioning that as a possible way forward, but I hate to
rely on special configurations with core.properties, particularly if the
newly built replica core instanceDirs aren't in the solr home (or
coreRootDirectory) at all.  I didn't want to try and explain the precise
steps required to get that plan to work.  I would expect to need some
arcane Collections API work or manual ZK modification to reach a correct
state -- steps that would be prone to error.

The idea I mentioned seemed to me to be the way forward that would
require the least specialized knowledge.  Here's a simplified stating of
the steps:

* Mount the new volume somewhere.
* Use multiple rsync passes to get the data copied.
* Stop Solr.
* Do a final rsync pass.
* Unmount the original volume.
* Remount the new volume in the original location.
* Start Solr.

Thanks,
Shawn

Re: Replication Question

[Erick Erickson]

And please do not use optimize unless your index is
totally static. I only recommend it when the pattern is
to update the index periodically, like every day or
something and not update any docs in between times.

Implied in Shawn's e-mail was that you should undo
anything you've done in terms of configuring replication,
just go with the defaults.

Finally, my bet is that your problematic Solr node is misconfigured.

Best,
Erick

On Tue, Aug 1, 2017 at 2:36 PM, Shawn Heisey  wrote:
> On 8/1/2017 12:09 PM, Michael B. Klein wrote:
>> I have a 3-node solrcloud cluster orchestrated by zookeeper. Most stuff
>> seems to be working OK, except that one of the nodes never seems to get its
>> replica updated.
>>
>> Queries take place through a non-caching, round-robin load balancer. The
>> collection looks fine, with one shard and a replicationFactor of 3.
>> Everything in the cloud diagram is green.
>>
>> But if I (for example) select?q=id:hd76s004z, the results come up empty 1
>> out of every 3 times.
>>
>> Even several minutes after a commit and optimize, one replica still isn’t
>> returning the right info.
>>
>> Do I need to configure my `solrconfig.xml` with `replicateAfter` options on
>> the `/replication` requestHandler, or is that a non-solrcloud,
>> standalone-replication thing?
>
> This is one of the more confusing aspects of SolrCloud.
>
> When everything is working perfectly in a SolrCloud install, the feature
> in Solr called "replication" is *never* used.  SolrCloud does require
> the replication feature, though ... which is what makes this whole thing
> very confusing.
>
> Replication is used to replicate an entire Lucene index (consisting of a
> bunch of files on the disk) from a core on a master server to a core on
> a slave server.  This is how replication was done before SolrCloud was
> created.
>
> The way that SolrCloud keeps replicas in sync is *entirely* different.
> SolrCloud has no masters and no slaves.  When you index or delete a
> document in a SolrCloud collection, the request is forwarded to the
> leader of the correct shard for that document.  The leader then sends a
> copy of that request to all the other replicas, and each replica
> (including the leader) independently handles the updates that are in the
> request.  Since all replicas index the same content, they stay in sync.
>
> What SolrCloud does with the replication feature is index recovery.  In
> some situations recovery can be done from the leader's transaction log,
> but when a replica has gotten so far out of sync that the only option
> available is to completely replace the index on the bad replica,
> SolrCloud will fire up the replication feature and create an exact copy
> of the index from the replica that is currently elected as leader.
> SolrCloud temporarily designates the leader core as master and the bad
> replica as slave, then initiates a one-time replication.  This is all
> completely automated and requires no configuration or input from the
> administrator.
>
> The configuration elements you have asked about are for the old
> master-slave replication setup and do not apply to SolrCloud at all.
>
> What I would recommend that you do to solve your immediate issue:  Shut
> down the Solr instance that is having the problem, rename the "data"
> directory in the core that isn't working right to something else, and
> start Solr back up.  As long as you still have at least one good replica
> in the cloud, SolrCloud will see that the index data is gone and copy
> the index from the leader.  You could delete the data directory instead
> of renaming it, but that would leave you with no "undo" option.
>
> Thanks,
> Shawn
>

Re: Move index directory to another partition

[Erick Erickson]

WARNING: what I currently understand about the limitations of AWS
could fill volumes so I might be completely out to lunch.

If you ADDREPLICA with the new replica's  data residing on the new EBS
volume, then wait for it to sync (which it'll do all by itself) then
DELETEREPLICA on the original you'll be all set.

In recent Solr's, theres also the MOVENODE collections API call.

Best,
Erick

On Tue, Aug 1, 2017 at 6:03 PM, Shawn Heisey  wrote:
> On 8/1/2017 4:00 PM, Mahmoud Almokadem wrote:
>> I'm using ubuntu and I'll try rsync command. Unfortunately I'm using one
>> replication factor but I think the downtime will be less than five minutes 
>> after following your steps.
>>
>> But how can I start Solr backup or why should I run it although I copied
>> the index and changed theo path?
>>
>> And what do you mean with "Using multiple passes with rsync"?
>
> The first time you copy the data, which you could do with cp if you
> want, the time required will be limited by the size of the data and the
> speed of the disks.  Depending on the size, it could take several hours
> like you estimated.  I would suggest using rsync for the first copy just
> because you're going to need the same command again for the later passes.
>
> Doing a second pass with rsync should go very quickly.  How fast would
> depend on the rate that the index data is changing.  You might need to
> do this step more than once just so that it gets faster each time, in
> preparation for the final pass.
>
> A final pass with rsync might only take a few seconds, and if Solr is
> stopped before that final copy is started, then there's no way the index
> data can change.
>
> Thanks,
> Shawn
>

Re: Inconsistency in results between replicas using CloudSolrClient

[Erick Erickson]

re: automated tests. In the Solr JUnit tests you'll see this pattern

- create collection
- add a bunch of docs
- do a commit
- now test consistency

Which, of course, doesn't particularly help if you're indexing after the commit.

For things like soft commit tests special care is taken to allow the
autocommit interval to expire.

bq: but not sure I like the performance implications

Totally agree. H, maybe commit every X amount of time from the
client?. Committing from the client is, BTW, something I loathe
but I suppose one could write a cron job that issued a hard commit
or a soft commit to the collection rather than relying on autocommit
settings, then set those very high in solrconfig for anything that
opens a searcher. If you do anything like that I'd still do a hard
commit for HA/DR reasons from solrconfig.xml but with
openSearcher=false.

Or, just wait until anyone really notices. All I can say is that it
hasn't been a big enough problem to garner the effort to push the JIRA
I mentioned over the finish line.

Best,
Erick

On Tue, Aug 1, 2017 at 3:18 PM, Chris Troullis  wrote:
> Thanks for the reply Erick, I feared that would be the case. Interesting
> idea with using the fq but not sure I like the performance implications. I
> will see how big of a deal it will be in practice, I was just thinking
> about this as a hypothetical scenario today, and as you said, we have a lot
> of automated tests so I anticipate this likely causing issues. I'll give it
> some more thought and see if I can come up with any other workarounds.
>
> -Chris
>
> On Tue, Aug 1, 2017 at 5:38 PM, Erick Erickson 
> wrote:
>
>> You're understanding is correct.
>>
>> As for how people cope? Mostly they ignore it. The actual number of
>> times people notice this is usually quite small, mostly it surfaces
>> when automated test suites are run.
>>
>> If you must lock this up, and you can stand the latency you could add
>> a timestamp for each document and auto-add an FQ clause like:
>> fq=timestamp:[* TO NOW-soft_commit_interval_plus_some_windage]
>>
>> Note, though, that this not an fq clause that can be re-used, see:
>> https://lucidworks.com/2012/02/23/date-math-now-and-filter-queries/ so
>> either it'd be something like:
>> fq=timestamp:[* TO NOW/MINUTE-soft_commit_interval_plus_some_windage]
>> or
>> fq=timestamp:{!cache=false}[* TO NOW-soft_commit_interval_plus_
>> some_windage]
>>
>> and would inevitably make the latency between when something was
>> indexed and available for search longer.
>>
>> You can also reduce your soft commit interval to something short, but
>> that has other problems.
>>
>> see: SOLR-6606, but it looks like other priorities have gotten in the
>> way of it being committed.
>>
>> Best,
>> Erick
>>
>> On Tue, Aug 1, 2017 at 1:50 PM, Chris Troullis 
>> wrote:
>> > Hi,
>> >
>> > I think I know the answer to this question, but just wanted to verify/see
>> > what other people do to address this concern.
>> >
>> > I have a Solr Cloud setup (6.6.0) with 2 nodes, 1 collection with 1 shard
>> > and 2 replicas (1 replica per node). The nature of my use case requires
>> > frequent updates to Solr, and documents are being added constantly
>> > throughout the day. I am using CloudSolrClient via SolrJ to query my
>> > collection and load balance across my 2 replicas.
>> >
>> > Here's my question:
>> >
>> > As I understand it, because of the nature of Solr Cloud (eventual
>> > consistency), and the fact that the soft commit timings on the 2 replicas
>> > will not necessarily be in sync, would it not be possible to run into a
>> > scenario where, say a document gets indexed on replica 1 right before a
>> > soft commit, but indexed on replica 2 right after a soft commit? In this
>> > scenario, using the load balanced CloudSolrClient, wouldn't it be
>> possible
>> > for a user to do a search, see the newly added document because they got
>> > sent to replica 1, and then search again, and the newly added document
>> > would disappear from their results since they got sent to replica 2 and
>> the
>> > soft commit hasn't happened yet?
>> >
>> > If so, how do people typically handle this scenario in NRT search cases?
>> It
>> > seems like a poor user experience if things keep disappearing and
>> > reappearing from their search results randomly. Currently the only
>> thought
>> > I have to prevent this is to write (or extend) my own solr client to
>> stick
>> > a user's session to a specific replica (unless it goes down), but still
>> > load balance users between the replicas. But of course then I have to
>> > manage all of the things CloudSolrClient manages manually re: cluster
>> > state, etc.
>> >
>> > Can anyone confirm/deny my understanding of how this works/offer any
>> > suggestions to eliminate the scenario in question from occurring?
>> >
>> > Thanks,
>> >
>> > Chris
>>

Re: Move index directory to another partition

[Shawn Heisey]

On 8/1/2017 4:00 PM, Mahmoud Almokadem wrote:
> I'm using ubuntu and I'll try rsync command. Unfortunately I'm using one
> replication factor but I think the downtime will be less than five minutes 
> after following your steps.
>
> But how can I start Solr backup or why should I run it although I copied
> the index and changed theo path?
>
> And what do you mean with "Using multiple passes with rsync"?

The first time you copy the data, which you could do with cp if you
want, the time required will be limited by the size of the data and the
speed of the disks.  Depending on the size, it could take several hours
like you estimated.  I would suggest using rsync for the first copy just
because you're going to need the same command again for the later passes.

Doing a second pass with rsync should go very quickly.  How fast would
depend on the rate that the index data is changing.  You might need to
do this step more than once just so that it gets faster each time, in
preparation for the final pass.

A final pass with rsync might only take a few seconds, and if Solr is
stopped before that final copy is started, then there's no way the index
data can change.

Thanks,
Shawn

Re: Inconsistency in results between replicas using CloudSolrClient

[Chris Troullis]

Thanks for the reply Erick, I feared that would be the case. Interesting
idea with using the fq but not sure I like the performance implications. I
will see how big of a deal it will be in practice, I was just thinking
about this as a hypothetical scenario today, and as you said, we have a lot
of automated tests so I anticipate this likely causing issues. I'll give it
some more thought and see if I can come up with any other workarounds.

-Chris

On Tue, Aug 1, 2017 at 5:38 PM, Erick Erickson 
wrote:

> You're understanding is correct.
>
> As for how people cope? Mostly they ignore it. The actual number of
> times people notice this is usually quite small, mostly it surfaces
> when automated test suites are run.
>
> If you must lock this up, and you can stand the latency you could add
> a timestamp for each document and auto-add an FQ clause like:
> fq=timestamp:[* TO NOW-soft_commit_interval_plus_some_windage]
>
> Note, though, that this not an fq clause that can be re-used, see:
> https://lucidworks.com/2012/02/23/date-math-now-and-filter-queries/ so
> either it'd be something like:
> fq=timestamp:[* TO NOW/MINUTE-soft_commit_interval_plus_some_windage]
> or
> fq=timestamp:{!cache=false}[* TO NOW-soft_commit_interval_plus_
> some_windage]
>
> and would inevitably make the latency between when something was
> indexed and available for search longer.
>
> You can also reduce your soft commit interval to something short, but
> that has other problems.
>
> see: SOLR-6606, but it looks like other priorities have gotten in the
> way of it being committed.
>
> Best,
> Erick
>
> On Tue, Aug 1, 2017 at 1:50 PM, Chris Troullis 
> wrote:
> > Hi,
> >
> > I think I know the answer to this question, but just wanted to verify/see
> > what other people do to address this concern.
> >
> > I have a Solr Cloud setup (6.6.0) with 2 nodes, 1 collection with 1 shard
> > and 2 replicas (1 replica per node). The nature of my use case requires
> > frequent updates to Solr, and documents are being added constantly
> > throughout the day. I am using CloudSolrClient via SolrJ to query my
> > collection and load balance across my 2 replicas.
> >
> > Here's my question:
> >
> > As I understand it, because of the nature of Solr Cloud (eventual
> > consistency), and the fact that the soft commit timings on the 2 replicas
> > will not necessarily be in sync, would it not be possible to run into a
> > scenario where, say a document gets indexed on replica 1 right before a
> > soft commit, but indexed on replica 2 right after a soft commit? In this
> > scenario, using the load balanced CloudSolrClient, wouldn't it be
> possible
> > for a user to do a search, see the newly added document because they got
> > sent to replica 1, and then search again, and the newly added document
> > would disappear from their results since they got sent to replica 2 and
> the
> > soft commit hasn't happened yet?
> >
> > If so, how do people typically handle this scenario in NRT search cases?
> It
> > seems like a poor user experience if things keep disappearing and
> > reappearing from their search results randomly. Currently the only
> thought
> > I have to prevent this is to write (or extend) my own solr client to
> stick
> > a user's session to a specific replica (unless it goes down), but still
> > load balance users between the replicas. But of course then I have to
> > manage all of the things CloudSolrClient manages manually re: cluster
> > state, etc.
> >
> > Can anyone confirm/deny my understanding of how this works/offer any
> > suggestions to eliminate the scenario in question from occurring?
> >
> > Thanks,
> >
> > Chris
>

Re: Move index directory to another partition

[Mahmoud Almokadem]

Thanks Shawn,

I'm using ubuntu and I'll try rsync command. Unfortunately I'm using one
replication factor but I think the downtime will be less than five minutes
after following your steps.

But how can I start Solr backup or why should I run it although I copied
the index and changed theo path?

And what do you mean with "Using multiple passes with rsync"?

Thanks,
Mahmoud


On Tuesday, August 1, 2017, Shawn Heisey  wrote:

> On 7/31/2017 12:28 PM, Mahmoud Almokadem wrote:
> > I've a SolrCloud of four instances on Amazon and the EBS volumes that
> > contain the data on everynode is going to be full, unfortunately Amazon
> > doesn't support expanding the EBS. So, I'll attach larger EBS volumes to
> > move the index to.
> >
> > I can stop the updates on the index, but I'm afraid to use "cp" command
> to
> > copy the files that are "on merge" operation.
> >
> > The copy operation may take several  hours.
> >
> > How can I move the data directory without stopping the instance?
>
> Use rsync to do the copy.  Do an initial copy while Solr is running,
> then do a second copy, which should be pretty fast because rsync will
> see the data from the first copy.  Then shut Solr down and do a third
> rsync which will only copy a VERY small changeset.  Reconfigure Solr
> and/or the OS to use the new location, and start Solr back up.  Because
> you mentioned "cp" I am assuming that you're NOT on Windows, and that
> the OS will most likely allow you to do anything you need with index
> files while Solr has them open.
>
> If you have set up your replicas with SolrCloud properly, then your
> collections will not go offline when one Solr instance is shut down, and
> that instance will be brought back into sync with the rest of the
> cluster when it starts back up.  Using multiple passes with rsync should
> mean that Solr will not need to be shutdown for very long.
>
> The options I typically use for this kind of copy with rsync are "-avH
> --delete".  I would recommend that you research rsync options so that
> you fully understand what I have suggested.
>
> Thanks,
> Shawn
>
>

Re: Move index directory to another partition

[Walter Underwood]

Way back in the 1.x days, replication was done with shell scripts and rsync, 
right?

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)


> On Aug 1, 2017, at 2:45 PM, Shawn Heisey  wrote:
> 
> On 7/31/2017 12:28 PM, Mahmoud Almokadem wrote:
>> I've a SolrCloud of four instances on Amazon and the EBS volumes that
>> contain the data on everynode is going to be full, unfortunately Amazon
>> doesn't support expanding the EBS. So, I'll attach larger EBS volumes to
>> move the index to.
>> 
>> I can stop the updates on the index, but I'm afraid to use "cp" command to
>> copy the files that are "on merge" operation.
>> 
>> The copy operation may take several  hours.
>> 
>> How can I move the data directory without stopping the instance?
> 
> Use rsync to do the copy.  Do an initial copy while Solr is running,
> then do a second copy, which should be pretty fast because rsync will
> see the data from the first copy.  Then shut Solr down and do a third
> rsync which will only copy a VERY small changeset.  Reconfigure Solr
> and/or the OS to use the new location, and start Solr back up.  Because
> you mentioned "cp" I am assuming that you're NOT on Windows, and that
> the OS will most likely allow you to do anything you need with index
> files while Solr has them open.
> 
> If you have set up your replicas with SolrCloud properly, then your
> collections will not go offline when one Solr instance is shut down, and
> that instance will be brought back into sync with the rest of the
> cluster when it starts back up.  Using multiple passes with rsync should
> mean that Solr will not need to be shutdown for very long.
> 
> The options I typically use for this kind of copy with rsync are "-avH
> --delete".  I would recommend that you research rsync options so that
> you fully understand what I have suggested.
> 
> Thanks,
> Shawn
> 

Re: Move index directory to another partition

[Shawn Heisey]

On 7/31/2017 12:28 PM, Mahmoud Almokadem wrote:
> I've a SolrCloud of four instances on Amazon and the EBS volumes that
> contain the data on everynode is going to be full, unfortunately Amazon
> doesn't support expanding the EBS. So, I'll attach larger EBS volumes to
> move the index to.
>
> I can stop the updates on the index, but I'm afraid to use "cp" command to
> copy the files that are "on merge" operation.
>
> The copy operation may take several  hours.
>
> How can I move the data directory without stopping the instance?

Use rsync to do the copy.  Do an initial copy while Solr is running,
then do a second copy, which should be pretty fast because rsync will
see the data from the first copy.  Then shut Solr down and do a third
rsync which will only copy a VERY small changeset.  Reconfigure Solr
and/or the OS to use the new location, and start Solr back up.  Because
you mentioned "cp" I am assuming that you're NOT on Windows, and that
the OS will most likely allow you to do anything you need with index
files while Solr has them open.

If you have set up your replicas with SolrCloud properly, then your
collections will not go offline when one Solr instance is shut down, and
that instance will be brought back into sync with the rest of the
cluster when it starts back up.  Using multiple passes with rsync should
mean that Solr will not need to be shutdown for very long.

The options I typically use for this kind of copy with rsync are "-avH
--delete".  I would recommend that you research rsync options so that
you fully understand what I have suggested.

Thanks,
Shawn

Re: Inconsistency in results between replicas using CloudSolrClient

[Erick Erickson]

You're understanding is correct.

As for how people cope? Mostly they ignore it. The actual number of
times people notice this is usually quite small, mostly it surfaces
when automated test suites are run.

If you must lock this up, and you can stand the latency you could add
a timestamp for each document and auto-add an FQ clause like:
fq=timestamp:[* TO NOW-soft_commit_interval_plus_some_windage]

Note, though, that this not an fq clause that can be re-used, see:
https://lucidworks.com/2012/02/23/date-math-now-and-filter-queries/ so
either it'd be something like:
fq=timestamp:[* TO NOW/MINUTE-soft_commit_interval_plus_some_windage]
or
fq=timestamp:{!cache=false}[* TO NOW-soft_commit_interval_plus_some_windage]

and would inevitably make the latency between when something was
indexed and available for search longer.

You can also reduce your soft commit interval to something short, but
that has other problems.

see: SOLR-6606, but it looks like other priorities have gotten in the
way of it being committed.

Best,
Erick

On Tue, Aug 1, 2017 at 1:50 PM, Chris Troullis  wrote:
> Hi,
>
> I think I know the answer to this question, but just wanted to verify/see
> what other people do to address this concern.
>
> I have a Solr Cloud setup (6.6.0) with 2 nodes, 1 collection with 1 shard
> and 2 replicas (1 replica per node). The nature of my use case requires
> frequent updates to Solr, and documents are being added constantly
> throughout the day. I am using CloudSolrClient via SolrJ to query my
> collection and load balance across my 2 replicas.
>
> Here's my question:
>
> As I understand it, because of the nature of Solr Cloud (eventual
> consistency), and the fact that the soft commit timings on the 2 replicas
> will not necessarily be in sync, would it not be possible to run into a
> scenario where, say a document gets indexed on replica 1 right before a
> soft commit, but indexed on replica 2 right after a soft commit? In this
> scenario, using the load balanced CloudSolrClient, wouldn't it be possible
> for a user to do a search, see the newly added document because they got
> sent to replica 1, and then search again, and the newly added document
> would disappear from their results since they got sent to replica 2 and the
> soft commit hasn't happened yet?
>
> If so, how do people typically handle this scenario in NRT search cases? It
> seems like a poor user experience if things keep disappearing and
> reappearing from their search results randomly. Currently the only thought
> I have to prevent this is to write (or extend) my own solr client to stick
> a user's session to a specific replica (unless it goes down), but still
> load balance users between the replicas. But of course then I have to
> manage all of the things CloudSolrClient manages manually re: cluster
> state, etc.
>
> Can anyone confirm/deny my understanding of how this works/offer any
> suggestions to eliminate the scenario in question from occurring?
>
> Thanks,
>
> Chris

Re: Replication Question

[Shawn Heisey]

On 8/1/2017 12:09 PM, Michael B. Klein wrote:
> I have a 3-node solrcloud cluster orchestrated by zookeeper. Most stuff
> seems to be working OK, except that one of the nodes never seems to get its
> replica updated.
>
> Queries take place through a non-caching, round-robin load balancer. The
> collection looks fine, with one shard and a replicationFactor of 3.
> Everything in the cloud diagram is green.
>
> But if I (for example) select?q=id:hd76s004z, the results come up empty 1
> out of every 3 times.
>
> Even several minutes after a commit and optimize, one replica still isn’t
> returning the right info.
>
> Do I need to configure my `solrconfig.xml` with `replicateAfter` options on
> the `/replication` requestHandler, or is that a non-solrcloud,
> standalone-replication thing?

This is one of the more confusing aspects of SolrCloud.

When everything is working perfectly in a SolrCloud install, the feature
in Solr called "replication" is *never* used.  SolrCloud does require
the replication feature, though ... which is what makes this whole thing
very confusing.

Replication is used to replicate an entire Lucene index (consisting of a
bunch of files on the disk) from a core on a master server to a core on
a slave server.  This is how replication was done before SolrCloud was
created.

The way that SolrCloud keeps replicas in sync is *entirely* different. 
SolrCloud has no masters and no slaves.  When you index or delete a
document in a SolrCloud collection, the request is forwarded to the
leader of the correct shard for that document.  The leader then sends a
copy of that request to all the other replicas, and each replica
(including the leader) independently handles the updates that are in the
request.  Since all replicas index the same content, they stay in sync.

What SolrCloud does with the replication feature is index recovery.  In
some situations recovery can be done from the leader's transaction log,
but when a replica has gotten so far out of sync that the only option
available is to completely replace the index on the bad replica,
SolrCloud will fire up the replication feature and create an exact copy
of the index from the replica that is currently elected as leader. 
SolrCloud temporarily designates the leader core as master and the bad
replica as slave, then initiates a one-time replication.  This is all
completely automated and requires no configuration or input from the
administrator.

The configuration elements you have asked about are for the old
master-slave replication setup and do not apply to SolrCloud at all.

What I would recommend that you do to solve your immediate issue:  Shut
down the Solr instance that is having the problem, rename the "data"
directory in the core that isn't working right to something else, and
start Solr back up.  As long as you still have at least one good replica
in the cloud, SolrCloud will see that the index data is gone and copy
the index from the leader.  You could delete the data directory instead
of renaming it, but that would leave you with no "undo" option.

Thanks,
Shawn

Inconsistency in results between replicas using CloudSolrClient

[Chris Troullis]

Hi,

I think I know the answer to this question, but just wanted to verify/see
what other people do to address this concern.

I have a Solr Cloud setup (6.6.0) with 2 nodes, 1 collection with 1 shard
and 2 replicas (1 replica per node). The nature of my use case requires
frequent updates to Solr, and documents are being added constantly
throughout the day. I am using CloudSolrClient via SolrJ to query my
collection and load balance across my 2 replicas.

Here's my question:

As I understand it, because of the nature of Solr Cloud (eventual
consistency), and the fact that the soft commit timings on the 2 replicas
will not necessarily be in sync, would it not be possible to run into a
scenario where, say a document gets indexed on replica 1 right before a
soft commit, but indexed on replica 2 right after a soft commit? In this
scenario, using the load balanced CloudSolrClient, wouldn't it be possible
for a user to do a search, see the newly added document because they got
sent to replica 1, and then search again, and the newly added document
would disappear from their results since they got sent to replica 2 and the
soft commit hasn't happened yet?

If so, how do people typically handle this scenario in NRT search cases? It
seems like a poor user experience if things keep disappearing and
reappearing from their search results randomly. Currently the only thought
I have to prevent this is to write (or extend) my own solr client to stick
a user's session to a specific replica (unless it goes down), but still
load balance users between the replicas. But of course then I have to
manage all of the things CloudSolrClient manages manually re: cluster
state, etc.

Can anyone confirm/deny my understanding of how this works/offer any
suggestions to eliminate the scenario in question from occurring?

Thanks,

Chris

Replication Question

[Michael B. Klein]

I have a 3-node solrcloud cluster orchestrated by zookeeper. Most stuff
seems to be working OK, except that one of the nodes never seems to get its
replica updated.

Queries take place through a non-caching, round-robin load balancer. The
collection looks fine, with one shard and a replicationFactor of 3.
Everything in the cloud diagram is green.

But if I (for example) select?q=id:hd76s004z, the results come up empty 1
out of every 3 times.

Even several minutes after a commit and optimize, one replica still isn’t
returning the right info.

Do I need to configure my `solrconfig.xml` with `replicateAfter` options on
the `/replication` requestHandler, or is that a non-solrcloud,
standalone-replication thing?

Michael

Add me in solr mailing list

[Santosh Kumar]

Hi,
Please add me in solr mailing list.


Thanks,
Santosh