Re: consistency in SolrCloud replication
> I want to know where SolrCloud stands in terms of CAP. SolrCloud is a CP system. In the face of partitions, SolrCloud favors consistency over availability (mostly concerning writes). The system is eventually consistent, but should become consistent with a pretty low latency, unlike many cases with an AP system (dynamo style systems). You can get things like read your own writes (simple out of the box behavior) and monotonic reads with the SolrCloud architecture if your application requires it. Hopefully, we will be adding more knobs to control some of the tradeoffs that the current architecture can make over time. -- - Mark
RE: consistency in SolrCloud replication
Solr is provides availability and it is tolerant to partioning so that leaves consistency. It is eventual consistent. -Original message- > From:Bill Au > Sent: Fri 16-Nov-2012 15:00 > To: solr-user@lucene.apache.org > Subject: Re: consistency in SolrCloud replication > > Yes, my original question is about search. And Mark did answered is in his > original reply. I am guessing that the replicas are updated sequentially > so the newly added documents will be available in some replicas before > other. I want to know where SolrCloud stands in terms of CAP. > > Bill > > > On Thu, Nov 15, 2012 at 10:31 PM, Otis Gospodnetic < > otis.gospodne...@gmail.com> wrote: > > > I think Bill was asking about search > > I think the Q is whether the query hitting the shard where a doc was sent > > for indexing would see that doc even before that doc has been copied to > > replicas. > > > > I didn't test it, but I'd think the answer would be positive because of the > > xa log. > > > > Otis > > -- > > Performance Monitoring - http://sematext.com/spm > > On Nov 15, 2012 11:30 AM, "Mark Miller" wrote: > > > > > It depends - no commit necessary for realtime get. Otherwise, yes, you > > > would need to do at least a soft commit. That works the same way though - > > > so if you make your update, then do a soft commit, you can be sure your > > > next search will see the update on all the replicas. And with realtime > > get, > > > of course no commit is necessary to see it. > > > > > > - Mark > > > > > > On Nov 15, 2012, at 10:40 AM, David Smiley (@MITRE.org) < > > dsmi...@mitre.org> > > > wrote: > > > > > > > Mark Miller-3 wrote > > > >> I'm talking about an update request. So if you make an update, when it > > > >> returns, your next search will see the update, because it will be on > > > >> all replicas. > > > > > > > > I presume this is only the case if (of course) the client also sent a > > > > commit. So you're saying the commit call will not return unless all > > > > replicas have completed their commits. Right? > > > > > > > > ~ David > > > > > > > > > > > > > > > > - > > > > Author: > > > http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > > > > -- > > > > View this message in context: > > > > > http://lucene.472066.n3.nabble.com/consistency-in-SolrCloud-replication-tp4020379p4020518.html > > > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > > > > >
Re: consistency in SolrCloud replication
Yes, my original question is about search. And Mark did answered is in his original reply. I am guessing that the replicas are updated sequentially so the newly added documents will be available in some replicas before other. I want to know where SolrCloud stands in terms of CAP. Bill On Thu, Nov 15, 2012 at 10:31 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > I think Bill was asking about search > I think the Q is whether the query hitting the shard where a doc was sent > for indexing would see that doc even before that doc has been copied to > replicas. > > I didn't test it, but I'd think the answer would be positive because of the > xa log. > > Otis > -- > Performance Monitoring - http://sematext.com/spm > On Nov 15, 2012 11:30 AM, "Mark Miller" wrote: > > > It depends - no commit necessary for realtime get. Otherwise, yes, you > > would need to do at least a soft commit. That works the same way though - > > so if you make your update, then do a soft commit, you can be sure your > > next search will see the update on all the replicas. And with realtime > get, > > of course no commit is necessary to see it. > > > > - Mark > > > > On Nov 15, 2012, at 10:40 AM, David Smiley (@MITRE.org) < > dsmi...@mitre.org> > > wrote: > > > > > Mark Miller-3 wrote > > >> I'm talking about an update request. So if you make an update, when it > > >> returns, your next search will see the update, because it will be on > > >> all replicas. > > > > > > I presume this is only the case if (of course) the client also sent a > > > commit. So you're saying the commit call will not return unless all > > > replicas have completed their commits. Right? > > > > > > ~ David > > > > > > > > > > > > - > > > Author: > > http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > > > -- > > > View this message in context: > > > http://lucene.472066.n3.nabble.com/consistency-in-SolrCloud-replication-tp4020379p4020518.html > > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > >
Re: consistency in SolrCloud replication
I think Bill was asking about search I think the Q is whether the query hitting the shard where a doc was sent for indexing would see that doc even before that doc has been copied to replicas. I didn't test it, but I'd think the answer would be positive because of the xa log. Otis -- Performance Monitoring - http://sematext.com/spm On Nov 15, 2012 11:30 AM, "Mark Miller" wrote: > It depends - no commit necessary for realtime get. Otherwise, yes, you > would need to do at least a soft commit. That works the same way though - > so if you make your update, then do a soft commit, you can be sure your > next search will see the update on all the replicas. And with realtime get, > of course no commit is necessary to see it. > > - Mark > > On Nov 15, 2012, at 10:40 AM, David Smiley (@MITRE.org) > wrote: > > > Mark Miller-3 wrote > >> I'm talking about an update request. So if you make an update, when it > >> returns, your next search will see the update, because it will be on > >> all replicas. > > > > I presume this is only the case if (of course) the client also sent a > > commit. So you're saying the commit call will not return unless all > > replicas have completed their commits. Right? > > > > ~ David > > > > > > > > - > > Author: > http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > > -- > > View this message in context: > http://lucene.472066.n3.nabble.com/consistency-in-SolrCloud-replication-tp4020379p4020518.html > > Sent from the Solr - User mailing list archive at Nabble.com. > >
Re: consistency in SolrCloud replication
It depends - no commit necessary for realtime get. Otherwise, yes, you would need to do at least a soft commit. That works the same way though - so if you make your update, then do a soft commit, you can be sure your next search will see the update on all the replicas. And with realtime get, of course no commit is necessary to see it. - Mark On Nov 15, 2012, at 10:40 AM, David Smiley (@MITRE.org) wrote: > Mark Miller-3 wrote >> I'm talking about an update request. So if you make an update, when it >> returns, your next search will see the update, because it will be on >> all replicas. > > I presume this is only the case if (of course) the client also sent a > commit. So you're saying the commit call will not return unless all > replicas have completed their commits. Right? > > ~ David > > > > - > Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > -- > View this message in context: > http://lucene.472066.n3.nabble.com/consistency-in-SolrCloud-replication-tp4020379p4020518.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: consistency in SolrCloud replication
Mark Miller-3 wrote > I'm talking about an update request. So if you make an update, when it > returns, your next search will see the update, because it will be on > all replicas. I presume this is only the case if (of course) the client also sent a commit. So you're saying the commit call will not return unless all replicas have completed their commits. Right? ~ David - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/consistency-in-SolrCloud-replication-tp4020379p4020518.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: consistency in SolrCloud replication
I'm talking about an update request. So if you make an update, when it returns, your next search will see the update, because it will be on all replicas. Another process that is searching rapidly may see an "eventually" consistent view though (very briefly). We have some ideas to make that view "more consistent", for example by allowing for searcher leases. - Mark On Thu, Nov 15, 2012 at 8:53 AM, Bill Au wrote: > Thanks for the info, Mark. By "a request won't return until it's affected > all replicas", are you referring to the update request or the query? > > Bill > > > On Wed, Nov 14, 2012 at 7:57 PM, Mark Miller wrote: > >> It's included as soon as it has been indexed - though a request won't >> return until it's affected all replicas. Low latency eventual consistency. >> >> - Mark >> >> On Nov 14, 2012, at 5:47 PM, Bill Au wrote: >> >> > Will a newly indexed document included in search result in the shard >> leader >> > as soon as it has been indexed locally or is it included in search result >> > only after it has been forwarded to and indexed in all the replicas? >> > >> > Bill >> >> -- - Mark
Re: consistency in SolrCloud replication
Thanks for the info, Mark. By "a request won't return until it's affected all replicas", are you referring to the update request or the query? Bill On Wed, Nov 14, 2012 at 7:57 PM, Mark Miller wrote: > It's included as soon as it has been indexed - though a request won't > return until it's affected all replicas. Low latency eventual consistency. > > - Mark > > On Nov 14, 2012, at 5:47 PM, Bill Au wrote: > > > Will a newly indexed document included in search result in the shard > leader > > as soon as it has been indexed locally or is it included in search result > > only after it has been forwarded to and indexed in all the replicas? > > > > Bill > >
Re: consistency in SolrCloud replication
It's included as soon as it has been indexed - though a request won't return until it's affected all replicas. Low latency eventual consistency. - Mark On Nov 14, 2012, at 5:47 PM, Bill Au wrote: > Will a newly indexed document included in search result in the shard leader > as soon as it has been indexed locally or is it included in search result > only after it has been forwarded to and indexed in all the replicas? > > Bill