Re: FYI Zookeeper.sync()

[Chris Hostetter]

: Thanks; I can definitely appreciate that Watchers (or more generally the
: idea of chaining async callbacks) is usually a more suitable mechanism than
: calling sync().  I've also seen some code patterns in which knowledge of

To be clear: i'm not suggesting that we *don't* need sync() calls 
anywhere, just that based on my knowledge of how we use ZK i wouldn't 
expect to see many sync() calls, ... if you see calls to getData() that 
are not inside of a Watcher callback, that smells like a potential bug ... 
but i would question if best solution to fixing any "getData() w/o sync()" 
code paths is really "add sync()" or if it's "move this code into a 
Watcher that then caches locally"


-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: FYI Zookeeper.sync()

[David Smiley]

Thanks; I can definitely appreciate that Watchers (or more generally the
idea of chaining async callbacks) is usually a more suitable mechanism than
calling sync().  I've also seen some code patterns in which knowledge of
the expected ZK node version can alleviate the need for doing a sync() as
well -- assuming you have an expected zk node version to do such a check.

~ David

On Thu, Apr 19, 2018 at 5:28 PM Chris Hostetter 
wrote:

>
> IIUC, the reason you don't see any calls to sync() is because Solr's use
> of ZK is mostly based on Watchers? ... so we have callback functions to be
> notified anytime something (like leaeders, overseer, cluster state,
> etc...) changes and those calbacks update local copies of that state,
> which other (less zk savy) code can read on demand ... there shouldn't
> be much "polling" of ZK data in Solr that would require syncs?
>
> : Date: Thu, 19 Apr 2018 21:16:41 +
> : From: David Smiley 
> : Reply-To: dev@lucene.apache.org
> : To: "dev@lucene.apache.org" 
> : Subject: FYI Zookeeper.sync()
> :
> : As I was contemplating how it is that I see some weird behavior while
> : working on SolrCloud stuff could happen, I started to question my basic
> : assumptions.  One assumption I held is that SolrZkClient.getData (which
> : calls  ZooKeeper.getData) would always return the most up to date
> : information from the ZK cluster.  The docs to ZooKeeper.getData say
> nothing
> : on this matter.  If it did work this way, I imagine it would need to talk
> : to at least a majority of the ZK ensemble. Now that I reed the ZK
> : Programmer's guide RE consistency guarantees --
> :
> http://zookeeper.apache.org/doc/r3.4.11/zookeeperProgrammers.html#ch_zkGuarantees
> : I
> : see reads may read stale data and that I can use ZooKeeper.sync() first
> to
> : get the latest.  Okay wow.  Interestingly we don't call this anywhere in
> : our codebase.  With this newfound realization, I think at least one place
> : pertaining to TimeRoutedAliases really wants the most up to date
> : information, so I'll need to add a call to sync.  Of course sync() should
> : be added deliberately not haphazardly; I'm sure it has overhead.
> : --
> : Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> : LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> : http://www.solrenterprisesearchserver.com
> :
>
> -Hoss
> http://www.lucidworks.com/
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
> --
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

Re: FYI Zookeeper.sync()

[Chris Hostetter]

IIUC, the reason you don't see any calls to sync() is because Solr's use 
of ZK is mostly based on Watchers? ... so we have callback functions to be 
notified anytime something (like leaeders, overseer, cluster state, 
etc...) changes and those calbacks update local copies of that state, 
which other (less zk savy) code can read on demand ... there shouldn't 
be much "polling" of ZK data in Solr that would require syncs?

: Date: Thu, 19 Apr 2018 21:16:41 +
: From: David Smiley 
: Reply-To: dev@lucene.apache.org
: To: "dev@lucene.apache.org" 
: Subject: FYI Zookeeper.sync()
: 
: As I was contemplating how it is that I see some weird behavior while
: working on SolrCloud stuff could happen, I started to question my basic
: assumptions.  One assumption I held is that SolrZkClient.getData (which
: calls  ZooKeeper.getData) would always return the most up to date
: information from the ZK cluster.  The docs to ZooKeeper.getData say nothing
: on this matter.  If it did work this way, I imagine it would need to talk
: to at least a majority of the ZK ensemble. Now that I reed the ZK
: Programmer's guide RE consistency guarantees --
: 
http://zookeeper.apache.org/doc/r3.4.11/zookeeperProgrammers.html#ch_zkGuarantees
: I
: see reads may read stale data and that I can use ZooKeeper.sync() first to
: get the latest.  Okay wow.  Interestingly we don't call this anywhere in
: our codebase.  With this newfound realization, I think at least one place
: pertaining to TimeRoutedAliases really wants the most up to date
: information, so I'll need to add a call to sync.  Of course sync() should
: be added deliberately not haphazardly; I'm sure it has overhead.
: -- 
: Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
: LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
: http://www.solrenterprisesearchserver.com
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org