Hey guys, This is my fault. I wrote a really long groups post about this a while ago and never had the chance to write a version that goes into the official docs:
https://groups.google.com/forum/#!searchin/google-appengine/ikai$20eventually$20consistent/google-appengine/X8O2pDZt5i8/NrG3JOkjTUsJ Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Fri, Jul 8, 2011 at 9:37 AM, Stephen Johnson <[email protected]>wrote: > Hi Robert, > This is what I kind of thought but I didn't want to go that far in a > statement without official clarification. If that is the case then I think > the documentation should clarify this then because under the Usage Notes of > http://code.google.com/appengine/docs/java/datastore/hr/overview.html The > last sentence which I've bolded below makes it seem that a get which I'm > assuming is by key will see the most recently written data even under > Eventual consistency. Also, many times when people have asked about this it > has always been responded that the queries are eventually consistent but > that get by key is strongly consistent (at least that has been my > impression). I do believe you're right but IMHO this should be better > documented. > Stephen > > With eventual consistency, more than 99.9% of your writes are available for > queries within a few seconds. The goal is to find a caching solution for > your application that provides the data for the current user for the period > of time in which they're posting to your app. The caching solution might > involve memcache, a cache in a cookie, some state you put in the URL, or > something else entirely. The point is that, if the solution provides the > data for the current user in context of their posts, it will likely be > sufficient to make the eventual consistency of High Replication completely > acceptable. *Remember, if you do a get(), put(), or a transaction, you > will always see the most recently written data. > * > > On Fri, Jul 8, 2011 at 3:35 AM, Robert Kluin <[email protected]> > wrote: > > Hi, > > This is because doing a strongly consistent fetch by key uses a > > transaction -- which means you'll always get the latest version. If > > you fetch a bunch of keys, these transactions are currently done > > serially which significantly slows the request. An eventually > > consistent fetch goes to the fastest datastore node and returns > > whatever version it has. > > > > > > Robert > > > > > > > > On Thursday, July 7, 2011, Stephen Johnson <[email protected]> > wrote: > >> Hi Pete, > >> Yes, I've also tested this and the speed improvement is VERY > >> noticeable. With the way that the documentation explains the > >> difference between the HR datastores STRONG and EVENTUAL settings you > >> would assume that getting by the key value would be the same speed > >> because according to the documentation gets always return the most > >> recent data. You would assume that queries on the other hand would be > >> the thing that would have the time difference because queries with the > >> EVENTUAL setting can use the replicas (unless an ancestor query) and > >> could have stale results. So something else is going on here that > >> hasn't been explained. It seems due to the speed increase with > >> EVENTUAL consistency on that gets by key are using the replicas (or > >> possibly master if that's the closest datastore) to return the entity > >> instead of the master, so what it could be is that with STRONG > >> consistency on you always use the master for all datastore operations > >> even when doing a get by key even though it could be handled by a > >> nearby replica and with EVENTUAL consistency you always use the > >> closest datastore which could be a replica or the master. Or it could > >> be something entirely else. Hopefully an informed Googler could > >> explain the subtle differences a little better. > >> Stephen > >> > >> On Tue, Jul 5, 2011 at 10:51 AM, Peter Murray <[email protected]> > wrote: > >>> > >>> Greetings, > >>> In a small test retrieving 100 independent (non entity-group'd) > entities by > >>> key (e.g. ds.get(keys) ) in an HR datastore, we found that the > ReadPolicy > >>> significantly affected performance - with Consistency.STRONG set, the > >>> entities were returned in about 500ms (plus or minus), with > >>> Consistency.EVENTUAL set the entities were returned in about 50ms. > Could > >>> someone please describe the practical differences between the two > settings > >>> under an HR datastore? I had thought, from the I/O talk on the > subject, > >>> that essentially in an HR environment you were not guaranteed strong > >>> consistency under any circumstances unless the entities are in the same > >>> entity-group. Of course, something is taking 10x longer - the question > is, > >>> what am I getting for the 10x? > >>> Best, > >>> pete > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > Groups > >>> "Google App Engine" group. > >>> To view this discussion on the web visit > >>> https://groups.google.com/d/msg/google-appengine/-/apsAu6MR-BoJ. > >>> To post to this group, send email to [email protected] > . > >>> To unsubscribe from this group, send email to > >>> [email protected]. > >>> For more options, visit this group at > >>> http://groups.google.com/group/google-appengine?hl=en. > >>> > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > [email protected]. > >> For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > >> > >> > > > > -- > > ------ > > Robert Kluin > > Ezox Systems, LLC > > > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > [email protected]. > > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
