Thanks Ikai! On Fri, Jul 8, 2011 at 11:59 AM, Ikai Lan (Google) <[email protected]>wrote:
> Hey guys, > > This is my fault. I wrote a really long groups post about this a while ago > and never had the chance to write a version that goes into the official > docs: > > > https://groups.google.com/forum/#!searchin/google-appengine/ikai$20eventually$20consistent/google-appengine/X8O2pDZt5i8/NrG3JOkjTUsJ > > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog: http://googleappengine.blogspot.com > Twitter: http://twitter.com/app_engine > Reddit: http://www.reddit.com/r/appengine > > > > On Fri, Jul 8, 2011 at 9:37 AM, Stephen Johnson <[email protected]>wrote: > >> Hi Robert, >> This is what I kind of thought but I didn't want to go that far in a >> statement without official clarification. If that is the case then I think >> the documentation should clarify this then because under the Usage Notes of >> http://code.google.com/appengine/docs/java/datastore/hr/overview.html The >> last sentence which I've bolded below makes it seem that a get which >> I'm assuming is by key will see the most recently written data even under >> Eventual consistency. Also, many times when people have asked about this it >> has always been responded that the queries are eventually consistent but >> that get by key is strongly consistent (at least that has been my >> impression). I do believe you're right but IMHO this should be better >> documented. >> Stephen >> >> With eventual consistency, more than 99.9% of your writes are available >> for queries within a few seconds. The goal is to find a caching solution for >> your application that provides the data for the current user for the period >> of time in which they're posting to your app. The caching solution might >> involve memcache, a cache in a cookie, some state you put in the URL, or >> something else entirely. The point is that, if the solution provides the >> data for the current user in context of their posts, it will likely be >> sufficient to make the eventual consistency of High Replication completely >> acceptable. *Remember, if you do a get(), put(), or a transaction, you >> will always see the most recently written data. >> * >> >> On Fri, Jul 8, 2011 at 3:35 AM, Robert Kluin <[email protected]> >> wrote: >> > Hi, >> > This is because doing a strongly consistent fetch by key uses a >> > transaction -- which means you'll always get the latest version. If >> > you fetch a bunch of keys, these transactions are currently done >> > serially which significantly slows the request. An eventually >> > consistent fetch goes to the fastest datastore node and returns >> > whatever version it has. >> > >> > >> > Robert >> > >> > >> > >> > On Thursday, July 7, 2011, Stephen Johnson <[email protected]> >> wrote: >> >> Hi Pete, >> >> Yes, I've also tested this and the speed improvement is VERY >> >> noticeable. With the way that the documentation explains the >> >> difference between the HR datastores STRONG and EVENTUAL settings you >> >> would assume that getting by the key value would be the same speed >> >> because according to the documentation gets always return the most >> >> recent data. You would assume that queries on the other hand would be >> >> the thing that would have the time difference because queries with the >> >> EVENTUAL setting can use the replicas (unless an ancestor query) and >> >> could have stale results. So something else is going on here that >> >> hasn't been explained. It seems due to the speed increase with >> >> EVENTUAL consistency on that gets by key are using the replicas (or >> >> possibly master if that's the closest datastore) to return the entity >> >> instead of the master, so what it could be is that with STRONG >> >> consistency on you always use the master for all datastore operations >> >> even when doing a get by key even though it could be handled by a >> >> nearby replica and with EVENTUAL consistency you always use the >> >> closest datastore which could be a replica or the master. Or it could >> >> be something entirely else. Hopefully an informed Googler could >> >> explain the subtle differences a little better. >> >> Stephen >> >> >> >> On Tue, Jul 5, 2011 at 10:51 AM, Peter Murray <[email protected]> >> wrote: >> >>> >> >>> Greetings, >> >>> In a small test retrieving 100 independent (non entity-group'd) >> entities by >> >>> key (e.g. ds.get(keys) ) in an HR datastore, we found that the >> ReadPolicy >> >>> significantly affected performance - with Consistency.STRONG set, the >> >>> entities were returned in about 500ms (plus or minus), with >> >>> Consistency.EVENTUAL set the entities were returned in about 50ms. >> Could >> >>> someone please describe the practical differences between the two >> settings >> >>> under an HR datastore? I had thought, from the I/O talk on the >> subject, >> >>> that essentially in an HR environment you were not guaranteed strong >> >>> consistency under any circumstances unless the entities are in the >> same >> >>> entity-group. Of course, something is taking 10x longer - the >> question is, >> >>> what am I getting for the 10x? >> >>> Best, >> >>> pete >> >>> >> >>> -- >> >>> You received this message because you are subscribed to the Google >> Groups >> >>> "Google App Engine" group. >> >>> To view this discussion on the web visit >> >>> https://groups.google.com/d/msg/google-appengine/-/apsAu6MR-BoJ. >> >>> To post to this group, send email to >> [email protected]. >> >>> To unsubscribe from this group, send email to >> >>> [email protected]. >> >>> For more options, visit this group at >> >>> http://groups.google.com/group/google-appengine?hl=en. >> >>> >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> Groups "Google App Engine" group. >> >> To post to this group, send email to [email protected] >> . >> >> To unsubscribe from this group, send email to >> [email protected]. >> >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> >> >> > >> > -- >> > ------ >> > Robert Kluin >> > Ezox Systems, LLC >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Google App Engine" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> [email protected]. >> > For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
