On Fri, Jan 7, 2011 at 7:57 PM, Ikai Lan (Google) <[email protected]> wrote: > Thanks for the update, Stephen (I am now realizing that there are 2 Stephens > in this thread and that you were not having a dialogue in public with > yourself)! I misspoke earlier: the numbers I cited were latency numbers in > the event of a failure in the Slave datastore. The actual numbers for the > majority (99+%) of requests is still a few hundred milliseconds. In most > cases, replication is even synchronous. It's in the error cases where the > average replication lag is 3 minutes. > > - The number of errors is less than a fraction of a fraction of a fraction > of a percent. I don't think a percent number makes sense here: it's the fact > that the errors are not evenly distributed. That is - a request that blocks > due to unavailability is likely to follow be several requests and lead to a > sudden spike of instability in your application that may depend on many > small, fast calls. > > To answer your last two questions, it makes more sense to explain how > eventually consistent reads work: > > 1. Fire off an RPC to the datastore service internally. > 2. Wait a grace period for this RPC to return. This grace period is in the > tens of milliseconds. Most requests (99+%) will return in a time that is > far, far under this grace period > 3. If the grace period has been surpassed, we send out an RPC to the slave > datastore. Now - we just wait to see who returned first: the original RPC to > the master datastore or the second RPC to the slave datastore. Again, in > most cases, the master datastore is likely to respond first simply because > it was given a head start
OK. So according to 2 and 3, almost always (99+%) you will hit a master server, so specifying eventual consistency makes no difference. So for the MS datastore this flag affects robustness not performance, right? Couple more questions... What about the HR datastore -- are queries across entity groups also likely to be 99+% consistent, or is the figure different? As cross-entity group HR queries are eventually consistent by default, is there ever a reason to use the eventually consistent flag, such as when querying a single entity group? Is there any difference between queries, gets, puts or deletes related to consistency, HR or MS datastore, or do the rules apply equally no matter the datastore operation? Thanks for taking the time to explain this so far -- much appreciated. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
