Hey guys, some answers to your questions that we've compiled from various team members (mostly Alfred, presenter of "Next Gen Queries" and the fourth one from the left in this video http://www.youtube.com/watch?v=dg0TEIRQePg):
- Typically, how long does it take for HR query data to become consistent (a ballpark figure)? 100ms is the estimate. However - we have seen cases where this can be higher. If you were to use the eventual consistency read option in the master-slave configuration, we've seen that the average replication lag is about 3 minutes with an upper bound of 10 minutes. - Given that developers now have this HR option - allowing them to avoid planned read-only datastore periods - should we expect the frequency of planned read-only periods for the Master/Slave setup to increase? No. We had roughly one maintenance period every 2 months last year. The number of scheduled maintenance periods should be about the same. Note that there will be a scheduled maintenance period on February 7th - I'll be posting the details to the downtime notify list shortly. - Will this release have an impact on the latencies and error rates for applications that use (and will use) Master/Slave? No. Master/Slave remains unchanged. However we are currently working on updates to Master/Slave that should increase availability. - Why do you limit developers to use only one datastore per app? High Replication can serve out of multiple places at the same time while Master/Slave cannot. If we allowed both types of datastores in the same app it would have one of the following problems: - we'd be limited to serving out of the single 'Master' location which would greatly reduce the availability of the app as a whole, and you wouldn't see a benefit - serving out of a non-'Master' location which would greatly reduce the availability and performance of the Master/Slave datastore, as it would have to reach across data centers for consistent data In the future we might find ways around these trade offs and offer this feature, but this is not something on our immediate roadmap. - The migration path should be automated from the back end when the developer selects the HRD from the admin application configuration console. Migrating from M/S to High Rep with out loss of data will always require re-writing all of an apps data. To do this while preserving data consistency requires a read-only period. Because of this we decided to give the developers direct control over the migration process. This allows for app specific optimization to reduce the read-only period (X kind needs to be copied perfectly, Y kind doesn't need to be perfect, Z kind is session data and can be ignored completely, etc). Out of the box, the solution we currently provide almost completely automates the actual copying of the data. We thought it was important to get the High Replication datastore to developers as soon as possible and we plan on smoothing out other aspects off the migration process, specifically, issues around creating or using a new app, in future releases. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Thu, Jan 6, 2011 at 10:56 AM, stevep <[email protected]> wrote: > I think this should be great for my new "high confidence" put > architecture changes (original thread below). > > If I'm reading this right, I can set up a separate High Reliability > application that will handle the put(). > > These puts are much lower volume than other client activities, but I > really need high reliability for them. > > So architecture at a summary level is: > > 1) Client initiates the put to MS application. > 2) MS application launches a task queue task to HR application. > 3) MS replies back to client that HR task is launched. > 4) Client pings HR application to confirm write (showing appropriate > dialog). > 5) HR task queue puts record. > 6) Client now gets HR confirmation to ping and shows confirmation > dialog. > > If I've missed something please advise. The higher cost and higher cpu > cycles for HR (link below) are not issues as only a subset of > critical, lower volume puts get HR costing. > > If I've not missed anything, then pleased to say in advanced (before > refactoring code) "Thanks Google". > > Links: > Previous write architecture thread: > > http://groups.google.com/group/google-appengine/browse_thread/thread/c0e44d36d54384a3 > > MS vs. HR performance testing (thanks kaekon): > > http://groups.google.com/group/google-appengine/browse_thread/thread/5fc3b6a4366de62f > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
