Hi Jeff, Megastore is very exciting and will help reduce timeouts and unplanned down time, but what I am most concerned about is this:
"Our maintenance periods usually last for about an hour, during which application serving is continuous, but access to the Datastore and memcache may be read-only or completely unavailable." If you are operating a webservice that does book keeping (needs persisent writes), and your app checks the capabilty api and finds that the primary datastore is in a read-only mode, it would be nice to be able to write incoming data somewhere else, anywhere else that could be considered persistent even for only a few hours. It would be nice if we could allocate a scratch space where we could persistently write some form of data while the primary datastore is read-only, then when the datastore comes out of maintenance mode, our own app, using the task-queue, can be responsible for syncing our scratch space with our primary data, this resyncing is done in userland. Such a scratch space would allow properly designed apps to operate mostly fine through a maintenance period, and maintenance periods do not have to mean a loss of service. And in the event of an unplanned datastore outage, the same fallback would allow apps basic writes to service requests until the primary datastore comes back. The idea is to give us some kind of writable persistent fallback, a different BigTable would be ideal, but even something extremely simple is far better than nothing. While far from idea, the simplest thing I could imagine would be an app uses logging to log data collected while the datastore was read-only, and then uses the task queue to parse/ replay the logged data when the datastore becomes writable again, but for something like that, apps need some API to access their logs. The goal would be that while the datastore is unwritable, whether planned or unplanned, apps can attempt to write vital data somewhere else. Currently the best candidate is to dump data to AWS SimpleDB until BigTable comes back, but I there must be a nicer fallback solution within AppEngine. Any ideas? Robin On Sep 30, 5:40 pm, "Jeff S (Google)" <[email protected]> wrote: > Hi Robin, > > This is a great idea, in fact we already replicate data into more than one > datacenter. At the moment, one datacenter is primary for a given app and > data is replicated in the background to other datacenters. There is a small > delay in replication, which we are in the process of shortening. When we do > fail over, we put the primary datacenter into read only mode while > replication completes, then we select the new primary datacenter. In the > future we may no longer need to have just one datacenter be the primary. > > This is covered in more detail in a recent blog post: > > http://googleappengine.blogspot.com/2009/09/migration-to-better-datas... > > Thank you, > > Jeff > > On Mon, Sep 28, 2009 at 1:03 PM, Robin B <[email protected]> wrote: > > > Here is an idea for better datastore fault tolerance: > > > Why not have >1 datastore available to each application. When Google > > needs to do maintenance, they put at most 1 datastore in read-only > > mode, and applications can revert to 1 or more alternate datastores > > until the maintenance is completed, then apps can use their task queue > > to resync their own data. > > > A person can accomplish this right now using AWS SDB when BigTable > > becomes unavailable, but sending that data over the internet is not > > ideal. > > > One of the big selling points of GAE is scalability, fault tolerance, > > and uptime, but having no persistent write access for any period of > > time prevents people from achieving those ideals. > > > Are temp/failover datastores a plausible idea? > > > Thanks, > > > Robin --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
