I'm working on a bit of an app that can receive requests for the same query several times in rapid succession, and it's almost always the same in a short timespan. Rather than doing datastore queries each time, I'd like to cache the information in memcache.
I first thought I'd model my mechanism on how ndb uses memcache. If I'm reading the docs right at https://cloud.google.com/appengine/docs/python/ndb/cache#memcache , ndb will delete a memcache key prior to a .put() and expect the first .get() to populate the memcache. But it sounds like this could cause a race if instance 1 is performing a put at the same time as instance 2 is performing a put, and that the stale data could be cached. 1. Instance 1 determines it needs to put changes to an object into the datastore. 2. Instance 2 determines it needs to get the same object from datastore. 3. Instance 1 deletes the memcache key. 4. Instance 2 queries the memcache, and finds it empty. 5. Instance 2 loads the old version from the datastore. 6. Instance 1 puts the new version in the datastore. 7. Instance 2 puts the old version's data into memcache. 8. Much later, Instance 3 queries the memstore for the object, and gets the old version. In this case, this is only using methods that are supposed to have strong consistency guarantees, but the way they're composed means that ndb's "fetch by key" would not be strongly consistent. I suspect the docs are simplifying the actual consistency protocol used by ndb, but this seems like a problem that needs to be solved pretty frequently. Anybody have insights to share? As I mentioned, my ultimate goal is to build a mechanism to cache a particular query in memcache, but I'm looking at just a get_by_id operation's caching to get ideas of how to do this well. Thanks, Piquan -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/90dfc15b-e4c0-45c8-906f-21f5cc01f68a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
