Hi Joshua, Can you not do a consistent get for the data when you have a cache miss, to ensure you're fetching the latest copy, and cache that?
Looking at how NDB does its caching may be instructive here. -Nick Johnson On Mon, Sep 5, 2011 at 1:16 AM, Joshua Smith <[email protected]>wrote: > My monkeypatching solution (see my recent post in the -python group), which > Guido says I shouldn't use, but which is just so darned pretty I can't help > it, has gotten me through the first challenge of switching to HR, which is > dealing with google search results containing keys into my old app's data > store. > > So now I'm looking at the big Kahuna problem of consistency. Here's my > first messy challenge there: > > My app puts a list of boards on the home page for a town, along with the > list of meetings. Generating that list of boards was taking a lot of CPU, > but they hardly ever change, so I put in a memcache system that built the > HTML when it wasn't in the cache, and then cached it before serving. I > clear the cache whenever the list of boards changes in some way. > > Well that ain't gonna work in HR. It's quite possible that I update a > board, clear the cache, and someone comes and hits that page before > "eventually consistent" comes to pass. So now I've got a cached copy of the > stale data. > > (Note that I cannot use entity groups to solve this because some boards are > municipal agencies, and therefore cannot be parented to the town that is > building its list. I could parent all boards to some global parent, but, > well, yuck.) > > I have some different ideas about how to fix this, but I'm wondering if > anyone else who's done the port to HR has come up with a solution they find > particularly elegant? I assume this is a pretty common problem, so there > must be a design pattern out there… somewhere. > > Here are my ideas: > > - Clear the cache with a periodic task that re-clears it several times. > I'm thinking a recurring geometric retry would be prudent (1, 2, 4, 8, 16, > 32, 64, 128, 256, 512 seconds, and then pray that we have consistency) > > - Checksum the modified or new board, and put that sum into memcache. When > generating the new board, confirm that any checksums are good. This seems > more deterministic, except I don't trust memache not to squelch the checksum > record. So perhaps I should do something in the datastore. This feels like > it's be about 10x as much code as the stupid geometric flush. > > Any suggestions? > > -Joshua > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Nick Johnson, Developer Programs Engineer, App Engine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
