"I clear the cache whenever the list of boards changes in some way"
How about update the cache at that point instead of clearing it? Need be you could even generate the HTML for the cache update with a URLFetch to your UI handler where you include the added/changed board key(s) as parameters, so they can be gotten with strong consistency on that request and merge into the query result. /Tom On Sep 4, 11:16 am, Joshua Smith <[email protected]> wrote: > My monkeypatching solution (see my recent post in the -python group), which > Guido says I shouldn't use, but which is just so darned pretty I can't help > it, has gotten me through the first challenge of switching to HR, which is > dealing with google search results containing keys into my old app's data > store. > > So now I'm looking at the big Kahuna problem of consistency. Here's my first > messy challenge there: > > My app puts a list of boards on the home page for a town, along with the list > of meetings. Generating that list of boards was taking a lot of CPU, but > they hardly ever change, so I put in a memcache system that built the HTML > when it wasn't in the cache, and then cached it before serving. I clear the > cache whenever the list of boards changes in some way. > > Well that ain't gonna work in HR. It's quite possible that I update a board, > clear the cache, and someone comes and hits that page before "eventually > consistent" comes to pass. So now I've got a cached copy of the stale data. > > (Note that I cannot use entity groups to solve this because some boards are > municipal agencies, and therefore cannot be parented to the town that is > building its list. I could parent all boards to some global parent, but, > well, yuck.) > > I have some different ideas about how to fix this, but I'm wondering if > anyone else who's done the port to HR has come up with a solution they find > particularly elegant? I assume this is a pretty common problem, so there > must be a design pattern out there… somewhere. > > Here are my ideas: > > - Clear the cache with a periodic task that re-clears it several times. I'm > thinking a recurring geometric retry would be prudent (1, 2, 4, 8, 16, 32, > 64, 128, 256, 512 seconds, and then pray that we have consistency) > > - Checksum the modified or new board, and put that sum into memcache. When > generating the new board, confirm that any checksums are good. This seems > more deterministic, except I don't trust memache not to squelch the checksum > record. So perhaps I should do something in the datastore. This feels like > it's be about 10x as much code as the stupid geometric flush. > > Any suggestions? > > -Joshua -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
