Yeah, that'll work, too. Unless there are two changes to the list of boards at about the same time. Then it will give a bizarre result.
The general idea of augmenting queries with extra results that you suspect it might not get seems to come up a lot as a "solution" to the eventual consistency problem. Other than entity groups (which doesn't seem to fit any of my real-world data models), this is the only suggestion in the docs. However, it really kinda sucks. Since query results are usually sorted in some way, you need to re-sort to get the fixed record into the right place. But, of course, you can't do that if you are using the query as an iterator. And, of course, the change may have made that record no longer fit the query criteria, or it may have caused it to fit where it didn't before, so you need to deal with all three cases: remove the record from the list of results, add the record to the list of results, and replace the record in a list of results. And then there is the question of where you put the record. Putting it into a cookie or query parameter is weird, and fragile. Putting it into memcache is fragile. I suppose it's possible that there just is no good design pattern to use here. In particular, I'm having trouble coming up with a way that doesn't require me to wrap 3 old lines of code with 20 lines of crap, then repeating that in 50 different places. I don't mind having a bunch of crap if there is some way to reuse it. But I'm struggling with that. Is ANYBODY out there happy with the way they solved this problem? -Joshua On Sep 4, 2011, at 5:51 PM, Tom Phillips wrote: > "I clear the cache whenever the list of boards changes in some way" > > How about update the cache at that point instead of clearing it? > > Need be you could even generate the HTML for the cache update with a > URLFetch to your UI handler where you include the added/changed board > key(s) as parameters, so they can be gotten with strong consistency on > that request and merge into the query result. > > /Tom > > On Sep 4, 11:16 am, Joshua Smith <[email protected]> wrote: >> My monkeypatching solution (see my recent post in the -python group), which >> Guido says I shouldn't use, but which is just so darned pretty I can't help >> it, has gotten me through the first challenge of switching to HR, which is >> dealing with google search results containing keys into my old app's data >> store. >> >> So now I'm looking at the big Kahuna problem of consistency. Here's my >> first messy challenge there: >> >> My app puts a list of boards on the home page for a town, along with the >> list of meetings. Generating that list of boards was taking a lot of CPU, >> but they hardly ever change, so I put in a memcache system that built the >> HTML when it wasn't in the cache, and then cached it before serving. I >> clear the cache whenever the list of boards changes in some way. >> >> Well that ain't gonna work in HR. It's quite possible that I update a >> board, clear the cache, and someone comes and hits that page before >> "eventually consistent" comes to pass. So now I've got a cached copy of the >> stale data. >> >> (Note that I cannot use entity groups to solve this because some boards are >> municipal agencies, and therefore cannot be parented to the town that is >> building its list. I could parent all boards to some global parent, but, >> well, yuck.) >> >> I have some different ideas about how to fix this, but I'm wondering if >> anyone else who's done the port to HR has come up with a solution they find >> particularly elegant? I assume this is a pretty common problem, so there >> must be a design pattern out there… somewhere. >> >> Here are my ideas: >> >> - Clear the cache with a periodic task that re-clears it several times. I'm >> thinking a recurring geometric retry would be prudent (1, 2, 4, 8, 16, 32, >> 64, 128, 256, 512 seconds, and then pray that we have consistency) >> >> - Checksum the modified or new board, and put that sum into memcache. When >> generating the new board, confirm that any checksums are good. This seems >> more deterministic, except I don't trust memache not to squelch the checksum >> record. So perhaps I should do something in the datastore. This feels like >> it's be about 10x as much code as the stupid geometric flush. >> >> Any suggestions? >> >> -Joshua > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
