How do I do a consistent get with a query? I thought that there was no way to
do a query that was guaranteed to get the lastest. Did I miss something in the
docs?
The code that builds the HTML fragment I'm caching uses a couple queries like
this:
def getBoards(self):
tid = self.id()
boards = BoardModel.gql("WHERE towns = :1 AND agency = :2 ORDER BY name",
tid, False).fetch(1000)
return filter(lambda x: not tid in x.disabled, boards)
The towns attribute is a list of towns that need notice for that board:
class BoardModel(db.Model):
towns = db.ListProperty(long)
...
How would you recommend rewriting that query to ensure the list of boards is
complete, have up-to-date names, etc.?
On Sep 4, 2011, at 7:24 PM, Nick Johnson wrote:
> Hi Joshua,
>
> Can you not do a consistent get for the data when you have a cache miss, to
> ensure you're fetching the latest copy, and cache that?
>
> Looking at how NDB does its caching may be instructive here.
>
> -Nick Johnson
>
> On Mon, Sep 5, 2011 at 1:16 AM, Joshua Smith <[email protected]> wrote:
> My monkeypatching solution (see my recent post in the -python group), which
> Guido says I shouldn't use, but which is just so darned pretty I can't help
> it, has gotten me through the first challenge of switching to HR, which is
> dealing with google search results containing keys into my old app's data
> store.
>
> So now I'm looking at the big Kahuna problem of consistency. Here's my first
> messy challenge there:
>
> My app puts a list of boards on the home page for a town, along with the list
> of meetings. Generating that list of boards was taking a lot of CPU, but
> they hardly ever change, so I put in a memcache system that built the HTML
> when it wasn't in the cache, and then cached it before serving. I clear the
> cache whenever the list of boards changes in some way.
>
> Well that ain't gonna work in HR. It's quite possible that I update a board,
> clear the cache, and someone comes and hits that page before "eventually
> consistent" comes to pass. So now I've got a cached copy of the stale data.
>
> (Note that I cannot use entity groups to solve this because some boards are
> municipal agencies, and therefore cannot be parented to the town that is
> building its list. I could parent all boards to some global parent, but,
> well, yuck.)
>
> I have some different ideas about how to fix this, but I'm wondering if
> anyone else who's done the port to HR has come up with a solution they find
> particularly elegant? I assume this is a pretty common problem, so there
> must be a design pattern out there… somewhere.
>
> Here are my ideas:
>
> - Clear the cache with a periodic task that re-clears it several times. I'm
> thinking a recurring geometric retry would be prudent (1, 2, 4, 8, 16, 32,
> 64, 128, 256, 512 seconds, and then pray that we have consistency)
>
> - Checksum the modified or new board, and put that sum into memcache. When
> generating the new board, confirm that any checksums are good. This seems
> more deterministic, except I don't trust memache not to squelch the checksum
> record. So perhaps I should do something in the datastore. This feels like
> it's be about 10x as much code as the stupid geometric flush.
>
> Any suggestions?
>
> -Joshua
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>
> --
> Nick Johnson, Developer Programs Engineer, App Engine
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
