Matt, It sounds like you are describing a system that makes use of both the scheduled task (cron) API and the task queue API. You can avoid DeadlineExceededException by creating many small tasks that are distributed amongst your application instances.
As far as persistence goes, I would probably recommend the Data Store API. You're doing a relatively small amount of writes. You could also use Memcached for a performance boost, but as with any use of Memcached, you don't have as high a level of confidence that the data will be there between jobs. It's a pretty low chance the data will have been evicted, but this is a tradeoff that you'll have to consider based on your application's needs. On Tue, Nov 10, 2009 at 6:44 PM, Matt <[email protected]> wrote: > > Hello all - > > New to python world but i'm having a go at it. > > I'm trying to figure out the best approach to do some report > generation. Wondering if there are some best practices that may > apply. > > > I have a table where votes are cast. Each vote is for a station, so > it has the stationId and the datetimestamp when it was cast. > > I want to calculate the # of votes each station had over the > previous 2 hours. > > So I am going to use a task queue item. When the task queue item > starts, it loads its previous state (the start time / end time of the > period to calculate for & the index of the vote it last counted). > It loads 1000 votes at a time starting with the first from the > period. It calculates totals and writes them to a sep. table at the > end of 1000 votes, or when the DeadlineExceededExeption is thrown. > > Does this sound like a reasonable approach? > > Should I be saving my scratch into the DataStore or into a memcache? > (Is there enough time to write to Datastore after I get a > DeadlineExceededExeption?) > > Thank you- > Matt > > > > -- Ikai Lan Developer Programs Engineer, Google App Engine --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
