Hi Jamie, How many entities will you be mapping over? How many summary records wil they reduce down to? How fast and how often will you be summarizing them?
If you are, for example, summarizing last month's data and you're running the summarize process early in the month, then a map-reduce would probably work very well. However, if you are reducing the data down to very few summary entities, you might also find a processes that runs over the data updating the summary entities serially (i.e. chained tasks) would work well enough. Just a couple ideas to consider. Robert On Sat, Apr 30, 2011 at 22:12, Jamie <[email protected]> wrote: > > I'm currently storing entities of a particular kind (e.g. > TrafficHits). I would like to begin removing older entities (e.g. > older than a month), but would like to calculate summary statistics on > these records before deleting them. > > What's the recommended approach for this (I'm using Java JDO)? > > 1) Perform a query to retrieve entities older than a month. Loop > through records and calculate stats. Delete entities using a delete > query with same where clause. This seems somewhat inefficient. > > 2) A mapreduce job? > > Any help would be appreciated. > > Thanks, > J > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
