Well, it has the benefit that all "B" tasks can be executed in parallel so it can be very fast. You can also wait until the "background services" functionality is added to App Engine (this is on the roadmap).
On Tue, Jul 13, 2010 at 2:13 AM, Phil McDonnell <[email protected]>wrote: > I see how this is possible, but it feels very clunky. Is this standard > practice? > > Thanks, > Phil > > > On Mon, Jul 12, 2010 at 7:09 PM, Pieter Coucke > <[email protected]>wrote: > >> For a similar problem I use a task A which fetches 10 entities, creates >> subtasks B for them and then creates a new task A with the datastore cursor >> as a parameter so this task can continue creating subtasks. >> >> >> >> On Mon, Jul 12, 2010 at 1:31 AM, Phil McDonnell < >> [email protected]> wrote: >> >>> I know there are a number of different frameworks and mechanisms to >>> process data offline on GAE. I need to have a daily job that scans one of >>> my datastore tables and computes an updated profile for all my users. What >>> would be the best mechanism to do this? Are task queues appropriate? >>> Should I be using some sort of mapreduce? If I should use some sort of >>> mapreduce framework, how does that integrate with GAE? My user base right >>> now is small, but I intend to scale this. >>> >>> Thanks, >>> Phil >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]<google-appengine%[email protected]> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> >> >> -- >> Pieter Coucke >> Onthoo BVBA >> http://www.onthoo.com >> http://www.koopjeszoeker.be >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<google-appengine%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Pieter Coucke Onthoo BVBA http://www.onthoo.com http://www.koopjeszoeker.be -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
