You couldn't even execute 100,000 tasks in a 5 minute time frame with the ideal 100/s execution rate.
Doing 100 tasks a second means it would take 16 minutes just to start those 100,000 tasks. Now, you might could create multiple queues.. but I'm not sure what the absolute tasks per second execution rate is. So, maybe you could group the activities into groups of 100.. so you only needed 1,000 recurring tasks. Then you could chain each individual task to re-add itself to the taskqueue with an ETA at the next 5 minute mark after it finished its current work. Though, any solutions will depend on what exactly these recurring activities are doing.. is it a URLfetch? or an email? or just datastore gets and puts? On Thu, Jan 13, 2011 at 6:19 PM, Supercobra Thatbytes <[email protected]>wrote: > Hello there, > > I am looking for ideas. > > I want to perform 100000 recurring activities every 5 minutes. Each > activity takes about 5 seconds. > > I think using 5-min cron to start 100k+ jobs would not be cpu/cost > effective. > > Any ideas on how to do this w/ the least overhead? > > Daniel > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
