Hi Benjamin. Just a thought: for a background task that's not critical 20/s and 50 concurrent looks a bit high. Maybe set it even lower (5/s, bucket 5, 5 concurrent) and then monitor the queue. If it often gets 'too long' (whatever that means for your app) you can turn it up.
I may be preaching to the choir but remember tasks also compete with incoming requests for your apps' resources. Also many long running tasks could push your average latency above 1000ms and encourage AppEngine to stop spinning up new instances. Dale On Mar 7, 2:47 pm, Benjamin <[email protected]> wrote: > This is a template for what I hope is how to best handle a low priory > task with very little impact on the system if it fails. It just cleans > up expired data and should not build up. Any other feedback is > appreciated. > > <queue> > <name>deletedata</name> > <rate>20/s</rate> > <bucket-size>40</bucket-size> > <max-concurrent-requests>50</max-concurrent-requests> > <task-retry-limit>1</task-retry-limit> > </queue> > > On Mar 7, 7:40 am, Benjamin <[email protected]> wrote: > > > I did not - i'm definitely going to add that, thank Dale. I re- > > enabled the queue the other day and it's been behaving as expected, so > > it was definitely some external factor that made it spin out of > > control. > > > My project is a cloud based data historian onwww.nimbits.comwhere > > users attach arduino devices to the services and record second by > > second data, so when they delete a data point I need to chew through > > the obsolete values that were recorded to that point. I ran out of > > quota on a day where a user deleted a point with millions of recorded > > values on it. It was supposed to just be a small background task. > > > On Mar 6, 11:44 am, Dale <[email protected]> wrote: > > > > Did you set max_concurrent_requests for that queue? Perhaps there was > > > some latency on the datastore and your queue was adding 5 tasks each > > > second but all the tasks were still running until the 10 s datastore > > > timeout and then retrying. In that case you would get 50 tasks > > > executing in parallel all querying 1000 records. Could explain the > > > high CPU usage. > > > > Dale > > > > On Mar 5, 5:33 pm, Benjamin <[email protected]> wrote: > > > > > Hi Nick, > > > > > The app id is nimbits1 (nimbits1.appspot.com) > > > > > The spike occurred around 12pm EST on March 3rd. 95% of the cpu being > > > > used at the time was > > > > Task Queues > deletedata which is set to 5/sec with a bucket size of > > > > 5. > > > > > Thanks! > > > > > On Mar 4, 4:21 pm, Nicholas Verne <[email protected]> wrote: > > > > > > Could you give us your app id and the approximate time of the gobbling > > > > > of CPU so we can investigate? > > > > > > Thanks, > > > > > > Nick > > > > > > On Sat, Mar 5, 2011 at 5:49 AM, Benjamin <[email protected]> wrote: > > > > > > I have a Task that's purpose is to chew away at data that needs to > > > > > > be > > > > > > deleted. I have millions of records that need to go, so the task > > > > > > uses > > > > > > the low level api and key only queries to grab 1000 delete them and > > > > > > restart the task until the remaining count is zero. > > > > > > > The task is set to 5/sec with a bucket size of 5. The other day it > > > > > > skyrocketed my CPU usage and maxed out my 5$ quota (400 CPU seconds > > > > > > / > > > > > > second, i think the lights in New York dimmed). Am i missing some > > > > > > setting here? How do i keep a task queue chugging along at a fixed > > > > > > rate without a burst like that, regardless of how many tasks are in > > > > > > the queue? > > > > > > > Thanks! > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "Google App Engine" group. > > > > > > To post to this group, send email to > > > > > > [email protected]. > > > > > > To unsubscribe from this group, send email to > > > > > > [email protected]. > > > > > > For more options, visit this group > > > > > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
