Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?

2010-05-24 Thread Waleed Abdulla
I'm noticing a big improvement today. After having a task queue backlog of 5 days and growing yesterday, today the tasks are running faster and the backlog is quickly going down (4 days as of this moment) and at this rate it should be down to 0 by the end of the day. It seems that someone fixed

Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?

2010-05-24 Thread Waleed Abdulla
Just an update. The problem is back, and my task queues are back to their very slow execution mode :( My tasks backlog now stands at 4 days :( On Mon, May 24, 2010 at 10:57 AM, Waleed Abdulla wal...@ninua.com wrote: I'm noticing a big improvement today. After having a task queue backlog of

Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?

2010-05-23 Thread Wesley C (Google)
thanks for all your feedback on the the current task queue execution (or lack thereof). we have reported this to the engineering team, but to help other users, can one of you file a new issue at http://code.google.com/p/googleappengine/issues/entry and everyone affected star it for us? please post

[google-appengine] Re: Task queue can't keep up with load. Suggestions?

2010-05-21 Thread bFlood
i agree waleed. ever since the 500 request/second limit was removed from GAE about a month ago, the Task queue has taken a pretty big hit. When they raised the queue limit to 40/sec it was flying but then the limit above was removed (a good thing!) and the amount of 500 errors in the queue went

[google-appengine] Re: Task queue can't keep up with load. Suggestions?

2010-05-21 Thread Brandon Thomson
1500 tasks per minute would be 2 160 000 tasks per day which exceeds the billing quota of 1 000 000. Maybe you can aggregate your tasks for better results. Regards, Brandon On May 21, 2:00 pm, Waleed Abdulla wal...@ninua.com wrote: It seems that the speed at which the task queue executes tasks

Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?

2010-05-21 Thread Waleed Abdulla
1500 tasks per minute would be 2 160 000 tasks per day which exceeds the billing quota of 1 000 000. I had my quota raised to 5M tasks per day as the original quota was not enough. Maybe you can aggregate your tasks for better results. That actually goes against the advice I get to

Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?

2010-05-21 Thread Waleed Abdulla
Google told me to just make sure all tasks ran below 1 second but this doesn't even really help and IMO, defeats one of the main goals of the Task Queue itself (longer running processing, albeit below 30 secs) brian I agree, Brian. My tasks cannot be done in less than a second because they