I have your problem too, I haven't checked the situation since weeks but right after development a mapreduce job, which was doing 800k operations was showing results in ~30 minutes, but that time increased to 3 hours ...
I guess this is because they are re-designing appengine in a wrong direction In future we will get charged by instance usage, so you wouldn't want 70 instances, and I guess the modifications are in that direction I don't like this, but I wouldn't want to pay a lot either, sad ... On Jun 19, 8:52 am, Waleed Abdulla <[email protected]> wrote: > I think I narrowed the issue a little bit. Added a lot of logging.info() > statements and I noticed the following: > > 1. The time between inserting a task and when it starts running is too long: > 5 - 20 seconds. Typically tasks should execute right away after inserting if > you're below the queue rate. That's issue #1 > > 2. I put a timer in my code to calculate the time it takes to execute each > task. When I compare that time with the time app engine reports in the logs, > there is a big difference. 5 - 20 seconds. For example, processing a task > takes 25 seconds, of which only 10 seconds is actually spent inside my code. > So the infrastructure is keeping requests on hold for a while before they > get executed. That's issue #2 > > When I compare the delay in #1 to the time difference calculated in #2, > they're very close and they correlate (i.e. they go up and down together). > So that means that issue #1 is probably caused by issue #2. (i.e. tasks do > get picked for execution right away, but the actual execution doesn't start > until 5 - 20 seconds later). > > There seems to be an infrastructure problem in GAE. Maybe GAE is > experiencing high load, too much network traffic, ..etc. Whatever the > reason, it seems that requests are waiting in a queue for too long before > they run. > > Does anyone notice any issues that confirm or deny the above? Any ideas > how to work around it? > > Waleed > > On Sat, Jun 18, 2011 at 7:45 PM, Chiguireitor > <[email protected]>wrote: > > > > > > > > > The other issue is that each task spawns a new instance... could you be > > hitting a max instance limit there? i really don't know if there's an > > instance limit... but could happen > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To view this discussion on the web visit > >https://groups.google.com/d/msg/google-appengine/-/nsxW4XGycnwJ. > > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
