Thanks, Wooble, that cleared things up. I was a little confused by
the following log entries:
04-12 07:58PM 50.107 /mailer 500 28785ms 10749ms-cpu 0kb
See details
0.1.0.1 - - [12/Apr/2009:19:59:18 -0700] "GET /mailer HTTP/1.1" 500 0
- -
04-12 07:59PM 18.778
<class 'google.appengine.runtime.DeadlineExceededError'>:
So the 28785ms is where the problem is (not the 10749ms cpu)...this is
consistent with the recent post on the AppEngine blog about the
removal of the high CPU errors.
I guess my next concern is the fact that what I consider to be a
fairly simple script is taking so long to run. It is basically doing
a query of a user table that has ~500 rows and about 4 columns. From
this, it sends out an HTML email (usually about 3-4k in size) to each
user. This would take 2-3 seconds tops when I used to run it in a
Perl script when I was hosting my site elsewhere.
I did offload some of the Django template work on to Python. Since
there is some minor custom info in each email (user name, user id), I
was using the Django templates to format each message as I iterated
over the user list. Although there were only 3 variables that were
being substituted in the template, it seems to me that this is the
only possible place that could be causing such a huge performance
bottleneck. I haven't taken the time to profile it yet so I'll have
to do that whenever time allows.
Thanks again.
On Apr 13, 10:57 am, Wooble <[email protected]> wrote:
> The request deadline is actually 30 seconds now, but yeah, as far as I
> can tell cron jobs are just intended to free you from needing a cron
> job on your own machine to hit a URL with wget or whatever
> periodically.
>
> Long-running processes are probably what you're looking for; hopefully
> we'll see them soon.
>
> In the meantime, the remote API might be the best solution for big
> database cleanup jobs.
>
> On Apr 12, 11:41 pm, jorge <[email protected]> wrote:
>
> > I guess I probably misunderstood and thought that cron jobs weren't
> > subject to the same 10 second request deadline for HTTP requests? I'm
> > trying to run a cron job that sends out a mailer. Unfortunately, it's
> > taking longer than 10 seconds so the job will fail about halfway
> > through. I guess I could rewrite it to only do batches at a time.
> > The means extra tables in my database to keep track of who has been
> > mailed, etc. Not a big deal, I guess. It just seems that this
> > restriction makes cron jobs fairly limited in what they can do. A
> > database cleanup job, for example, is going to take longer and longer
> > as tables grow and will almost certainly always hit this limit, and
> > create a situation where the cron scripts will have to be rewritten
> > and additional tables written to manage intermediate state, etc.
>
> > Is the intent that the cron jobs will remain subject to the same
> > restrictions as other requests?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
