I don't think backend CPU time counts against your front-end instance CPU hours quota. Backends are purely billed based on uptime - and I think this is true for both current and new pricing (starting November 1). But you mentioned something about datastore CPU? It is likely that the most of your billing is because of this.
Few more questions: Is the reason for 20 backend instances that you want to execute all your 20 deferred tasks parallely? You mentioned you have ~4500 tasks. How long does each one take, and how often does each one need to execute in a day? Lets assume that neither of these tasks are sensitive to latency, and can be executed at any time during the entire day. If each task takes 10 seconds on average, and needs to execute, for example, 6 times a day... that's a total of 4500 * 10 * 6 / 3600 = 75 instance hours. You should try to "schedule" your backends yourself so that you only pay for 75 instance hours. If you allow 20 instances to get created, then at some point all these 20 instances will complete their work, and then idle for 15 minutes (or at least billed for 15 minutes of idle time after the last task completes processing), before they're shutdown. You'll be wasting 5 instance hours each time this happens. I think your focus should be to minimize your instance hours by minimizing the number of parallel instances you allow running. In my calculation above you only need a total of 75 instance hours, and so you should set "instances" to 3 or 4 in backends.yaml. A yet another way of doing this in a more controlled fashion is by using "pull" queues instead of enqueueing tasks on the regular "push" type task queues. You can enqueue all your 4500 tasks on a single "pull" queue, and all your backends will constantly run pulling tasks out of pull queues and executing them, until the pull queue is empty. Then they can be woken up again by a cron job to go check the pull queue again. Lastly, any cost you're incurring because of Datastore CPU hours in the current pricing model, or because of Datastore writes in the new pricing model - those can't be avoided. You will incur those costs regardless of the context of execution of your tasks - front-end or backend. Hope this helps. On Tue, Oct 4, 2011 at 9:33 AM, someone1 <[email protected]> wrote: > Hello, > > Thank you for the replies. > > The tasks are indeed being run on my backend as I am being charged for > backend usage (I don't use backends in any other way otherwise). I > also see the _ah/deferred logs on my backend ID but not on my app. The > CPU time being used seems to correspond to the Datastore CPU time, are > the two currently linked? Even under the new pricing scheme, would I > be billed for my main app up-time when executing deferred tasks on my > backend? What part of my main app is considered under use when > executing tasks on the backend? > > The max number of instances allowed on any backend is 20. My backend > is setup as dynamic B1 with 20 instances. I let Google determine how > many instances needed to be up and running when I queue up my tasks > (usually all 20 run for 20-30 minutes each). > > Again, I'd just like to know what part of my main app is being used > during the backend operation that is eating up my CPU. I really > haven't coded anything else on my app except for this data mining > portion which should all be run on the backend now. > > Thanks, > Prateek > > On Oct 4, 8:38 am, Rishi Arora <[email protected]> wrote: > > I think deferred tasks is an excellent use case for backends. That's how > I > > use my backend as well. Can you confirm from your logs that your tasks > are > > indeed being processed on the backend? In the drop down for app > versions, > > there's a special "version" which is named after your backend. Select > that > > to check your logs specific to the backend. Also, I'm assuming the > reason > > you're blowing through your budget is because you're spanning out > multiple, > > possibly hundreds of instances. Can you find out how many instances get > > spawned for your deferred tasks? Can you find out how many backend > > instances are being spawned, if the backend is indeed being used for your > > tasks? Finally, when you configured your backend, what did you set as > your > > "instances" parameter in backends.yaml? I don't know what the default > is, > > but it is likely "unlimited". In your case, a instance of 1 or 2 sounds > > sufficient, but you'll have to play around with that, based on how much > > queueing occurs for your tasks. > > > > > > > > > > > > > > > > On Tue, Oct 4, 2011 at 12:06 AM, Gerald Tan <[email protected]> > wrote: > > > I believe CPU time will no longer be billable after the new pricing is > out > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To view this discussion on the web visit > > >https://groups.google.com/d/msg/google-appengine/-/Crry-7yTG4QJ. > > > > > To post to this group, send email to [email protected] > . > > > To unsubscribe from this group, send email to > > > [email protected]. > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
