Each deferred task has the potential to spin up a new (parallel) instance, though there is some dependence on the scheduler and your Min/Max Pending Latency settings (in Application Settings). If $$ is not an object, you could increase the Min Idle Instances to ensure the instances are around. There is some instance start-up overhead, though that mostly depends on your own code/framework.
You'd need to use larger front-end instances to get the memory footprint you need. F2's have 256MB, so you'd likely need the big F4 (512MB). For the channel API, you only need to know the channel ID to send the message. It is just a string, so you can include it in your task context easily. Sounds like your plan will work out fine. j On Wednesday, 26 December 2012 20:46:21 UTC-6, J.Ganesan wrote: > > Each task runs a genetic program. In desktop, each task takes 5 mins > and consumes 250 MB ram. A user-level job consists of 10 such tasks. > In desktop ( a normal Windows PC ), It takes about 50 minutes to > complete one user-level job. > > Can I use App Engine to run the tasks in parallel and achieve a > response of ~5 mins for the user-level job ? > > Specifically, a single http request triggers 10 DeferredTasks. The > DeferredTask need not do any datastore operation. Can taskqueue > process the DeferredTasks in parallel and handle the 10 DeferredTasks > simultaneously? > > As each DeferredTask runs, it has to notify the progress to the > client using Channels API. Is it possible ? Channel being not > serializable, how can I keep a channel as a member variable to a > DeferredTask? > > Could you please share your experience in CPU and memory intensive > DeferredTasks ? > > J.Ganesan > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/FkM58UczFHYJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
