+1 to this. There's currently a limit of 10 async requests pending at once, but the worst case will still be the latency of two calls.
Jeff On Tue, Sep 13, 2011 at 5:09 PM, JH <[email protected]> wrote: > sounds like you could use async urlfetch instead of 20 separate tasks? > > On Sep 13, 6:43 pm, "ESPR!T" <[email protected]> wrote: >> Hello guys, >> >> I would like to get an advice how to deal with long running request >> when the new pricing will apply. >> >> My app is searching for the cheapest book prices on the internet. So >> when user searches for the book and the price is not in database, I >> have to check 20+ sellers for the current price (some of those have >> API and sometime I get the info from their web by parsing the page). I >> have to display these prices instantly to user - basically I would >> like to get to 10 seconds at max). I am using tasks so when user >> displays a page, I create a task for each supplier and update the page >> trhough ajax while user is waiting for results. On the old pricing >> scheme I haven't had a reason for any optimization as these request >> are low CPU intensive, they just take 1-10 seconds to finish depending >> on speed of supplier source data. App engine just started multiple >> instances when needed and I've never get over 1 hour of CPU in a day. >> >> But now when they switch to instance time my prognozed payments went >> up from $0 to $2+ per day (just for a instance time + plus another >> fees for database writes/reads). I've put the max idle instance to 1 >> and raised the latency to max which got me back 15 hours of instance >> time (so I would pay 0-5 cents per day just for DB operations now). >> Then I've implemented memcache + I am going to play with cach headers >> for generated pages. >> >> The main issue for me is now that my one instance is not able to >> handle 20+ task requests + users browsing so its slow and it >> eventually still starts new instance on peaks (but I was expecting >> this). So my idea is to have one fron instance for all user related >> stuff and process tasks instantly by other instance. On the app engine >> I could use backends but I am not really keen to pay almost $2 per day >> for running a minimal backend for this type of tasks. >> >> So I am deciding to 'outsource' the task processing to some external >> service which is more userfriendly for low CPU/long latency requests. >> AFAIK there is an Amazon with the AWS Elastic Beans (http:// >> aws.amazon.com/elasticbeanstalk/) and the Heroku (which is capable to >> run java now) - I also still have an option to put my little worker >> app to some very cheap VPS or my own machine. So basically my GAE >> instance will queue up all taks and send them to my external workers >> which then will call my app back with an HTTP post with results. >> >> Do you think this is a good approach for my task or can you see some >> issues in it? Or maybe you have some other ideas for other providers >> compatible with java environment? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
