sounds like you could use async urlfetch instead of 20 separate tasks? On Sep 13, 6:43 pm, "ESPR!T" <[email protected]> wrote: > Hello guys, > > I would like to get an advice how to deal with long running request > when the new pricing will apply. > > My app is searching for the cheapest book prices on the internet. So > when user searches for the book and the price is not in database, I > have to check 20+ sellers for the current price (some of those have > API and sometime I get the info from their web by parsing the page). I > have to display these prices instantly to user - basically I would > like to get to 10 seconds at max). I am using tasks so when user > displays a page, I create a task for each supplier and update the page > trhough ajax while user is waiting for results. On the old pricing > scheme I haven't had a reason for any optimization as these request > are low CPU intensive, they just take 1-10 seconds to finish depending > on speed of supplier source data. App engine just started multiple > instances when needed and I've never get over 1 hour of CPU in a day. > > But now when they switch to instance time my prognozed payments went > up from $0 to $2+ per day (just for a instance time + plus another > fees for database writes/reads). I've put the max idle instance to 1 > and raised the latency to max which got me back 15 hours of instance > time (so I would pay 0-5 cents per day just for DB operations now). > Then I've implemented memcache + I am going to play with cach headers > for generated pages. > > The main issue for me is now that my one instance is not able to > handle 20+ task requests + users browsing so its slow and it > eventually still starts new instance on peaks (but I was expecting > this). So my idea is to have one fron instance for all user related > stuff and process tasks instantly by other instance. On the app engine > I could use backends but I am not really keen to pay almost $2 per day > for running a minimal backend for this type of tasks. > > So I am deciding to 'outsource' the task processing to some external > service which is more userfriendly for low CPU/long latency requests. > AFAIK there is an Amazon with the AWS Elastic Beans (http:// > aws.amazon.com/elasticbeanstalk/) and the Heroku (which is capable to > run java now) - I also still have an option to put my little worker > app to some very cheap VPS or my own machine. So basically my GAE > instance will queue up all taks and send them to my external workers > which then will call my app back with an HTTP post with results. > > Do you think this is a good approach for my task or can you see some > issues in it? Or maybe you have some other ideas for other providers > compatible with java environment?
-- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
