Sorry could you be more specific? I have to display a page to user while I am fetching the prices on background (sometimes I have prices for 10 suppliers already in db and I need to fetch just 10 more). I though that async url fetch is more like like that you can call a fetch from url on background and then display the prices so in my case if the fastest supplier answer in 1 sec and slowest in 30 secs the user would have to wait 30 sec for the results) - or maybe just didn't get what did you mean.
On Sep 14, 12:09 pm, JH <[email protected]> wrote: > sounds like you could use async urlfetch instead of 20 separate tasks? > > On Sep 13, 6:43 pm, "ESPR!T" <[email protected]> wrote: > > > > > > > > > Hello guys, > > > I would like to get an advice how to deal with long running request > > when the new pricing will apply. > > > My app is searching for the cheapest book prices on the internet. So > > when user searches for the book and the price is not in database, I > > have to check 20+ sellers for the current price (some of those have > > API and sometime I get the info from their web by parsing the page). I > > have to display these prices instantly to user - basically I would > > like to get to 10 seconds at max). I am using tasks so when user > > displays a page, I create a task for each supplier and update the page > > trhough ajax while user is waiting for results. On the old pricing > > scheme I haven't had a reason for any optimization as these request > > are low CPU intensive, they just take 1-10 seconds to finish depending > > on speed of supplier source data. App engine just started multiple > > instances when needed and I've never get over 1 hour of CPU in a day. > > > But now when they switch to instance time my prognozed payments went > > up from $0 to $2+ per day (just for a instance time + plus another > > fees for database writes/reads). I've put the max idle instance to 1 > > and raised the latency to max which got me back 15 hours of instance > > time (so I would pay 0-5 cents per day just for DB operations now). > > Then I've implemented memcache + I am going to play with cach headers > > for generated pages. > > > The main issue for me is now that my one instance is not able to > > handle 20+ task requests + users browsing so its slow and it > > eventually still starts new instance on peaks (but I was expecting > > this). So my idea is to have one fron instance for all user related > > stuff and process tasks instantly by other instance. On the app engine > > I could use backends but I am not really keen to pay almost $2 per day > > for running a minimal backend for this type of tasks. > > > So I am deciding to 'outsource' the task processing to some external > > service which is more userfriendly for low CPU/long latency requests. > > AFAIK there is an Amazon with the AWS Elastic Beans (http:// > > aws.amazon.com/elasticbeanstalk/) and the Heroku (which is capable to > > run java now) - I also still have an option to put my little worker > > app to some very cheap VPS or my own machine. So basically my GAE > > instance will queue up all taks and send them to my external workers > > which then will call my app back with an HTTP post with results. > > > Do you think this is a good approach for my task or can you see some > > issues in it? Or maybe you have some other ideas for other providers > > compatible with java environment? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
