Hello guys, I would like to get an advice how to deal with long running request when the new pricing will apply.
My app is searching for the cheapest book prices on the internet. So when user searches for the book and the price is not in database, I have to check 20+ sellers for the current price (some of those have API and sometime I get the info from their web by parsing the page). I have to display these prices instantly to user - basically I would like to get to 10 seconds at max). I am using tasks so when user displays a page, I create a task for each supplier and update the page trhough ajax while user is waiting for results. On the old pricing scheme I haven't had a reason for any optimization as these request are low CPU intensive, they just take 1-10 seconds to finish depending on speed of supplier source data. App engine just started multiple instances when needed and I've never get over 1 hour of CPU in a day. But now when they switch to instance time my prognozed payments went up from $0 to $2+ per day (just for a instance time + plus another fees for database writes/reads). I've put the max idle instance to 1 and raised the latency to max which got me back 15 hours of instance time (so I would pay 0-5 cents per day just for DB operations now). Then I've implemented memcache + I am going to play with cach headers for generated pages. The main issue for me is now that my one instance is not able to handle 20+ task requests + users browsing so its slow and it eventually still starts new instance on peaks (but I was expecting this). So my idea is to have one fron instance for all user related stuff and process tasks instantly by other instance. On the app engine I could use backends but I am not really keen to pay almost $2 per day for running a minimal backend for this type of tasks. So I am deciding to 'outsource' the task processing to some external service which is more userfriendly for low CPU/long latency requests. AFAIK there is an Amazon with the AWS Elastic Beans (http:// aws.amazon.com/elasticbeanstalk/) and the Heroku (which is capable to run java now) - I also still have an option to put my little worker app to some very cheap VPS or my own machine. So basically my GAE instance will queue up all taks and send them to my external workers which then will call my app back with an HTTP post with results. Do you think this is a good approach for my task or can you see some issues in it? Or maybe you have some other ideas for other providers compatible with java environment? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
