My app is nearly identical to yours - several concurrent URL fetches are performed to "gather" content. And when users access my site, that content is nicely formatted for them for display. My solution - part of if has already been suggested by Jeff - use async URL fetch. Spinning an instance for 5 to 10 seconds waiting for your supplier's website to return book pricing data is a waste of resources. So, whenever you need to do a URL fetch, consider deferring it by queueing it up in a pull queue. Then once you have enough deferred (10 is a good number), then call URL fetch simultaneously for all the 10 requests. Another optimization is to use by the 9 free hours of backend time. I agree you can't have the backend running all the time. So, wake up the backend by a cron job that runs once every hour. This will incur a minimum cost of 15 minutes per hour = 6 instance hours - which is under the free quota. Each time your backend wakes up, it looks up all the URL fetch requests deferred in the last hour, and processes them. My app does exactly this, and it takes me about 45 seconds to fetch and process all data for ~300 URL fetches every hour.
If you are attempting to stay within the free quota, absolutely use the backend hours in any way you can. it'll be a pity to not use those free 9 instance hours. On Tue, Sep 13, 2011 at 8:43 PM, Tim Hoffman <[email protected]> wrote: > Hi > > You could submit the request via ajax back to your appengine app > and it can then do an async requuest on all the urls, . > > In your case you have some of the info already and have to fetch some of > it. > So it might be two ajax calls, one to get the list of books, the result is > book prices for stuff you know, plus an indicator of the books that a > further request > will be required, your front end can then display the details you have, > submit another > ajax request to appengine to fetch results for the books you currently have > no info on. > Which can then async urlfetch the rest of the details. > > This way user gets some info straight away and you get to keep you requests > to a minimum > and fill in the results later. > > Just a thought ;-) > > T > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/3dA05F9-QDsJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
