On Wed, Jan 29, 2014 at 8:07 PM, max kraun <[email protected]> wrote: > > When I run it on localhost it works, but when I run it on appspot.com it > looks like if some pages are not counted. Sometimes it works, sometimes it > does not. Sometimes it returns some errors ("Deadline exceeded while > waiting for HTTP response from URL" or similar), too. >
Hi Max, It's difficult to say exactly what the problem is without a more detailed look at the logs, but you can try a few things: 1. Try adding a time.sleep call between each page retrieval. See http://docs.python.org/2/library/time.html for usage. Sometimes App Engine doesn't like it when repeated URL Fetch calls are made. 2. Try making sure the authentication token you're using is still valid. For instance, you could make sure that the page you retrieved shows text consistent with a logged-in user - for example, only "sign out" links and not "sign in" links. 3. The deadline exceeded error comes because the other server took too long to send back the response. You can increase this time by setting the default fetch timeout: urlfetch.set_default_fetch_deadline(60) ----------------- -Vinny P Technology & Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
