I wanted to take this opportunity to mention I just spent a week or so feverishly trying to diagnose a similar problem, which also came down to the URL encoding in my requests.
I'm working on prototyping a Facebook application where the middleware is hosted by GAE, and the backend is a third-party web service. The flow of information looks something like this: client PC <--> apps.facebook.com <--> myapp.appspot.com <-- mydatasource.com Because fancy authentication is needed, I'm using urllib2 to fetch the data from mydatasource.com. In GAE, urllib and urllib2 are built on the native urlfetch library. I've been finding that for many requests, the call to urlfetch throws DownloadError: ApplicationError 2. This led me to believe, after some searching around, that the problem was a timeout in urlfetch. Documentation states the timeout is fixed at 5 seconds. If my services were slower than that, I'd have very few options for fixing the problem. But it turned out the data source was actually quite fast - subsecond responses to most queries. Assuming it was a timeout issue, this could only mean that Google's cloud was taking more than four seconds to find, connect to and download from mydatasource.com, which seemed absurd. If nothing else, the Cloud is very good at moving information quickly. Today I finally discovered that the DownloadError: ApplicationError 2 had *nothing* to do with timing. Not that there was anything in this sparsely worded and sparsely documented exception, or in its call stack, to implicate the actual problem, which was a malformed URL. It dawned on me what was happening when I compared some of the requests that were and weren't working. I found a pattern: ?param1=foo¶m2=bar <-- this query always succeeds. ?what=foo bar&where=bar, baz <-- this query always fails. What kept me from seeing it was for some reason I thought the latter was failing probabilistically, say 90% of the time, and I concluded that the slightly more complex query was taking too long. Turns out it failed 100% of the time and I assumed otherwise. In hindsight the problem is very obvious. Some of my parameters were unchecked for invalid URI encoding, and contained spaces and commas. There's a very simple way to tell this is the problem, and alokiN and djidjadji touched on it: ***if a GET request fails with urlfetch, but is valid when typed in the browser***, it is likely a URL-encoding bug. This is because browsers will silently correct your URLs, but App Engine does not. There are other significant possibilities to consider, but I had ruled them all out. Once I rewrote my code to assemble requests with urllib.urlencode(), it started working properly. None of this is strictly App Engine doing wrong - in my experience Python exceptions have always been kind of hard to follow. Given that Google controls this API, though, it would be nice if urlfetch checked for this, or if the exception made it clear that illegal characters may be choking the system. On Feb 8, 9:22 am, admin go2 <[email protected]> wrote: > You should use POST instead of GET. Since GET request would hit the google > cache server. > If you must use GET request, please try this > urlhttp://server/index.php?auth=authstring&random=a_random_number > > And make sure authstring is urlencoded. > > go2.appspot.com > > http://go2.appspot.com > > > > > Hi everyone, > > > I'm having trouble with using urlfetch. I'm trying to fetch a page > > that needs authentication. The authentication is done by using an url > > parameter (iehttp://server/index.php?auth=authstring). > > > When I write the url in my browser, the page I want is shown. When I > > call the fetch function with the same url, the login page is retrieved > > instead of the expected page, as if the authentication parameter > > wasn't recognised. I'm calling the function only with the url as the > > parameter ( urlfetch.fetch(http://server/index.php?auth=authstring) ), > > so the request should be a normal GET request. > > > Could you please tell me what am I doing wrong? > > > Thanks in advance. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
