Hi Jeff. Thanks for your reply. I did some measurements.
First is a request made to a different remote host: LTA Fetch on App Engine =================== # 04-06 04:35PM 03.011 /api/v1/16061/30/ 200 768ms 168ms-cpu 0kb Mozilla/ 5.0 (X11; U; Linux i686; en-US; rv:1.9.1b3) Gecko/20090305 Firefox/ 3.1b3,gzip(gfe) See details 218.186.10.253 - - [06/Apr/2009:16:35:03 -0700] "GET /api/ v1/16061/30/ HTTP/1.1" 200 275 - "Mozilla/5.0 (X11; U; Linux i686; en- US; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3,gzip(gfe)" # I 04-06 04:35PM 03.027 Method get called for APIEndPoint # D 04-06 04:35PM 03.036 Cache hit for stop 16061 # I 04-06 04:35PM 03.043 Fetching URL: http://www.publictransport.sg/public/ptp/en/Getting-Around/ArrivaltimeResults.html?hidServiceNoValue=&hidBusStopValue=16061 # I 04-06 04:35PM 03.626 Fetched URL: http://www.publictransport.sg/public/ptp/en/Getting-Around/ArrivaltimeResults.html?hidServiceNoValue=&hidBusStopValue=16061 ========= As you can see, this urlfetch took just over half a second to complete. Now a request that caused a fetch from sbstransit.com.sg NextBus fetch on App Engine ====================== # 04-06 04:34PM 11.864 /api/v1/17069/030/ 200 3276ms 279ms-cpu 0kb Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3,gzip(gfe) See details 218.186.10.253 - - [06/Apr/2009:16:34:15 -0700] "GET /api/ v1/17069/030/ HTTP/1.1" 200 173 - "Mozilla/5.0 (X11; U; Linux i686; en- US; rv:1.9.1b3) Gecko/20090305 Firefox/3.1b3,gzip(gfe)" # I 04-06 04:34PM 12.073 Method get called for APIEndPoint # I 04-06 04:34PM 12.107 Fetching URL: http://www.sbstransit.com.sg/mobileiris/index_svclist.aspx?stopcode=17069 # W 04-06 04:34PM 15.120 Error fetching url http://www.sbstransit.com.sg/mobileiris/index_svclist.aspx?stopcode=17069 Traceback (most recent call last): File "/base/data/home/apps/sbsnextbus/6.332607848382532409/ utils.py", line 72, in get_url result = urlfetch.fetch(url) File "/base/python_lib/versions/1/google/appengine/api/ urlfetch.py", line 273, in fetch raise DownloadError(str(e)) DownloadError: ApplicationError: 5 ======================= Now, the urlfetch got cut-off in 3 seconds. This is longer than usual but still under the 5 second limit. Now for comparison, here is a log of the same request executed on my localhost in the development server. On localhost ========== INFO 2009-04-06 23:50:15,961 main.py] Method get called for APIEndPoint DEBUG 2009-04-06 23:50:15,962 nextbus.py] Cache hit for stop 17069 INFO 2009-04-06 23:50:15,962 utils.py] Fetching URL: http://www.sbstransit.com.sg/mobileiris/index_mobresult.aspx?stop=17069&svc=051 DEBUG 2009-04-06 23:50:15,964 urlfetch_stub.py] Making HTTP request: host = www.sbstransit.com.sg, url = http://www.sbstransit.com.sg/mobileiris/index_mobresult.aspx?stop=17069&svc=051, payload = None, headers = {'Host': 'www.sbstransit.com.sg', 'Accept': '*/*'} DEBUG 2009-04-06 23:50:16,077 urlfetch_stub.py] Making HTTP request: host = www.sbstransit.com.sg, url = http://www.sbstransit.com.sg/mobileiris/(yvyrjnatocvc0y55rjqlocu3)/index_mobresult.aspx?stop=17069&svc=051, payload = None, headers = {'Host': 'www.sbstransit.com.sg', 'Accept': '*/*'} INFO 2009-04-06 23:50:17,259 utils.py] Fetched URL: http://www.sbstransit.com.sg/mobileiris/index_mobresult.aspx?stop=17069&svc=051 DEBUG 2009-04-06 23:50:17,260 nextbus.py] Saved cache for stop, svc 17069, 051 INFO 2009-04-06 23:50:17,274 dev_appserver.py] "GET /api/ v1/17069/051/ HTTP/1.1" 200 - ============= On localhost, the urlfetch completed in just under 1.5 seconds. Doesn't this indicate that (a) this could be a latency issue between Google's servers & SBS Transit's servers? and (b) request should still not be cut-off since it is well under the ~5 seconds limit. I know the 5 seconds limit is not documented (at least I can't find it) but it is an issue if it keeps changing all the time. Any thoughts? Cheers, Deepak On Apr 7, 5:16 am, Jeff S <[email protected]> wrote: > Hi Deepak, > > The error you are seeing usually means that the urlfetch service did > not receive the reply within ~5 seconds. There could be several > possible sourced of latency which could be involved as you noted, so > do you think the issue could be that the sbstransit site is responding > more slowly than it used to? > > Thank you, > > Jeff > > On Apr 6, 5:45 am, Deepak Sarda <[email protected]> wrote: > > > Hi > > > One of my apps is getting DownloadError (ApplicationError 5) for > > pretty much all urlfetch remote url fetch requests to a particular > > remote server for the past week. Fetch requests to other servers are > > not affected. Also, there are no problems fetching content from this > > remote server from the local development server. Here's a traceback > > from the logs: > > > Error fetching > > urlhttp://www.sbstransit.com.sg/mobileiris/index_svclist.aspx?stopcode=1... > > Traceback (most recent call last): > > File "/base/data/home/apps/sbsnextbus/6.332480362470588654/utils.py", > > line 71, in get_url > > result = urlfetch.fetch(url) > > File "/base/python_lib/versions/1/google/appengine/api/urlfetch.py", > > line 273, in fetch > > raise DownloadError(str(e)) > > DownloadError: ApplicationError: 5 > > > As can be seen from the trace, my app (http://sbsnextbus.appspot.com) > > is trying to fetch pages from sbstransit.com.sg > > > This appears to be a deadline exceeded error (error code 5) but I > > can't figure out why this should occur. Everything was running > > smoothly till last week when this started happening. > > > Since I can't reproduce this problem locally and there is no problem > > fetching pages from other remote servers in the production > > environment, I suspect the issue is related to network connectivity > > issues between AppEngine data centers & the servers ofwww.sbstransit.com.sg > > > This problem has been around for almost a week now and I can't seem to > > figure out a solution or work-around. Searches in the group archive > > didn't yield anything helpful. Any help would be greatly appreciated! > > > Thanks, > > > Deepak > > > PS: Is there a process to open a 'support ticket' with appengine team > > regarding infrastructure issues? --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
