I need to use the URL Fetch API to load dynamic content (a JSP or servlet-generated content) from the same application that uses URL Fetch. It looks like this is not allowed on AppEngine -- and it doesn't seem related to the documented limitation that URL Fetch cannot get its own URL. (The URL is different, even though is has the same domain). Another point to note is that this works if I'm using URL Fetch to get static content (an HTML page).
* Background * I need to do this to serve content generated with HTMLUnit to make my app crawlable by search engines, as described in: http://code.google.com/web/ajaxcrawling/ * AppEngine log analysis * >From my browser I request http://puzzlebazaar.appspot.com?_escaped_fragment_=main >From the AppEngine log I see that the request for http://puzzlebazaar.appspot.com?_escaped_fragment_=main starts, it goes through HTMLUnit which then uses URL Fetch to get http://puzzlebazaar.appspot.com#!main. Following this I get an IOException : com.philbeaudoin.gwtp.crawler.server.CrawlFilter logStackTrace: java.util.concurrent.ExecutionException: java.io.IOException: Timeout while fetching: http://puzzlebazaar.appspot.com#!main The page content is empty. This exception is caught, the servlet continues and terminates normally. Then the request for http://puzzlebazaar.appspot.com#!main starts. >From the timings in the AppEngine logs it's clear that this request starts too late, that is, after the original request has terminated. * What I tried * 1) To make sure it wasn't a problem with HTMLUnit, I used URL Fetch directly, but got the same behavior. 3) I tried fetching dynamic content from the same domain but with more differences in the URL. (Not just the parameters.) It fails. 2) I tried fetching a statically-served HTML page from the same domain. This works. * Questions * 1) Is this documented behavior from AppEngine? 2) Is there a way to work around this? 3) Are there any plans to allow this in the future? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
