Thanks for your comments. HTMLUnit (a headless browser) uses URLFetch to get the content of any URL it needs to fetch when parsing a webpage or running javascript code. I think it would be really tricky to modify it in order to use a different mechanism when fetching from the same app.
@Nick, is there a way to make sure a new instance starts up? Or to check if I can get a spare instance? Does it depend on my level of subscription? (I'm using the free quota for now.) Philippe On Jun 22, 7:03 am, Alon Carmel <[email protected]> wrote: > if i wanted to hack this without nick's question i'd leave urlfetch and just > output a javascript jquery ajax call to that url. that might work. but its a > terrible hack. > - > Cheers, > public static function AlonCarmel() { > //Contact me > var email = '[email protected]'; > var twitter = '@aloncarmel'; > var web = 'http://aloncarmel.me'; > var phone = '+972-54-4860380'; > > } > > * If you received an unsolicited email from by mistake that wasn't of your > matter please delete immediately. All E-mail sent from Alon Carmel is > copyrighted to Alon Carmel 2008. Any details revealed in e-mails sent by > Alon Carmel are owned by the Author only. Any attempt to duplicate or > imitate any of the Content is prohibited under copyright law 2008. > > On Mon, Jun 21, 2010 at 9:01 PM, PhilBeaudoin > <[email protected]>wrote: > > > > > I need to use the URL Fetch API to load dynamic content (a JSP or > > servlet-generated content) from the same application that uses URL > > Fetch. It looks like this is not allowed on AppEngine -- and it > > doesn't seem related to the documented limitation that URL Fetch > > cannot get its own URL. (The URL is different, even though is has the > > same domain). Another point to note is that this works if I'm using > > URL Fetch to get static content (an HTML page). > > > * Background * > > > I need to do this to serve content generated with HTMLUnit to make my > > app crawlable by search engines, as described in: > >http://code.google.com/web/ajaxcrawling/ > > > * AppEngine log analysis * > > > From my browser I request > >http://puzzlebazaar.appspot.com?_escaped_fragment_=main > > > From the AppEngine log I see that the request for > >http://puzzlebazaar.appspot.com?_escaped_fragment_=mainstarts, it > > goes through HTMLUnit which then uses URL Fetch to get > >http://puzzlebazaar.appspot.com#!main. Following this I get an > > IOException : > > com.philbeaudoin.gwtp.crawler.server.CrawlFilter logStackTrace: > > java.util.concurrent.ExecutionException: java.io.IOException: Timeout > > while fetching: > >http://puzzlebazaar.appspot.com#!main > > > The page content is empty. This exception is caught, the servlet > > continues and terminates normally. > > > Then the request forhttp://puzzlebazaar.appspot.com#!mainstarts. > > From the timings in the AppEngine logs it's clear that this request > > starts too late, that is, after the original request has terminated. > > > * What I tried * > > > 1) To make sure it wasn't a problem with HTMLUnit, I used URL Fetch > > directly, but got the same behavior. > > 3) I tried fetching dynamic content from the same domain but with more > > differences in the URL. (Not just the parameters.) It fails. > > 2) I tried fetching a statically-served HTML page from the same > > domain. This works. > > > * Questions * > > > 1) Is this documented behavior from AppEngine? > > 2) Is there a way to work around this? > > 3) Are there any plans to allow this in the future? > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%2Bunsubscrib > > [email protected]> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
