I've looked for an issue in the tracker for this, didn't find anything so I created one. Please star it: http://code.google.com/p/googleappengine/issues/detail?id=3602
Cheers, Philippe On Aug 19, 3:44 am, Matt H <[email protected]> wrote: > This really NEEDS to be supported. > > On Jul 12, 6:26 pm, Krishna <[email protected]> wrote: > > > > > +1 > > > I'd love support for search engine crawling as well. > > > On Jul 4, 5:33 am, Jaroslav Záruba <[email protected]> wrote: > > > > +1 > > > I'm having the same issue. :( > > > > My goal is to store the generated markup in blob. > > > > On Jun 28, 9:12 pm, PhilBeaudoin <[email protected]> wrote: > > > > > Bumping this. > > > > > Is it a good idea to put an issue up for this, as it seems deeply > > > > rooted in AppEngine's internals? > > > > > On Jun 22, 12:56 pm, PhilBeaudoin <[email protected]> wrote: > > > > > > Thanks for your comments. > > > > > > HTMLUnit (a headless browser) uses URLFetch to get the content of any > > > > > URL it needs tofetchwhen parsing a webpage or running javascript > > > > > code. I think it would be really tricky to modify it in order to use a > > > > > different mechanism when fetching from the same app. > > > > > > @Nick, is there a way to make sure a new instance starts up? Or to > > > > > check if I can get a spare instance? Does it depend on my level of > > > > > subscription? (I'm using the free quota for now.) > > > > > > Philippe > > > > > > On Jun 22, 7:03 am, Alon Carmel <[email protected]> wrote: > > > > > > > if i wanted to hack this without nick's question i'd leave urlfetch > > > > > > and just > > > > > > output a javascript jquery ajax call to that url. that might work. > > > > > > but its a > > > > > > terrible hack. > > > > > > - > > > > > > Cheers, > > > > > > public static function AlonCarmel() { > > > > > > //Contact me > > > > > > var email = '[email protected]'; > > > > > > var twitter = '@aloncarmel'; > > > > > > var web = 'http://aloncarmel.me'; > > > > > > var phone = '+972-54-4860380'; > > > > > > > } > > > > > > > * If you received an unsolicited email from by mistake that wasn't > > > > > > of your > > > > > > matter please delete immediately. All E-mail sent from Alon Carmel > > > > > > is > > > > > > copyrighted to Alon Carmel 2008. Any details revealed in e-mails > > > > > > sent by > > > > > > Alon Carmel are owned by the Author only. Any attempt to duplicate > > > > > > or > > > > > > imitate any of the Content is prohibited under copyright law 2008. > > > > > > > On Mon, Jun 21, 2010 at 9:01 PM, PhilBeaudoin > > > > > > <[email protected]>wrote: > > > > > > > > I need to use the URLFetchAPI to load dynamic content (a JSP or > > > > > > > servlet-generated content) from the same application that uses URL > > > > > > >Fetch. It looks like this is not allowed on AppEngine -- and it > > > > > > > doesn't seem related to the documented limitation that URLFetch > > > > > > > cannot get its own URL. (The URL is different, even though is has > > > > > > > the > > > > > > > same domain). Another point to note is that this works if I'm > > > > > > > using > > > > > > > URLFetchto get static content (an HTML page). > > > > > > > > * Background * > > > > > > > > I need to do this to serve content generated with HTMLUnit to > > > > > > > make my > > > > > > > app crawlable by search engines, as described in: > > > > > > >http://code.google.com/web/ajaxcrawling/ > > > > > > > > * AppEngine log analysis * > > > > > > > > From my browser I request > > > > > > >http://puzzlebazaar.appspot.com?_escaped_fragment_=main > > > > > > > > From the AppEngine log I see that the request for > > > > > > >http://puzzlebazaar.appspot.com?_escaped_fragment_=mainstarts, it > > > > > > > goes through HTMLUnit which then uses URLFetchto get > > > > > > >http://puzzlebazaar.appspot.com#!main. Following this I get an > > > > > > > IOException : > > > > > > > com.philbeaudoin.gwtp.crawler.server.CrawlFilter logStackTrace: > > > > > > > java.util.concurrent.ExecutionException: java.io.IOException: > > > > > > > Timeout > > > > > > > while fetching: > > > > > > >http://puzzlebazaar.appspot.com#!main > > > > > > > > The page content is empty. This exception is caught, the servlet > > > > > > > continues and terminates normally. > > > > > > > > Then the request forhttp://puzzlebazaar.appspot.com#!mainstarts. > > > > > > > From the timings in the AppEngine logs it's clear that this > > > > > > > request > > > > > > > starts too late, that is, after the original request has > > > > > > > terminated. > > > > > > > > * What I tried * > > > > > > > > 1) To make sure it wasn't a problem with HTMLUnit, I used URLFetch > > > > > > > directly, but got the same behavior. > > > > > > > 3) I tried fetching dynamic content from the same domain but with > > > > > > > more > > > > > > > differences in the URL. (Not just the parameters.) It fails. > > > > > > > 2) I tried fetching a statically-served HTML page from the same > > > > > > > domain. This works. > > > > > > > > * Questions * > > > > > > > > 1) Is this documented behavior from AppEngine? > > > > > > > 2) Is there a way to work around this? > > > > > > > 3) Are there any plans to allow this in the future? > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the > > > > > > > Google Groups > > > > > > > "Google App Engine" group. > > > > > > > To post to this group, send email to > > > > > > > [email protected]. > > > > > > > To unsubscribe from this group, send email to > > > > > > > [email protected]<google-appengine%2Bunsubscrib > > > > > > > [email protected]> > > > > > > > . > > > > > > > For more options, visit this group at > > > > > > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
