Hi Dusty, I noticed the same thing when testing the Ajax crawlability for my website:
Googlebot didn't return me the static page if I run 'fetch as googlebot' for http://www.bookwhack.com/us/ But it works fine for http://www.bookwhack.com/us/?_escaped_fragment_= However, I can see in webmaster tools that my pages are now being crawled. So I suspect that the 'Fetch as Googlebot' tool takes the request at face value and does not process the <meta name="fragment" content="!"> but the actual crawler notices the fragment and ensures the bot also crawls the escaped fragment . So I believe that as long as you are seeing the correct page when you fetch as googlebot with the ?_escaped_fragment_= you should be ok. Just sharing my experience, happy to be corrected if any of the above is incorrect. -R www.bookwhack.com On Apr 20, 12:57 am, Dusty <[email protected]> wrote: > I asked it to fetchhttp://friendorfollow.com/dustyreagan/following/ > > If I ask it to fetch > either:http://friendorfollow.com/dustyreagan/following/#!http://friendorfollow.com/dustyreagan/following/?_escaped_fragment_= > > It works correctly. > > Shouldn't it also work for the plain > URL?http://friendorfollow.com/dustyreagan/following/ > > When I try the plain URL it doesn't show the whole indexable HTML. > > Thanks again! :) > > On Apr 19, 6:40 pm, Katharina Probst <[email protected]> wrote: > > > > > > > Hi Dusty, > > > what URL did you enter in "Fetch as Googlebot"? It should work > > forhttp://friendorfollow.com/dustyreagan/following/?_escaped_fragment_=(or > > forhttp://friendorfollow.com/dustyreagan/following/<http://friendorfollow.com/dustyreagan/following/?_escaped_fragment_=>#!, > > but that doesn't really apply to your site, because you're not using hash > > fragments). If you as it to > > fetchhttp://friendorfollow.com/dustyreagan/following<http://friendorfollow.com/dustyreagan/following/?_escaped_fragment_=>/, > > it should give you just the HTML, because that's what the crawler will see > > (and it will then extract the meta tag, construct the _escaped_fragment_ URL > > and fetch that). > > > Does that do the trick? I hope so.:) > > > kathrin > > > On Mon, Apr 19, 2010 at 7:32 PM, Dusty <[email protected]> wrote: > > > Thanks so much for your help Kathrin! :) > > > > However, I've tried these steps and Googlebot is still giving me no > > > love. > > > > If you go tohttp://friendorfollow.com/dustyreagan/following/youcan > > > see I've placed the meta tag in the header. > > > > Then if you go to > > >http://friendorfollow.com/dustyreagan/following/?_escaped_fragment_= > > > you can see the results fragment is embedded inline in the static > > > HTML. > > > > However, if I try "Fetch as Googlebot" (http://www.google.com/support/ > > > webmasters/bin/answer.py?hl=en&answer=158587) it shows the original > > > pages HTML, not the _escaped_fragment_ version. > > > > Any idea what I might have done wrong? > > > > On Apr 19, 2:02 pm, Katharina Probst <[email protected]> wrote: > > > > Yes, that's right! > > > > > kathrin > > > > > On Mon, Apr 19, 2010 at 3:00 PM, Dusty <[email protected]> wrote: > > > > > So, if I understand you right, it should work like this?: > > > > > > 1) I put <meta name="fragment" content="!"> in my static page (ie my > > > > > example:http://friendorfollow.com/dustyreagan/following/) > > > > > 2) Google will see it, then request > > > > >http://friendorfollow.com/dustyreagan/following/?_escaped_fragment_= > > > > > 3) When my webserver sees _escaped_fragment_ = null, I return my page > > > > > with the results HTML fragment inline instead of through jQuery load. > > > > > 4) Google associates that HTML to > > > > >http://friendorfollow.com/dustyreagan/following/ > > > > > in it's index. > > > > > > Sound correct? > > > > > > On Apr 19, 11:02 am, Katharina Probst <[email protected]> wrote: > > > > > > You might use <meta name="fragment" content="!"> in your html page, > > > and > > > > > when > > > > > > the crawler requests the corresponding _escaped_fragment_= URL, > > > you'll > > > > > have > > > > > > a version without jQuery, where you load the results page fragment > > > into > > > > > the > > > > > > static HTML and return that. Depending on what your server looks > > > like, > > > > > it > > > > > > first has to recognize that it is getting a request for such a URL, > > > and > > > > > then > > > > > > it will have to produce the results page fragment and put it > > > > > > together > > > > > with > > > > > > the static HTML - so everything happens on your server before you > > > return > > > > > a > > > > > > result. > > > > > > > As long as the resulting DOM is the same as what the user would get, > > > it > > > > > > should be ok. > > > > > > > kathrin > > > > > > > On Mon, Apr 19, 2010 at 2:15 AM, Dusty <[email protected]> > > > wrote: > > > > > > > I've been reading the "Making AJAX Applications Crawlable" specs > > > > > > > (http://code.google.com/intl/sv-SE/web/ajaxcrawling/docs/getting- > > > > > > > started.html< > > > > >http://code.google.com/intl/sv-SE/web/ajaxcrawling/docs/getting-%0Ast.. > > > .>) > > > > > > > and I'm at a loss at how to apply this to my > > > > > > > application. > > > > > > > > I use jQuery to load an HTML page fragment into a DOM element on a > > > > > > > mostly pure HTML page. It takes a few seconds for the 'results' > > > page > > > > > > > fragment to load from my server, this is why I use AJAX to > > > dynamically > > > > > > > load that content into the static page. Meanwhile the static page > > > > > > > shows a 'throbber' and loading message. How exactly do I apply > > > these > > > > > > > specs in this scenario? I don't use the hash mark in my URLs. > > > > > > > > The page in question can be seen here: > > > > > > >http://friendorfollow.com/dustyreagan/following/ > > > > > > > > Any pointers or recommendations on how to make the results > > > crawlable? > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > Groups > > > > > > > "Google Web Toolkit" group. > > > > > > > To post to this group, send email to > > > > > [email protected]. > > > > > > > To unsubscribe from this group, send email to > > > > > > > [email protected]<google-web-toolkit%2Bunsubs > > > > > > > [email protected]><google-web-toolkit%2Bunsubs > > > [email protected]><google-web-toolkit%2Bunsubs > > > > > [email protected]> > > > > > > > . > > > > > > > For more options, visit this group at > > > > > > >http://groups.google.com/group/google-web-toolkit?hl=en. > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > Groups > > > > > "Google Web Toolkit" group. > > > > > > To post to this group, send email to > > > [email protected] > > > > > . > > > > > > To unsubscribe from this group, send email to > > > > > [email protected]<google-web-toolkit%2Bunsubs > > > > > [email protected]><google-web-toolkit%2Bunsubs > > > [email protected]> > > > > > . > > > > > > For more options, visit this group athttp:// > > > > > groups.google.com/group/google-web-toolkit?hl=en. > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > Groups > > > > > "Google Web Toolkit" group. > > > > > To post to this group, send email to > > > [email protected]. > > > > > To unsubscribe from this group, send email to > > > > > [email protected]<google-web-toolkit%2Bunsubs > > > > > [email protected]><google-web-toolkit%2Bunsubs > > > [email protected]> > > > > > . > > > > > For more options, visit this group at > > > > >http://groups.google.com/group/google-web-toolkit?hl=en. > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups > > > "Google Web Toolkit" group. > > > > To post to this group, send email to [email protected] > > > . > > > > To unsubscribe from this group, send email to > > > [email protected]<google-web-toolkit%2Bunsubs > > > [email protected]> > > > . > > > > For more options, visit this group athttp:// > > > groups.google.com/group/google-web-toolkit?hl=en. > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Google Web Toolkit" group. > > > To post to this group, send email to [email protected]. > > > To unsubscribe from this group, send email to > > > [email protected]<google-web-toolkit%2Bunsubs > > > [email protected]> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-web-toolkit?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google Web Toolkit" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/google-web-toolkit?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google Web Toolkit" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group > athttp://groups.google.com/group/google-web-toolkit?hl=en. -- You received this message because you are subscribed to the Google Groups "Google Web Toolkit" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
