Brandon is both right and wrong. I'd suggest you take his comments with a bit of a grain of salt since he starting a fledging CloudFlare-like service called CDN In A Box. The short answer is: CloudFlare today will not hurt your SEO (and in fact usually helps it fairly significantly) and provides a high availability solution to the AppEngine SSL problem.
Here's the longer answer: When CloudFlare first began, we did have challenges with Google's crawler. While Brandon's reasoning may seem sound, it's actually an incorrect diagnosis. It was a puzzle for us for a while until we learned what the actual issue was by talking directly with the head of the Google Crawl team. At the root of the problem is the fact that Google sets crawl velocity based on an IP address. If multiple sites share an IP address and one of them has an issue then Google turns down crawl velocity in order to make sure they aren't contributing to excess load on the server that may be causing the problem. CloudFlare clusters multiple sites behind a pool of IP addresses. If one of those sites has an issue, we faithfully pass through the server error response code. Google's crawler was picking up that error response code and turning down crawl velocity for all the sites using that IP address. As a result, sites that weren't having issues but shared a CloudFlare IP with sites that were had their crawl rates decreased and therefore their SEO hurt. Google's crawl team had seen this problem before with other major CDNs like Akamai. The way they had dealt with it there was by detecting the CDN's CNAME in the DNS chain and writing a special rule for the crawler. In our case, a CloudFlare CNAME would not always appear in the DNS chain since we may return an IP address of our proxies directly as an A Record, so the solution for other CDNs would not work. We worked directly with the Google crawl team, as well as the crawl teams from other major search providers, in order to come up with a solution. Today, there are special rules in place for CloudFlare's IP ranges that assign the highest crawl velocity to sites using the IPs. We have an established channel to feed new CloudFlare IPs to the crawl teams as we are allocated them. You can see this yourself if your site is behind CloudFlare by logging in to Google Webmaster Tools and seeing that the option to adjust your crawl rate is no longer available. Search engines know we can handle their maximum crawl load, so they hammer away at us -- which is great for our users. While Brandon is correct that this was a problem before, our work with search crawl teams turned this problem into a feature and it is part of the reason why today being on CloudFlare can help your overall SEO. If you're interested in learning more, I've written about this on our blog: http://blog.cloudflare.com/cloudflare-and-seo And, related: http://blog.cloudflare.com/losing-seo-link-juice-to-traditional-cdns In terms of SSL and AppEngine<http://blog.cloudflare.com/ssl-on-custom-domains-for-appengine-and-other>, we had a number of users ask if we could help. We spent a significant amount of time building a cloud-based solution that allowed for custom domains to have SSL. Since we'd already built the frontend of that, it was relatively easy for us to extend the solution to the backend and, essentially, mask AppEngine's non-custom domain with your own custom domain. It was minor feature for us, but we've been surprised by how many AppEngine users have adopted it. Today, CloudFlare powers more than 100,000 websites. We typically will double the performance of a site and add a security layer which you can enable or disable depending on your preferences. If there are ways in which we can make CloudFlare better for the AppEngine community, please don't hesitate to let us know. Cheers, Matthew Prince CEO, CloudFlare @eastdakota <https://twitter.com/#!/eastdakota> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/XNFWzT0YH3gJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
