I do SEO consulting, so feel free to contact me off-list for more info. Google will rarely index all the URLs in your sitemap just because you put them out there. If they did, unscrupulous SEOs would fill sitemaps full of garbage and get tons of pages indexed.
How much time has passed since you published the sitemap? Typically Google will add more pages from your site to its index as time passes, if they determine that people are searching for your content and that it helps answer people's questions. If it's been less than 6 months, you should definitely give it more time. Kevin On 2/28/11 10:44 AM, Chris McCann wrote:
This isn't specifically a Ruby or Rails question but I suspect there's plenty of real-world experience in this group that can help. I created a sitemap for our Rails app at principalsmarket.com and submitted it to Google. The sitemap contains URLs for our members' public profiles as well as for each public post each member has made. So, let's say we have 1000 members and they've made 200 posts total, the sitemap contains 1200 URLs. The webmaster tools show no errors when the sitemap was processed. The number of URLs submitted in the sitemap is 1200, but the number indexed is around 1030. If I Google the titles of many of the public posts nothing comes up, even if I use "site:principalsmarket.com". Putting the exact URL of the public post into the Google search bar often yields nothing, indicating to me that for some reason the URLs to the posts aren't getting indexed, period. A few of the wall posts are indexed, but not many. The public member profile pages are clearly getting indexed. Googling some of our users' names returns their public PrincipalsMarket.com profile on the first page of results. Not so for the public wall posts. I'm a noob when it comes to this whole SEO thing, and I've spent a fair amount of time reading all of the Google Webmaster help pages. There are some explanations as to why pages or content don't get indexed but it doesn't seem to me we're committing any of the sins mentioned. If anyone has insight or experience into this type of problem I'd appreciate any words you have. Cheers, Chris
-- SD Ruby mailing list [email protected] http://groups.google.com/group/sdruby
