I have noticed that googlebot crawls my app-engine site extremely
slow. I've been monitoring this with the Google Webmaster Tools.
I can't change my crawl rate. It says:
"Your site has been assigned special crawl rate settings. You
will not be able to change the crawl rate."
>From reading articles on the web, it appears that google controls the
rate for large sites like blogger.com. I'm guessing that googlebot has
determined that this site is hosted on app engine and slowed the rate
(even with a custom domain name).
The site was previously written in ASP and had no problems in getting
googlebot to crawl the site quickly.
The site has 11,000 pages and only 5000 have been crawled by
googlebot. The site has been on app engine for 2 months.
Has anyone else ran into this same problem?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
