[EMAIL PROTECTED] wrote:
Over at Simpy.com I've been watching Googlebot for weeks now, hitting
the same (invalid) feed URLs over and over and over and over and
getting a 500 error response every time:
66.249.66.241 - - [26/Aug/2005:13:40:31 -0400] "GET
/simpy/LinksFeed.do?type=3amp;op=rssamp;version=2.0 HTTP/1.1" 500 6370
"-" "Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)"
If there is anyone from Google on this list, could you please remove
this URL (and a few other similar ones from simpy.com with the same
problem) from Googlebot's queue?
A couple of things you could try:
* Try blocking the URL path in your robots.txt file (specificly for
Googlebot if you like) and submit the URL for quick removal (see
http://www.google.com/webmasters/remove.html for details).
* Identify if there are any inbound links from other sites, the
Googlebot even if you remove the content may come back if other sites
have links to this URL (which looks like a misprinted version of
http://www.simpy.com/simpy/BookmarksFeeds.do?type=3&op=rss&version=2.0)
* The HTTP response is rather unusual with a very long error message,
this might confuse some bots, try replacing it with 500 Server error or
even better a 400 Bad request or 404 Not found, instead of 500
javax%2Eservlet%2Ejsp%2EJspException%3A+An+error+occurred+while+evaluating+custom+action+attribute+%22test%22+with+value+%22%24%7Bparam%2Etype+%3D%3D+3%7D%22%3A+An+exception+occured+trying+to
+convert+String+%223amp%3Boprssamp%3Bversion2%2E0%22+to+type+%22java%2Elang%2ELong%22+%28null%29,
or redirect to a valid page with a 301 response.
--
Klaus Johannes Rusch
[EMAIL PROTECTED]
http://www.atmedia.net/KlausRusch/
_______________________________________________
Robots mailing list
[email protected]
http://www.mccmedia.com/mailman/listinfo/robots
