>>So I propose to change robots.txt to only disallow entries which causes >>searchs in bugzilla, and allow http://bugzilla.mozilla.org/show_bug.cgi >>at least. > I'm not sure that would work: many (most?) search engines disallow > searching anything with a ? in the URL. The only way to make this > workable would be to use mod_rewrite in the server and allow URLs like > http://bugzilla.mozilla.org/bugs/103097.html to view a bug. > > (In order to make it usefully indexable, you'd also have to have an > autogenerated page containing 130000 links, one to each bug. Otherwise > your carefully crafted URLs would never get hit in the first place. > Fortunately that page, while humungous, wouldn't take too much CPU to > generate).
Google doesn't care about '?' in URLs. Squid is who does. Just removing the robots.txt limitation would be anough, as bugs URLs are anywhere in the net and the bug themselves point to other bugs. > PS The mod_rewrite rule could look something like this: > > RewriteEngine On > RewriteRule ^/bugs/(.*)\.html$ /show_bug.cgi?id=$1 > > I may have the syntax slightly wrong, but it's something like that. On the ather hand, having this showtut to access bugs by number will be nice and it will provide a permanent URL for a bug. URLs that doesn't depend on the technology which is being used in the server.
