Faustus von Goethe wrote:
> You're misunderstanding me. I said that I am not putting "D&D", "D20", or
> "OGL" in my robots.txt file (unlike many other D20 sites) or ANYWHERE on my
> commercial pages, AND
I suspect that very few people that _want_ search engines to find their D20
stuff have any mention of it in their robots.txt file. In their META tags,
oh yeah, but not in their robot file.
You are aware that robots.txt is to control where search engine spiders are to
NOT go, not to control where they do go, at least that is what it was designed
for.
For giggles and grins type in OGF 1066 into www.google.com. Seven pages come
up. The Earth 1066 FAQ page is number one.....
In my day job I spend a lot of time mucking with search engines. They find
links that you would not think they will, and will organize them in way that
may or may not make sense.
Because of the fact that your Unofficial OGF FAQ is on the same server as your
Earth 1066 FAQ, the Earth 1066 FAQ page is number 2 on a google search for OGF
FAQ. The Unofficial FAQ is number 1 and the Party line FAQ is number 3. Most
of the rest are in German and can be ignored in this context. :-)
