I'm sorry, I am a bit new to this, so this may be a bit simplistic. Nonetheless, I must ask.
Do robots.txt files have stemming/wildcard abilities? ********************** User-agent: * Disallow: /cgi-bin Disallow: /site ********************** Will this only disallow the cgi-bin and the directory site, or will it also disallow anything that begins with site off the root domain. If it does not, is there any way to disallow file names in this manner? Thanks. Jonathan.
