Hi Guys, Is the syntax of the robots.txt correct? Could be wrong.
To my knowledge this is what google likes, User-agent: * Disallow: / User-agent: Googlebot Allow: / Happy flying! Rob On 02/18/2010 02:03 AM, John Denker wrote: > On 02/17/2010 04:54 PM, Jon Stockill wrote: > > >> Presumably because there are some truly awful bots out there, and google >> at least is known to be well behaved. >> > But the truly awful bots don't look at robots.txt. > > In fact one of the easiest ways to catch rogue bots > is to disallow a small part of the site and then > blacklist anybody who goes there. > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Flightgear-devel mailing list > Flightgear-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/flightgear-devel > > ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel