On Dec 9, 2011 2:25 AM, "Steve Bennett" <[email protected]> wrote: > > > On 09/12/2011, at 9:35 AM, Lluís Batlle i Rossell wrote: > > > On Fri, Dec 09, 2011 at 09:32:43AM +1000, Steve Bennett wrote: > >> On 09/12/2011, at 2:59 AM, Lluís Batlle i Rossell wrote: > >> > >> Why not just add a robots.txt? > > > > Ah, yes, that could help. I did not think of it. :) > > > > I'd like to have the wiki pages available though; I'll have to learn > > those robots rules better. > > http://www.robotstxt.org/robotstxt.html
Just remember that robots.txt is a friendly suggestion which the bot is free to ignore. Google honors it but malicious bots may not. (mobile keyboards take some getting used to) > > -- > µWeb: Embedded Web Framework - http://uweb.workware.net.au/ > WorkWare Systems Pty Ltd > W: www.workware.net.au P: +61 434 921 300 > E: [email protected] F: +61 7 3391 6002 > > > > > > _______________________________________________ > fossil-users mailing list > [email protected] > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
_______________________________________________ fossil-users mailing list [email protected] http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
