On Nov 19, 2007, at 8:29 AM, Trevor Watson wrote: > For the handling of robots.txt, I chose to return a file rather > than hard-coding the content on the basis that administrators _may_ > want to allow crawlers to attempt to crawl their pkg depot.
Or even part of it. Seems correct. > The content of the checked-in file disallows crawling of the depot > by default. I'd be happy to revise this if you guys think I'm on > the wrong track. The only real problem I see in doing it this way > is that the robots.txt file is located on PKG_ROOT/usr, which is > probably not the best place for a file which could be edited by the > admin. Speaking as a long-time website admin, if it's not in the servers "/" directory, I wouldn't have the vaguest idea where else to start looking. Sigh. -Tim _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
