>When do you use a robots.txt file? Ken: I've personally been evangelizing the use of robots.txt for controlling access to RSS feeds. Some publishers want to offer full-text feeds to individual users, but don't want that full-text harvested by robots and republished... robots.txt is the best means we have right for the publisher to exercise a little control. (The problem is getting other aggregator developers on board, naturally.)
-- Roger Benningfield JournURL http://admin.support.journurl.com/ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Message: http://www.houseoffusion.com/lists.cfm/link=i:4:229113 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
