For Webmasters: The Robots Exclusion Protocol is a method that allows Web site administrators to indicate to visiting robots which parts of their site should not be visited by the robot.
Please note, "Should Not". http://www.robotstxt.org/wc/exclusion.html#robotstxt It's just a useful information in Robots.Txt file, such as "do not go there: it is a Shopping Card (Chat, Forum, etc.)".
