Hi, I got to this list from robotstxt.org. From the looks of the archive, most here are building robots. My question is from the other side, having robots visit my site. I hope that my question can be answered here. I have a site that utilizes a shopping cart. The page to add goes like shoppingcart.asp?item=add&item=w123456. My robots.txt file has and entry that ends with shoppingcart.asp. I see many robots that visit the site, read the robots.txt file and go merrily on their way to add every item in the catalog to the shopping cart. I have had to resort to keeping records of all of the UAs that look at the robots.txt file and adding them to a list that will return a 404 error if they try to add items to the cart. I don't see this as a real solution due to the overhead involved. It does keep the items out of the shopping cart database but the overhead cost is too much. The site in question is a sub web. The robots file is in the domain root. What am I doing wrong? Thanks, Paul
_______________________________________________ Robots mailing list [EMAIL PROTECTED] http://www.mccmedia.com/mailman/listinfo/robots
