John - robots.txt is a standard tool, as you said, that directs search engine robots away from specified directories and files. I use this at work to ensure, for instance, that maintenance areas of our website will not be crawled.
I'm not sure why one would want to lock out the information that one is hoping to disseminate, however. Obviously, this is a personal choice and one should do whatever fits within their comfort level, but were someone to click on an image of mine via Google, they would be directed to my website, which would be just fine. Cheers - george --- John Yeo <jonn...@thegrid.net> wrote: > This is a little off topic, as it doesn't relate directly to pinhole, but > many members of this list have websites exhibiting their work, so I decided > to put it out there. I have recently noticed a few requests every day on my > webserver for "robots.txt". After getting that file (which doesn't even > exist on my webserver!), they would download various other html files. I > found all these requests for the non-existant file strange, and also that > the html files were being downloaded, but the images weren't being viewed > (well, not usually). > > After looking a bit on google, I discovered that these are actually robots > or "spiders" that scour the internet for various reasons, such as collecting > information for search engines, but more importantly, linking or stealing > your images. Fortunately, you can lock these robots out of parts of your > website, or the whole site itself. Usually you would want to leave the site > open for search engines to increase traffic, but lock out your gallery so > sites like http://images.google.com, can't get at your artwork. [clip] ===== Handmade Photographic Images - http://GLSmyth.com DRiP Investing - http://DRiPInvesting.org __________________________________________________ Do You Yahoo!? Yahoo! Health - Feel better, live better http://health.yahoo.com
