Rasmus Mohr writes: > Yes, that would be the case. For some unknown reason Looksmart allows > recognized robots/crawlers/spider and other non-standard user-agents > unlimited access according to the the robots.txt - all others are excluded. > I'd guess the weird looking "java" user-agent originates from an Java > application running on a platform/JVM unable to set the user-agent property. > The guys at Looksmart probably detected it in their logfiles...
I don't think so. I think they just processed the web robots list autocatically. In fact, that's what it says at the top of the robots.txt file. If you look at http://www.robotstxt.org/wc/active/html/contact.html you'll see where it comes from. > eh...beef? gripes, wrath, criticism, complaints, etc. General feeling of displeasure directed to some person or thing, Richard