At 12:20 PM 9/22/2007 -0700, you wrote: >Craig, > > Everything that is posted here can be searched for on Google. Its > always unsettling to be doing a search and see your own post come > up. They do. Everything that is here goes to www.hard-core-dx.com, and > that means Google crawlers hit it. What I am not clear on (and I'm not > sure Google would have it in their FAQs) is whether they honor the > presence of a "robots.txt" file on a web server. That file lays out > rules for what is or is not off limits. Looking at the Internet Wayback > Machine (www.archive.org), I see that hard-core-dx.com was last available > in November 2005. On or before that date, I could see hard-core-dx as it > was for any time starting in the year 2000. Because 11/05 is the cutoff, > that means they put robots.txt in place then and prevented archive.org > from crawling them. > > Mike
Mike, Every once in a while I do a Google on myself. There are quite a few distinguished people named Charles A Taylor. I guess their parents didn't want them but were too lazy to offer them for adoption. Then there seem to be a few who are on the other side of the law. Probably in solitary confinement to protect the other prisoners. I ALWAYS find a reference to myself on Google, and NEVER deeper than the second page of thousands of pages. I usually get there from a remark made that gets included in Glenn Hauser's World of Radio. So I guess I'd better be very, very careful about using that word "cxxxxxr" because some chief engineer that read my remark may get the vapors and faint and fall on a high voltage point and get electrocuted. All because I used that nasty, nasty word! Just like the Chief Boatswains Mate says aboard the ship: "Oh, dear; oh, dear! Whatever shall I do? One of my men just fell overboard! Oh, my! Oh, dear! Gosh! CATmudgeon _______________________________________________ IRCA mailing list [email protected] http://montreal.kotalampi.com/mailman/listinfo/irca Opinions expressed in messages on this mailing list are those of the original contributors and do not necessarily reflect the opinion of the IRCA, its editors, publishing staff, or officers For more information: http://www.ircaonline.org To Post a message: [email protected]
