It was a creed written into the founding documents that declared the destiny of a nation
--- from: http://www.kottke.org/09/01/the-countrys-new-robotstxt-file "The country's new robots.txt file" Here's a small and nerdy measure of the huge change in the executive branch of the US government today. Here's the robots.txt file from whitehouse.gov yesterday: User-agent: * Disallow: /cgi-bin Disallow: /search Disallow: /query.html Disallow: /omb/search Disallow: /omb/query.html Disallow: /expectmore/search Disallow: /expectmore/query.html Disallow: /results/search Disallow: /results/query.html Disallow: /earmarks/search Disallow: /earmarks/query.html Disallow: /help Disallow: /360pics/text Disallow: /911/911day/text Disallow: /911/heroes/text And it goes on like that for almost 2400 lines! Here's the new Obamafied robots.txt file: User-agent: * Disallow: /includes/ That's it! BTW, the robots.txt file tells search engines what to include and not include in their indexes. (thx, ian) By Jason Kottke • Jan 20, 2009 at 02:29 pm ------------------------------------ Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/videoblogging/ <*> Your email settings: Individual Email | Traditional <*> To change settings online go to: http://groups.yahoo.com/group/videoblogging/join (Yahoo! ID required) <*> To change settings via email: mailto:[email protected] mailto:[email protected] <*> To unsubscribe from this group, send an email to: [email protected] <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
