Just wanted to clarify this area with some references.
Meta keywords - no - no search engine publically acknowdges that they refer to them.
Meta descriptions - yes - see below - but DMoz is often a factor as well
Meta robots - yes - see below
1. you can use robots.txt OR meta robots:
[quote]Use a robots.txt file or meta tags to control how MSNBot and other web crawlers index your site. The robots.txt file tells web crawlers which files and folders it is not allowed to crawl. The
Web Robots Pages provide detailed information on the robots.txt Robots Exclusion standard. This site may be available in English only.[/quote]
http://search.msn.com/docs/siteowner.aspx?t=SEARCH_WEBMASTER_REF_GuidelinesforOptimizingSite.htm&FORM=WGDD
Yahoo: [quote]
create a "robots.txt" file on your web site to prevent our crawler from indexing your site
add a "noindex" meta tag to your documents [/quote] http://help.yahoo.com/help/us/ysearch/indexing/indexing-13.html
Google: [quote]
robots.txt is a standard document that can tell Googlebot not to download some or all information from your web server.......
......To keep Googlebot from following links on your pages to other pages or documents, you'd place the following meta tag in the head of your HTML document: <META NAME="Googlebot" CONTENT="nofollow"> [/quote]
2. As far as meta description is concerned - Meta Description is still important to MSN and Yahoo!:
[quote]As the MSN Search web crawler MSNBot crawls your website, it analyzes the content on indexed web pages and generates keywords to associate with each we page. Then MSNBot extracts web page content that is highly relevant to the keywords (often sentence segments that contain keywords or information in the
description meta tag) and constructs the website description displayed in search results. [/quote] http://search.msn.com/docs/siteowner.aspx?t=SEARCH_WEBMASTER_CONC_AboutYourSiteDescription.htm
[quote]Pages Yahoo! Wants Included in its index:<snip>Metadata (including title and description) that accurately describes the contents of a web page [/quote]
http://help.yahoo.com/help/us/ysearch/indexing/indexing-14.html
3. Also - Google often also often uses the ODP Dmoz description rather than the Meta Description:
E.g. search Google for w3c http://www.google.com/search?hl=en&q=w3c
W3C - The World Wide Web Consortium
The W3C was founded in October 1994 to lead the World Wide Web to its full
potential by developing common protocols that promote its evolution and ensure ...
The W3C was founded in October 1994 to lead the World Wide Web to its full
potential by developing common protocols that promote its evolution and ensure ...
Check the Dmoz listing: http://dmoz.org/Computers/Internet/Policy/
W3C - The World Wide Web Consortium - The World Wide Web Consortium was created to lead the World Wide Web to its full potential by developing common protocols that promote its evolution and ensure its interoperability.
Now look at the meta description at http://www.w3.org/
<meta name="description" content="W3C's nearly 400 member organizations lead the World Wide Web to its full potential. Founded by Tim Berners-Lee, the Web's inventor. The W3C Web site hosts specifications, guidelines, software and tools. Public participation is welcome. W3C supports universal access, the semantic Web, trust, interoperability, evolvability, decentralization, and cooler multimedia." />
Best
Chris
Cogentis Search Engine marketing & Optimisation
