Re: [WSG] SEO, fact or fiction and myths
On Sat, 8 Mar 2008 16:11:53 -0800, tee wrote: Anybody knows about this? The robots text is good for search robots, but I read from somewhere, that robots text no longer is needed when Google Sitemap is implemented for the site. For Google bots, there are some elements of Google Sitemaps that interact with the robots.txt file. No other bots have access to Google Sitemaps info. You still want a robot.txt file even if you are using Google Sitemaps I didn't know robots text was important for accessibility, however I learned from the accessites team that it is. Umm... no - you (or someone) has mixed up robots.txt with something else (not sure what!) robots.txt is generally used to tell bots where they can't go. People who benefit from high accessability in a site are not bots! :) I can't think of *any* overlap between accessability and robots.txt True statement: 'if it is accessible then a bot will be able to crawl it well' (a general rule, anyway) False statement: 'if you block a bot with robots.txt then the site is not accessible' IMHO warmly, Lea -- Lea de Groot Elysian Systems Brisbane, Australia *** List Guidelines: http://webstandardsgroup.org/mail/guidelines.cfm Unsubscribe: http://webstandardsgroup.org/join/unsubscribe.cfm Help: [EMAIL PROTECTED] ***
Re: [WSG] SEO, fact or fiction and myths
I didn't know robots text was important for accessibility, however I learned from the accessites team that it is. Tee, The reasons we (Accessites) look for a robots.txt file is because it keeps honest bots from wasting their time and your bandwidth indexing directories/files you don't want indexed. We don't look at this as part of a web accessibility requirement. Our focus is on quality sites for which accessibility must be an integral part. Thus, we like to see things like a robots.txt file, PICS label, semantics, good looks, and more, of course. Regarding a site map, that we like to see for accessibility and not for bots at all. A site map is important to accessibility as some user will seek out a site map right away to grasp a site's overview and offerings. For some users, this is the best way to begin the exploration of a site. In my opinion, html site maps don't have anything to do with indexing other than just being another indexable page. It is my understanding, though, that an XML site map can help indexing but being that I've never used one or looked into it much, I can neither confirm or deny this. Sorry for the misunderstanding. Respectfully, Mike Cherim *** List Guidelines: http://webstandardsgroup.org/mail/guidelines.cfm Unsubscribe: http://webstandardsgroup.org/join/unsubscribe.cfm Help: [EMAIL PROTECTED] ***
RE: [WSG] SEO, fact or fiction and myths
Hi Michael, That seems incredibly arbitrary when a robots.txt is purely optional - especially as the default spider behavior is to index all unless told otherwise. So you're penalizing people by having your robot behave in the opposite manner? And regarding PICS labels, most people don't know how to set them or don't have the requisite server access. How do you justify these? Cheers, John -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Mike at Green-Beast.com Sent: Monday, 10 March 2008 12:52 AM To: wsg@webstandardsgroup.org Subject: Re: [WSG] SEO, fact or fiction and myths I didn't know robots text was important for accessibility, however I learned from the accessites team that it is. Tee, The reasons we (Accessites) look for a robots.txt file is because it keeps honest bots from wasting their time and your bandwidth indexing directories/files you don't want indexed. We don't look at this as part of a web accessibility requirement. Our focus is on quality sites for which accessibility must be an integral part. Thus, we like to see things like a robots.txt file, PICS label, semantics, good looks, and more, of course. Regarding a site map, that we like to see for accessibility and not for bots at all. A site map is important to accessibility as some user will seek out a site map right away to grasp a site's overview and offerings. For some users, this is the best way to begin the exploration of a site. In my opinion, html site maps don't have anything to do with indexing other than just being another indexable page. It is my understanding, though, that an XML site map can help indexing but being that I've never used one or looked into it much, I can neither confirm or deny this. Sorry for the misunderstanding. Respectfully, Mike Cherim *** List Guidelines: http://webstandardsgroup.org/mail/guidelines.cfm Unsubscribe: http://webstandardsgroup.org/join/unsubscribe.cfm Help: [EMAIL PROTECTED] *** *** List Guidelines: http://webstandardsgroup.org/mail/guidelines.cfm Unsubscribe: http://webstandardsgroup.org/join/unsubscribe.cfm Help: [EMAIL PROTECTED] ***
Re: [WSG] SEO, fact or fiction and myths
That seems incredibly arbitrary when a robots.txt is purely optional - especially as the default spider behavior is to index all unless told otherwise. So you're penalizing people by having your robot behave in the opposite manner? And regarding PICS labels, most people don't know how to set them or don't have the requisite server access. How do you justify these? John, We don't necessarily penalize for not having one, we just credit for having one (offering one is not part of our criteria [1]). It's something we like to see. For the reasons I stated: we grade a site on many levels, and we see that providing a robots.txt as a positive thing that helps make a site/domain complete. Same with a PICS label, it's not a requirement, though I believe a PICS label can actually help with access in that some schools districts won't allow network access to site that doesn't claim to be appropriate for the level of the students the system serves. Regarding requisite server access I don't understand. The PICS label is put into the head of the document. If a developer doesn't understand how to get a PICS label or can't add one to the head and don't have access to such, I doubt they'd be submitting a site for possible awarding. But, regardless, the main point of my reply was to clarify that the robots.txt file has no bearing on the site's accessibility (that I'm aware of) and that's it's just one of the many things we look for in a quality submission. Cheers. Mike [1] http://accessites.org/site/criteria/ *** List Guidelines: http://webstandardsgroup.org/mail/guidelines.cfm Unsubscribe: http://webstandardsgroup.org/join/unsubscribe.cfm Help: [EMAIL PROTECTED] ***
Re: [WSG] SEO, fact or fiction and myths
Nice to hear again about PICS. I use to label all my websites, but I've ofter wondered if I'm the last one using this (and P3P...). djn Mike at Green-Beast.com wrote: That seems incredibly arbitrary when a robots.txt is purely optional - especially as the default spider behavior is to index all unless told otherwise. So you're penalizing people by having your robot behave in the opposite manner? And regarding PICS labels, most people don't know how to set them or don't have the requisite server access. How do you justify these? John, We don't necessarily penalize for not having one, we just credit for having one (offering one is not part of our criteria [1]). It's something we like to see. For the reasons I stated: we grade a site on many levels, and we see that providing a robots.txt as a positive thing that helps make a site/domain complete. Same with a PICS label, it's not a requirement, though I believe a PICS label can actually help with access in that some schools districts won't allow network access to site that doesn't claim to be appropriate for the level of the students the system serves. Regarding requisite server access I don't understand. The PICS label is put into the head of the document. If a developer doesn't understand how to get a PICS label or can't add one to the head and don't have access to such, I doubt they'd be submitting a site for possible awarding. But, regardless, the main point of my reply was to clarify that the robots.txt file has no bearing on the site's accessibility (that I'm aware of) and that's it's just one of the many things we look for in a quality submission. Cheers. Mike [1] http://accessites.org/site/criteria/ -- - Dejan Kozina Web design studio Dolina 346 (TS) - I-34018 Italy tel./fax: +39 040 228 436 - cell.: +39 348 7355 225 skype: dejankozina http://www.kozina.com/ - e-mail: [EMAIL PROTECTED] *** List Guidelines: http://webstandardsgroup.org/mail/guidelines.cfm Unsubscribe: http://webstandardsgroup.org/join/unsubscribe.cfm Help: [EMAIL PROTECTED] ***
Re: [WSG] SEO, fact or fiction and myths
Okay then. What is an example of an accessible robots.txt file? Are you also talking about the site map link you see on large web sites? Angus MacKinnon Infoforce Services http:ééwww.infoforce-services.com It is impossible to rightly govern the world without God and the Bible. George Washington *** List Guidelines: http://webstandardsgroup.org/mail/guidelines.cfm Unsubscribe: http://webstandardsgroup.org/join/unsubscribe.cfm Help: [EMAIL PROTECTED] ***
