Forwarded this on to an old friend who works overseas now who has experience building this sort of thing.
>> His reply Yeah I remember this stuff - bad memories mate :) Another option might be to use JavaScript to build navigation menu items - the crawler wouldn't be able to interpret the script. It kinda sucks to do this and it might break accessibility requirements, screen readers and a bunch of other things. The solution you describe works, but you still need to provide some sort of index of all site content if there's no other way for the crawler to follow links. MondoSearch (remember that?!) used to support a <noindex>...</noindex> tag that told the crawler to ignore a chunk of HTML. That's nice because links can still be followed and the content is not indexed, but the tag is not conformant to any HTML standards and wouldn't validate. A nice idea would be to add an attribute to an existing tag. Something like: <div rel="noindex">...</div>. I *think* this is standards compliant but would have to check. Certainly Google are pushing a rel="nofollow" attribute and some other search engines are catching on. It would be nice if the SharePoint search could support these. I haven't looked into it, but it might be possible to extend (or replace) the existing iFilters for HTML to add this behaviour. I guess hiding the navigation controls is the best approach right now. Even though that still kinda sucks! I'll stop with the ranting. You shouldn't have brought back those memories :) -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Paul Culmsee Sent: Thursday, 3 July 2008 4:25 PM To: [email protected] Subject: RE: [OzMOSS] Custom Menu text appearing in search results Nice work Sezai. I find WGET excellent for this purpose also (for what it's worth many sites abuse google crawls using UserAgent methods like this) -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Sezai KOMUR Sent: Thursday, 3 July 2008 3:22 PM To: [email protected] Subject: RE: [OzMOSS] Custom Menu text appearing in search results See this : http://sharepoint.microsoft.com/blogs/LKuhn/Lists/Posts/Post.aspx?ID=49 You could possibly implement your own exclusion system by building in "cloaking" for your navigation controls. http://en.wikipedia.org/wiki/Cloaking 1. Implement code in your navigation control to detect the User Agent string of the current request which will determine if the page is being indexed by the Enterprise Search Crawler. You can get the crawler user agent string from the following registry key. [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\12.0\Search\Global\Gathering Manager] "UserAgent"="Mozilla/4.0 (compatible; MSIE 4.01; Windows NT; MS Search 5.0 Robot)" 2. If the user agent matches the above than this.Visible = false; in your navigation control will hide it completely and the text will not be indexed. It's quite complicated to test and implement this sort of thing - Firefox has a User Agent Switcher add-on which you can use to test your code is working correctly. https://addons.mozilla.org/en-US/firefox/addon/59 ------------------------------------------------------------------- OzMOSS.com - to unsubscribe from this list, send a message back to the list with 'unsubscribe' as the subject. Powered by mailenable.com ------------------------------------------------------------------- OzMOSS.com - to unsubscribe from this list, send a message back to the list with 'unsubscribe' as the subject. Powered by mailenable.com
