Hello Everybody, I was checking out what pages Google did index, and I found out Google did pretty well! This is because a Wiki is full of links to eachother, making it easy for Google to crawl. However, Google also indexed some pages that do not exist anymore! However, since BoltWire generates a "page does not exist" page instead of a 404 error, the page stays indexed in google!
Now this does not matter much for the everyday user, since this useless page will never get high when somebody searches with google. However, if somebody searches the site (http://www.google.nl/search? q=site:http://markmarijnissen.ruhosting.nl/wiki/robot/) there are some invalid results ("onderdelen" for example). These results could easily be blocked by google, if the "missing page"-page contains the following tag: <meta name="robots" content="noindex"> This tag tells the Google Webcrawler it should not index this page, so the next time Google indexes this page, it will be removed. (I wonder if google will ever reindex the page since no page links to it, but I sure hope so). I would have added this tag myself to the "page missing" page, but I do not know where to find this in the source code. Best regards & enjoying your software, Mark Marijnissing (As a side note: The indexing is a lot harder with for example a wordpress blog, because the blog entries tend to get lost in archive pages and dont have many links to eachother. Fortunaly, there is a plugin that generates a Sitemaps XML file that lists all the URLs google has to index. Google recognises this XML file. Sitemaps for BoltWire still would be nice, but it should be created automatically as the wiki changes - would be a nice plugin!.) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "BoltWire" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/boltwire?hl=en -~----------~----~----~----~------~----~------~--~---
