The problem is probably to dynamically update the robot.txt file when content (read new page) is added. Maybe it helkps the content is stored in a OBJECT and displayed only when a certain programmatically boolean is set
On 16 nov 2008, at 17:33, Sergiu Dumitriu wrote: >>>> Besides that, How can I control what content is indexed by external >>>> search engines (like Google, Yahoo) >>>> I have specific content that I would like to exclude from indexes, >>>> although it is on public accessible pages. >>>> In our system it is accessible by clcking a disclosure (because it >>>> contains medical advise) >> >> I think you need to use a robots.txt file with the names of the >> pages you >> want to exclude from indexation. >> > > Something like: http://platform.xwiki.org/robots.txt > > -- > Sergiu Dumitriu > http://purl.org/net/sergiu/ > _______________________________________________ > users mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/users _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
