Author: Alexander Barkov
Email: [EMAIL PROTECTED]
Message:
Did you disallow /forums/moderator.php in robots.txt BEFORE
first indexing or during it?
> I am running vBulletin, a nice PHP and mySQL based BBS system. It uses a bunch of
>.php files to access the features of the board.
>
> I am trying to keep mnoGoSearch from indexing pages like:
>
> /forums/moderator.php?s=232093423908
> /forums/postings.php?s=&action=editthread&threadid=24
>
> Yet still have it index the actual forum messages, which would be at addresses like:
>
> /forums/forumdisplay.php?s=&forumid=20
> /forums/announcement.php?s=&forumid=20
> /forums/showthread.php?s=&threadid=24
>
> I have a robots.txt file set up with names like:
>
> /forums/moderator.php
>
> however these apparently don't match when the URL is compared against the robots
>table.
>
> It appears to me from reading the robots.txt specification that I should be able to
>exclude:
>
> /forums/moderator.php?s=340293840
>
> with the above entry, based on the statement from the spec that:
>
> "An affirmative comparison is one in which each and every character of a path
>root exactly matches the corresponding character in the complete path of the absolute
>url"
>
> The full URL of /forums/moderator.php?s=12345678 would match the path root of
>/forums/moderator.php and would be excluded. This does not happen today.
>
> Is this a bug? A misinterpretation of the robots.txt spec? Any suggestions on
>getting around this?
>
> Thanks,
>
> -Tim
>
>
Reply: <http://search.mnogo.ru/board/message.php?id=2006>
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
