Author: Tim Hewitt
Email: [EMAIL PROTECTED]
Message:
I am running vBulletin, a nice PHP and mySQL based BBS system. It uses a bunch of .php
files to access the features of the board.
I am trying to keep mnoGoSearch from indexing pages like:
/forums/moderator.php?s=232093423908
/forums/postings.php?s=&action=editthread&threadid=24
Yet still have it index the actual forum messages, which would be at addresses like:
/forums/forumdisplay.php?s=&forumid=20
/forums/announcement.php?s=&forumid=20
/forums/showthread.php?s=&threadid=24
I have a robots.txt file set up with names like:
/forums/moderator.php
however these apparently don't match when the URL is compared against the robots table.
It appears to me from reading the robots.txt specification that I should be able to
exclude:
/forums/moderator.php?s=340293840
with the above entry, based on the statement from the spec that:
"An affirmative comparison is one in which each and every character of a path root
exactly matches the corresponding character in the complete path of the absolute url"
The full URL of /forums/moderator.php?s=12345678 would match the path root of
/forums/moderator.php and would be excluded. This does not happen today.
Is this a bug? A misinterpretation of the robots.txt spec? Any suggestions on getting
around this?
Thanks,
-Tim
Reply: <http://search.mnogo.ru/board/message.php?id=1979>
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
