Author: Alexander Barkov
Email: [EMAIL PROTECTED]
Message:
Robots exclusion standard can be found here:
http://info.webcrawler.com/mak/projects/robots/robots.html
Check indexer.conf-dist how to disallow subdirectories.
> Thanks,
>
> I have just used allow/disallow, but not robots.txt (where can I find
>information about "robots.txt" I have udmsearch 3.0.23)
>
> The problem is that I want to index a subdirectory that belongs to a directory
>that is indexed:
>
> /somethingtoindex1/ -> will be indexed
> ...
> /somethingtoindexn/ -> will be indexed
>
> But:
>
> /somethingtoindex1/anythingelse/ -> won't be indexed
> ...
> /somethingtoindexn/anythingelse/ -> won't be indexed
>
>
> I don't know the sintax to write:
>
> */anythingelse/* -> don't index
>
> Thanks a lot,
>
> Antonio Jos�
Reply: <http://search.mnogo.ru/board/message.php?id=1554>
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
