Webboard: Excluding a directory

2001-02-27 Thread Alexander Barkov 

Author: Alexander Barkov
Email: [EMAIL PROTECTED]
Message:
 Hi everybody,
 
 How can I exclude a directory from de indexing process?
 


Using allow/disallow commands, or using robots.txt file 
if you are indexing your own site.

Reply: http://search.mnogo.ru/board/message.php?id=1541

___
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]

Webboard: Excluding a directory

2001-02-27 Thread nobody 

Author: Antonio Jose Moran
Email: [EMAIL PROTECTED]
Message:
Thanks,

I have just used allow/disallow, but not robots.txt (where can I find information 
about "robots.txt" I have udmsearch 3.0.23)

The problem is that I want to index a subdirectory that belongs to a directory 
that is indexed:

/somethingtoindex1/ - will be indexed
...
/somethingtoindexn/ - will be indexed

But:

/somethingtoindex1/anythingelse/  - won't be indexed
...
/somethingtoindexn/anythingelse/  - won't be indexed


I don't know the sintax to write:

*/anythingelse/*   - don't index

Thanks a lot,

Antonio José

Reply: http://search.mnogo.ru/board/message.php?id=1548

___
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]

Webboard: Excluding a directory

2001-02-27 Thread nobody 

Author: Alexander Barkov
Email: [EMAIL PROTECTED]
Message:
Robots exclusion standard can be found here:
http://info.webcrawler.com/mak/projects/robots/robots.html

Check indexer.conf-dist how to disallow subdirectories.

 Thanks,
 
 I have just used allow/disallow, but not robots.txt (where can I find 
information about quot;robots.txtquot; I have udmsearch 3.0.23)
 
 The problem is that I want to index a subdirectory that belongs to a directory 
that is indexed:
 
 /somethingtoindex1/ -gt; will be indexed
 ...
 /somethingtoindexn/ -gt; will be indexed
 
 But:
 
 /somethingtoindex1/anythingelse/  -gt; won't be indexed
 ...
 /somethingtoindexn/anythingelse/  -gt; won't be indexed
 
 
 I don't know the sintax to write:
 
 */anythingelse/*   -gt; don't index
 
 Thanks a lot,
 
 Antonio José

Reply: http://search.mnogo.ru/board/message.php?id=1554

___
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]