Title: Message
Hi there...
I'm trying to index everything I have a server
statement for but *also* permit everything that Aspseek finds with .CA after it.
The country I am trying to index has all kinds of tld's besides just .ca
(canada) but I"d like aspseek to discover all .ca sites without having to put a
server entry in the config for each one.
Previous posts were answered
with "use allow/disallow"... well I've tried and can't get it figured out.
Here's a rundown of my config:
DeleteBad yes
DeleteNoServer
no
FollowOutside yes
Server
http://www.nexicom.net/ Server
http://www.cbc.ca/ Server
http://www.canoe.ca/ Server
http://www.nexicom.com/
DisallowNoMatch \/$|\.htm$|\.html$|\.shtml$|\.phtml$|\.php$|\.txt$|\.asp$
Disallow /cgi-bin/ \.cgi /nph
Disallow \?
Disallow \&
I've tried putting DisallowNoMatch
://[^/]*\.ca/
into the config but then it won't index .com
.net etc.
I'm sure this is just an order issue but I can't seem to
figure it out.. am I trying something that can't be done?
Can someone
take the above example and rework it for my purpose??
Thanks very very
much!
Paul