Hi!

Thursday, March 21, 2002, 10:12:27 PM, you wrote:

LS> Here is my complete indexer.conf file

LS> # Do not index the index pages
LS> HrefOnly \/$
LS> HrefOnly \/index\.html$ \.index\.html$ \_index\.html$ \/*_index\.html$
LS> HrefOnly \/.*index\.html$
LS> HrefOnly \/1999\.html$ \/2000\.html$ \/1998\.html$ \/1997\.html$ \/1996\.html$
LS> HrefOnly \/1995\.html$ \/2001\.html$ \/2002\.html$ \/film.html$ \/adverts\.html$
LS> HrefOnly \/arts\.listings\/ \/music\.clubs\.html$ \/screens\.film\.html$
LS> HrefOnly \/Film\d\d\.html$ \/screens\.filmtimes\.html$ \/showtimes\.html$
LS> HrefOnly \/clubs\.html \/music\.clubs\/ \/music\.roadshows\.html$

Try this :
HrefOnly Regex \/$
HrefOnly Regex \/index\.html$ \.index\.html$ \_index\.html$ \/*_index\.html$

and so on...

LS> # Get rid of junk
LS> Disallow \.bak$ /nav/ /temp/ /current/ /deep_focus/ /not_current/
LS> Disallow /authors/ /calendar/ /filmvault/ /tmp/
LS> Disallow /weeklywire\.com/ /bin/ /etc/ /lib/ /musicreg/\d
LS> Disallow /cgi-bin/ /cgi/ /images/ /temp/

The same here.
Or you can make it faster without regex such way:
Disallow */authors/* */calendar/* */filmvault/* */tmp/*

-- 
Regards, Sergey aka gluke.

___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]

Reply via email to