The Hermit Hacker wrote:
>
> %sbin/indexer -h
>
> indexer from mnogosearch-3.1.11/PgSQL
> http://search.mnogo.ru (C) 1998-2000, LavTech Corp.
>
> I have it disallow'd in my config file:
>
> # Exclude cgi-bin and non-parsed-headers
> Disallow /cgi-bin/ \.cgi /nph \?
>
> yet, if I run the indexer, its indexing a *load* of cgi URLs, of the format:
>
> Indexer[83032]: [1]
>http://www.postgresql.org/cgi/cvsweb.cgi/pgsql/configure.in.diff?r1=1.59&r2=1.45&sortby=rev&only_with_tag=MAIN
>
> why? and how can I get it to stop?
In 3.1.9 we've changed default format of Disallow command to take string
wildcards, i.e. string with ? and * signs. They are much faster
comparing with
regular expression match.
You have to use either:
Disallow regex /cgi-bin/ \.cgi /nph \?
or
Disallow */cgi-bin/* *.cgi* *?*
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
