While searching the archives on www.htdig.org I found the following email
(see below) that should solve my problem. The only issues is I dont know
what the ?M=D parts of the url mean. Also my config of htdig seems to be
finding other combinations such as ?M=A. What are these? Is there now a
better way to stop htdig from including directory indexes in the search
results.
The URLs you are referring to result from alternate index views generated by the web server. You might want to take a look at http://www.htdig.org/FAQ.html#q4.23, especially the last part, for techniques to avoid indexing them.
Jim
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
