Hi Pete!
robots.txt is a file that you can use to tell any RFC compliant robot (search engine, crawler, whatever) what directories / files to ignore. - Danish On 8 Mar 2002, Peter J. Milanese wrote: > Author: Peter J. Milanese > Email: [EMAIL PROTECTED] > Message: > Greetings- > > Although the error below is obvious in nature, I'd like to ask exactly what the >file does, and how to rectify this.... Thanks- > > Indexer[8716]: indexer from mnogosearch-3.1.19/MySQL started with >'/usr/local/mnsearch/etc/indexer.conf' > Indexer[8716]: [1] Load stopword table 'stopword' > Indexer[8716]: [1] http://www.milanese.cc/robots.txt > Indexer[8716]: [1] Server 'http://www.milanese.cc/docsrv/' > Indexer[8716]: [1] Allow NoCase * > Indexer[8716]: [1] Alias: 'file:/usr/local/www/docs/txt' > Indexer[8716]: [1] HTTP/1.0 404 Not found > Indexer[8716]: [1] HTTP/1.0 404 Not found ? 0 > Indexer[8716]: [1] Deleting URL > Indexer[8716]: [1] Done (0 seconds) > > Reply: <http://www.mnogosearch.org/board/message.php?id=4314> > > ___________________________________________ > If you want to unsubscribe send "unsubscribe general" > to [EMAIL PROTECTED] > > -- Danish Qadri Systems Programmer Globix Corporation [EMAIL PROTECTED] ___________________________________________ If you want to unsubscribe send "unsubscribe general" to [EMAIL PROTECTED]
