Author: Alex Barkov Email: [EMAIL PROTECTED] Message: > > Differently to other user, the way I have installed and configured mnogoSearch >3.1.19 it indexes my whole site, although I have excluded a bunch of subdirectories >with a robots.txt file. > > The structure of the robots.txt is correct, in the way: > > # robots.txt for http://www.zambra.com > > User-agent: * > Disallow: /cgi-bin/ > Disallow: /blah1/ > Disallow: /blah2/ > Disallow: /blah3/ > Disallow: /blah4/ > Disallow: /blah5/ > Disallow: /blah6/ > > mnogoSearch indexes everything contained in these directories (checked by doing a >search). The robots table is empty. > I have noticed that when executing a indexing, it reads robots.txt, then the >different directories of the site, and after a while it shows "Killed". I >don't know whether this is normal. > > Thanks in advance for any hints. > > Michael >
What version of mnogosearch, what database and OS do you use? Reply: <http://www.mnogosearch.org/board/message.php?id=3384> ___________________________________________ If you want to unsubscribe send "unsubscribe general" to [EMAIL PROTECTED]
