- - - - - - - - - - - - - - - - - - - - - - - - - - - - Name: Torvas A Subject: HOWTO: Config indexer.conf for whole web
Hi: Was this really thrid. I am planning something similar tho for a much smaller part of the web .. mainly all information about a city .... Regards, A Tor > At 08:34:59 13/04/07, Robert wrote: >Hey all, > >I installed dpsearch properly. > >I have the databases created. > >I have a HUGGGGGEEE amount of db storage (well over 2 TB) > >All I want to do is give dpsearch a list of seed urls, and have it crawl the >web. > >The whole web. No restrictions, no list of allowed servers. >Just-The-Whole-Darn-Web. Period. > >Can someone please give me the proper indexer.conf settings to do this? >Pleeeeaaassseee? I understand most of the configuration files. But for some >reason I cannot get indexer to crawl outside of the seed domains I'm giving it. > >Any help would be great. If you could just tell me 'Server' , 'URL' or >'Realm' settings, I would be ETERNALLY grateful. > >I am willing to grovel and beg, if you want. After two years of marriage, I >am a professional at begging. =P > > > >Thanks in advance, > >Robert > - - - - - - - - - - - - - - - - - - - - - - - - - - - - Read the full topic here: http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=02;topic_id=1176438899
