- - - - - - - - - - - - - - - - - - - - - - - - - - - - Name: Maxime Subject: Re: bugs in indexer (importing)
By default, DataparkSearch checks urls against robots.txt rules before insert them into database. You may switch this by "FastHrefCheck yes" command in the latest snapshot. - - - - - - - - - - - - - - - - - - - - - - - - - - - - Read the full topic here: http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=03;topic_id=1228476784
