- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Name: Maxime
Subject: Re: 2 quick stupid questions
1. It depends what sorting switches you would use with indexer, i.e. -o switch
instructs to sort targets by hops ("mouse clicks" from seeding URLs), -e switch
instructs to sort targets by the next index time, -r swicth intruct to sort
first by randomly selected seeding value to shuffle URL from different servers.
As well, most likely your site or even network would rather blcoked by
Wikipedia team as they don't like be indexed by any robot software.
2. DataparkSearch supports quasi-clustering, i.e. you can setup dedicated PC
(or even a cluster) for SQL database, dedicated PC to run indexer and dedicated
PC to serve serch queries (as well ypu can setup several PC for indexing and
several PC to serve search queries, but you need to use load balancer to
distribute load between your search servers and you need to copy yourself the
search index between seach PC).
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Read the full topic here:
http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=03;topic_id=1220985782
