Re: [lucy-user] SearchServer / ClusterSearcher - massive performance hit

[Dag Lem]

Dag Lem <d...@nimrod.no> writes:

[...]

> I have attached a small test program, which only uses one shard to
> demonstrate the problem. Provided an available index and appropriate
> modification to the hard coded query, the program can be run as
> ./test_shard.pl to test SearchServer / ClusterSearcher, and as
> ./test_shard.pl 0 to test IndexSearcher on the same index. BTW, my
> index contains about 5 million documents.

[...]

My attachment seems to have been dropped; please find the test program
below.

-- 
Best regards,

Dag Lem


#!/usr/bin/perl

use strict;
use warnings;

use Lucy::Search::IndexSearcher;
use LucyX::Remote::SearchServer;
use LucyX::Remote::ClusterSearcher;
$SIG{CHLD} = "IGNORE";

# Pass 0 to test normal search.
my $cluster = !@ARGV || $ARGV[0] eq '1';

my $searcher = Lucy::Search::IndexSearcher->new(index => "/db/disk1/lucy/full");
my $schema = $searcher->get_schema();

if ($cluster) {
    if (fork() == 0) {
        # Start server in child process.
        my $search_server = LucyX::Remote::SearchServer->new(searcher => 
$searcher);
        $search_server->serve(port => "8000");
        exit;
    }

    # Give server some time to start up.
    sleep(1);

    $searcher = LucyX::Remote::ClusterSearcher->new(
                                               schema => $schema,
                                               shards => [ "localhost:8000" ],
                                                    );
}

my $query_parser = Lucy::Search::QueryParser->new(schema => $schema);
$query_parser->set_heed_colons(1);

for (1..10000) {
    my $q = $query_parser->parse("fornavn:(dag) AND etternavn:(lem)");

    my $hits = $searcher->hits(
                               query      => $q,
                               offset     => 0,
                               num_wanted => 100,
                               );
#     while (my $hit = $hits->next) {
#       print "$hit->{fodselsdato}\t$hit->{navn}\n";
#     }
}

# Stop server.
$searcher->terminate() if $cluster;