Hello Lucy hackers, I've started playing around a bit with Lucy, and I have to say it's really, really nice!
However I've run into a problem trying to increase performance using sharding with SearchServer / ClusterSearcher. In my tests, I get close to a tenfold *drop* in performance using a few local shards (say, 3 shards on a 4 core server). I have attached a small test program, which only uses one shard to demonstrate the problem. Provided an available index and appropriate modification to the hard coded query, the program can be run as ./test_shard.pl to test SearchServer / ClusterSearcher, and as ./test_shard.pl 0 to test IndexSearcher on the same index. BTW, my index contains about 5 million documents. While I would expect some overhead using SearchServer / ClusterSearcher, the close to tenfold increase in search time I experience does seem rather excessive. I'd need an exorbitant amount of shards just to get the same performance as by using a single index, if I'd ever get there... So, I'm wondering whether I'm just doing something really stupid, or whether I've run into a problem in Lucy. Help!? :-) If there is anything I can do to help isolate any possible problem, please do tell me so (e.g. strace / perl profiling / ...) In case that hase any relevance, the test system is a (really old) RedHat 5.8 x86_64 system with two dual-core AMD Opteron 2218 processors and 4GB RAM. -- Best regards, Dag Lem
