Toke Eskildsen wrote:

On Sun, 2008-01-20 at 05:44 -0500, Michael McCandless wrote:
These results are very interesting.  With 3 threads on SSD your
searches run 87% faster if you use 3 IndexSearchers instead of
sharing a single one.

That is my observation, yes. Please note that this is with Lucene 2.1.
I've tried with Lucene 2.3pr2 with the old index though. The relative
results are the same, although overall performance is quite a bit lower.

Well that is not good news!! From your results below, it looks like 2.3 searching is 13.6% slower with hard disks and 8.9% slower with SSD.

We focused heavily on speeding up indexing in 2.3, but did not expect searching to slow down.

Please post the code you're using, and any details about the test. It seems like your test is a very long running test?

== Lucene 2.1 - Harddisks (2 * 15000 RPM in software RAID 1) ==
1 thread,  1 searcher:  109 queries/sec
2 threads, 1 searcher:  118 queries/sec
2 threads, 2 searchers: 157 queries/sec
3 threads, 1 searcher:  111 queries/sec
3 threads, 3 searchers: 177 queries/sec
4 threads, 1 searcher:  108 queries/sec
4 threads, 4 searchers: 169 queries/sec

== Lucene 2.1 - SSD (2 * 32 GB Samsung in software RAID 0) ==
1 thread,  1 searcher:  193 queries/sec
2 threads, 1 searcher:  295 queries/sec
2 threads, 2 searchers: 357 queries/sec
3 threads, 1 searcher:  197 queries/sec
3 threads, 3 searchers: 369 queries/sec
4 threads, 1 searcher:  192 queries/sec
4 threads, 4 searchers: 302 queries/sec

== Lucene 2.3pr2 - Harddisks (2 * 15000 RPM in software RAID 1) ==
1 thread,  1 searcher:   93 queries/sec
2 threads, 1 searcher:  112 queries/sec
2 threads, 2 searchers: 135 queries/sec
3 threads, 1 searcher:   96 queries/sec
3 threads, 3 searchers: 153 queries/sec
4 threads, 1 searcher:   98 queries/sec
4 threads, 4 searchers: 150 queries/sec

== Lucene 2.3pr2 - SSD (2 * 32 GB Samsung in software RAID 0) ==
1 thread,  1 searcher:  171 queries/sec
2 threads, 1 searcher:  261 queries/sec
2 threads, 2 searchers: 325 queries/sec
3 threads, 1 searcher:  185 queries/sec
3 threads, 3 searchers: 336 queries/sec
4 threads, 1 searcher:  182 queries/sec
4 threads, 4 searchers: 280 queries/sec

I don't know if the lower performance is because I'm using the old index
instead of generating a new one with Lucene 2.3pr2?

I don't think that should cause the slowdown. Though this was a good test of backwards compatibility on the index format!

Have you done any profiling to understand where the threads are
waiting when you share one IndexSearcher?

Not at all. The whole threads/searchers-thing in Lucene is quite new to me - in our production system we use synchronized access to the searcher
with only one search at a time.

OK

 EG YourKit can tell you where the threads are waiting...

I'll be happy to investigate further. It'll have to wait until friday
though, if it involves more than a trivial amount of time. I'll also put the (very simple) code online for review. Unfortunately I can't provide
neither the index, nor the queries as I'm not allowed to do so.

OK

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to