On Mon, 2008-01-21 at 08:32 -0500, Michael McCandless wrote: > Well that is not good news!! From your results below, it looks like > 2.3 searching is 13.6% slower with hard disks and 8.9% slower with SSD.
As can be seen, it depends on the configuration. But the overall picture is very consistent. > We focused heavily on speeding up indexing in 2.3, but did not expect > searching to slow down. > > Please post the code you're using, and any details about the test. > It seems like your test is a very long running test? It takes a day or so to go through the combinations for a given Lucene version, but the machine I use is dedicated to testing our search- implementation, so it's not a problem to run lengthy tests. If somebody comes up with a test that can help clarify the issue, I'll be happy to run it on the hardware. One could say that the test is fairly unrealistic, as we don't even begin to approach the 340.000 queries before our index is rotated. If we only look at the forst 50.000 queries, the difference in speed for Lucene versions using harddisks is negligible. For SSDs it's quite visible: (sorry about the rough dump below, I'm running out of time) i37 is index size tx is the number of threads txu means one searcher/thread "only" 50.000 queries average response time: metis_15000_RAID1_8GB_i37_t2_l21.log - 55.9 q/sec metis_15000_RAID1_8GB_i37_t2_l22.log - 51.8 q/sec metis_15000_RAID1_8GB_i37_t2_l23.log - 56.3 q/sec metis_15000_RAID1_8GB_i37_t3_l21.log - 54.7 q/sec metis_15000_RAID1_8GB_i37_t3_l22.log - 54.1 q/sec metis_15000_RAID1_8GB_i37_t3_l23.log - 52.9 q/sec metis_15000_RAID1_8GB_i37_t3u_l21.log - 70.0 q/sec metis_15000_RAID1_8GB_i37_t3u_l22.log - 70.1 q/sec metis_15000_RAID1_8GB_i37_t3u_l23.log - 70.3 q/sec metis_flash_RAID0_8GB_i37_t2_l21.log - 279.6 q/sec metis_flash_RAID0_8GB_i37_t2_l22.log - 266.1 q/sec metis_flash_RAID0_8GB_i37_t2_l23.log - 202.3 q/sec metis_flash_RAID0_8GB_i37_t3_l21.log - 138.9 q/sec metis_flash_RAID0_8GB_i37_t3_l22.log - 129.6 q/sec metis_flash_RAID0_8GB_i37_t3_l23.log - 131.9 q/sec metis_flash_RAID0_8GB_i37_t3u_l21.log - 273.0 q/sec metis_flash_RAID0_8GB_i37_t3u_l22.log - 254.0 q/sec metis_flash_RAID0_8GB_i37_t3u_l23.log - 251.8 q/sec The code I'm using relies on a non-trivial amount of classes in order to parse the queries, which I can't provide directly due to legal issues. I've packaged the core files, which can be downloaded at http://wiki.statsbiblioteket.dk/summa/Hardware?action=AttachFile&do=get&target=SBLucenePerformance0.1.zip The getSearcher and getNewSearcher methods differ by reusing the previously created searcher and making a new searcher. The reader for the searcher uses IndexReader.open(indexPath). I'll see if I can find the time to make a self-contained test on friday, but no promises. Toke: > > I don't know if the lower performance is because I'm using the old > > index instead of generating a new one with Lucene 2.3pr2? > > I don't think that should cause the slowdown. Though this was a good > test of backwards compatibility on the index format! The compatibility-part worked great. It was just a matter of changing the classpath. I'll see if I can steal our production-server for a day and create a 2.3 index to rule out the possibility that the slowdown is due to the index-version. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
