In line with checking the perf issues between 2.3 and 2.4, I've started looking at how well each release scales. The previous benchmarks were taken from saturating the server with 48 slamd clients starting with 1 thread, and then increasing the number of threads per client across each iteration. I.e., iteration 1 has 48 connections doing queries. Iteration 2 has 96 connections doing queries, etc.
For the scalability test, I switched to 9 slamd clients, incrementing 1 thread per iteration. What I see as a result is that 2.3 scales better, and keeps substantially more consistent results as the number of clients increases, whereas 2.4 gets consistently worse as the number of clients increases. I.e., 2.4 does not scale particularly well. The test ends after 3 consecutive non-improving increments. OL 2.3: Job Iterations 20090805192650-64030272-1 14028.304 Stopped because the maximum duration had been reached 20090805192650-64030272-2 18210.740 Stopped because the maximum duration had been reached 20090805192650-64030272-3 19405.395 Stopped because the maximum duration had been reached 20090805192650-64030272-4 20729.611 Stopped because the maximum duration had been reached 20090805192650-64030272-5 21265.811 Stopped because the maximum duration had been reached 20090805192650-64030272-6 21481.113 Stopped because the maximum duration had been reached 20090805192650-64030272-7 21621.295 Stopped because the maximum duration had been reached 20090805192650-64030272-8 21670.312 Stopped because the maximum duration had been reached 20090805192650-64030272-9 21665.673 Stopped because the maximum duration had been reached 20090805192650-64030272-10 21627.581 Stopped because the maximum duration had been reached 20090805192650-64030272-11 21581.896 Stopped because the maximum duration had been reached OL 2.4: Job Iterations 20090805154205-07048070-1 13382.487 Stopped because the maximum duration had been reached 20090805154205-07048070-2 18201.523 Stopped because the maximum duration had been reached 20090805154205-07048070-3 17137.449 Stopped because the maximum duration had been reached 20090805154205-07048070-4 16470.136 Stopped because the maximum duration had been reached 20090805154205-07048070-5 16127.879 Stopped because the maximum duration had been reached And again, it is painfully clear how much slower OL 2.4 is than OL 2.3 as the number of clients increases. OL 2.4 maxes out at 18 querying clients. OL 2.3 maxed out at 81 querying clients, and although it started to decline after that point, the decline was very gradual compared to OL 2.4. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
