Good summary Martin. To this day, I'm not convinced there was ever any performance regression - but throw some aberrant graphs at management without sufficient context and things get escalated quickly...
One of the issues with testing across versions is considering the baseline. In the case of EAP 6.0 vs 6.1, the non-clustered performance is actually faster in 6.1. When this is the case, the cross version performance comparison should really include a handicap for 6.1, meaning that we shouldn't be comparing clustering throughput given the same number of concurrent clients, but rather the clustering throughput given the same "clustering load", which in 6.1, is actually at a lower number of clients than 6.0. When considering the handicap, the perf numbers are much more in line with the numbers I've seen from Infinispan 5.1 vs 5.2 directly. On Tue, 2013-05-07 at 14:28 +0200, Martin Gencur wrote: > Kind of....their configuration is replicated & embedded cache + async > replication + batching + file store + passivation off + write-heavy > > We, indeed, do *not* test this configuration for regression (especially > the batching + file store + write-heavy). And even if we did, we > wouldn't spot the problem easily. The performance regression was > revealed in a client stress test where the number of clients is > gradually increased. The peak throughput in their tests was higher for > 5.2 than for 5.1 - this was OK. They complained about the max. number of > clients that can be handled and the throughput after the peak (not > before it). And this has to do with JGroups tuning (thread pools etc.) > This is a very specific test case and issue and can be revealed only by > the client stress tests which are the most time consuming. > > Anyway, we'll try to incorporate this scenario into the regular > regression performance testing we're working on so that we catch these > problems sooner than the respective version of ISPN gets to AS. > > Martin > > > Dne 2.5.2013 19:43, Galder Zamarreño napsal(a): > > My gut feeling is: AS uses some cache configuration that's affecting > > performance and it's not currently in the test plan? > > > > Cheers, > > > > On May 2, 2013, at 12:41 PM, Radim Vansa <[email protected]> wrote: > > > >> Hi, I've checked it on 4 nodes, stress test 15 minutes with 80% writes. > >> Both reads and writes have improved in 5.2 > >> > >> READS/sec 247k -> 263k > >> WRITES/sec 4547 -> 4771 (note: this is average, coordinator has about 2x > >> more writes as it is the lock owner for all locks in replicated mode) > >> that makes > >> > >> Paul, how many operations within transaction do you use? The results above > >> are from single operation (put/get) per transaction (we have improvement > >> in TRANSACTIONS/sec 5655 -> 5936 with this setting), I'll try to rerun > >> with more ops/transaction as well. > >> > >> Radim > >> > >> ----- Original Message ----- > >> | From: "Paul Ferraro" <[email protected]> > >> | To: "Mircea Markus" <[email protected]> > >> | Cc: "infinispan" <[email protected]>, "Paul Ferraro" > >> <[email protected]> > >> | Sent: Wednesday, May 1, 2013 5:05:39 PM > >> | Subject: Re: [infinispan-dev] write-heavy performance degradation > >> between 5.1 and 5.2 > >> | > >> | FYI - here's a summary of EAP 6.0 vs 6.1 performance: > >> | https://bugzilla.redhat.com/attachment.cgi?id=741896 > >> | > >> | and here's the relevant BZ: > >> | https://bugzilla.redhat.com/show_bug.cgi?id=956988 > >> | > >> | On Wed, 2013-05-01 at 15:55 +0100, Mircea Markus wrote: > >> | > Hi Martin, > >> | > > >> | > Paul mentioned a severe degradation in performance between 5.1 and 5.2 > >> for > >> | > replicated + embedded + transactional + pessimistic caches, for > >> | > write-heavy access. Do we have any tests we can run to check this? > >> | > > >> | > Cheers, > >> | > >> | > >> | _______________________________________________ > >> | infinispan-dev mailing list > >> | [email protected] > >> | https://lists.jboss.org/mailman/listinfo/infinispan-dev > >> | > >> _______________________________________________ > >> infinispan-dev mailing list > >> [email protected] > >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > -- > > Galder Zamarreño > > [email protected] > > twitter.com/galderz > > > > Project Lead, Escalante > > http://escalante.io > > > > Engineer, Infinispan > > http://infinispan.org > > > > > > _______________________________________________ > > infinispan-dev mailing list > > [email protected] > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
