It is strongly linked to our application. I think it would be simpler to write new test cases to include them with others. Ideally, the results (e.g. the throughput) could be plotted automatically by Jenkins with the Plot plugin (https://wiki.jenkins-ci.org/display/JENKINS/Plot+Plugin) to easily check differences between builds.
Laurent On Wed, Jun 13, 2012 at 10:57 PM, Andy Seaborne <[email protected]> wrote: > On 13/06/12 16:06, Laurent Pellegrino wrote: >> >> Hello, >> >> I also noticed the same thing by updating jena-tdb only. I switched >> from version 0.9.0-incubating to snapshots on 6 June and I noticed >> around ~50% decrease on one of our benchmark which is read/write >> intensive. >> >> Kind Regards, >> >> Laurent > > > Can you share the benchmark system? > > Andy > > >> >> On Wed, Jun 13, 2012 at 4:33 PM, Simon Helsen<[email protected]> wrote: >>> >>> before you ask, of course, we are looking into whether this is something >>> on our end. We only made very few changes, but we should be able to >>> isolate this. I'll report on that once I know for sure. My question was >>> more whether anyone knew of any changes in tx that could affect >>> concurrent >>> performance negatively since May 15 >>> >>> thanks >>> >>> Simon >>> >>> >>> >>> >>> >>> From: >>> Simon Helsen/Toronto/IBM@IBMCA >>> To: >>> [email protected] >>> Date: >>> 06/13/2012 10:09 AM >>> Subject: >>> concurrency and slower results >>> >>> >>> >>> hi everyone, Andy especially, >>> >>> it looks like some of the changes to TDB since May 15 have negatively >>> affected the concurrent behavior of Tx. To clarify, we are doing some >>> serious internal performance testing on a product which is based on >>> Jena/TDB. We initially did these runs to compare Tx (2.7.1 snapshot of >>> May >>> >>> 15) with our old TDB implementation (which used a conservative locking >>> model). The results were good, especially with long-running/expensive >>> index update operations. >>> >>> Now, I had asked the performance team to rerun their tests with a new >>> build of the product based on the release candidate (of last weekend) and >>> they are reporting a concerning degradation. Compared to the May 15 TDB, >>> they are reporting 12-15% decrease in query performance when doing a >>> "light update load" and about 40% query and 40% update performance >>> degradation when doing a "heave update load". >>> >>> That is quite a serious regression. The question is what changed since >>> May >>> >>> 15 that could have such a bad effect on performance. The tests included >>> only 25 concurrent users on about 15 million triples. Yet, I wonder if >>> some of the recent lock changes (see JENA-252) are responsible >>> >>> thanks >>> >>> Simon >>> >>> >>> >
