[
https://issues.apache.org/jira/browse/DERBY-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Knut Anders Hatlen closed DERBY-5206.
-------------------------------------
Resolution: Invalid
I've had a very hard time reproducing this, but at last I remembered that the
test itself was changed in DERBY-5067 around the time that it started
happening. So now, if I run the test with Derby 10.8.1.2, I am able to see that
the throughput is slightly higher if I take derbyTesting.jar from 10.7.1.1
(before DERBY-5067) than it is with derbyTesting.jar from 10.8.1.2.
I only see this if I run the test with the -init parameter to create a new test
table in the same process that's running the actual test. If I run without
-init, so that the test table from a previous run is reused, I don't see any
difference in the throughput.
The changes in DERBY-5067 only affect how the test table is created (the
primary key constraint is now added after the table has been populated, not
when the table is created), which would explain why we only see it when running
with -init. But the exact reason why is a bit of a mystery.
This issue sounds very similar to DERBY-4233. There we saw a drop in throughput
in the table-scan test (same test as here) only when -init was specified, and
the change that triggered the drop was a seemingly harmless change in a code
path only executed when populating the test table.
The theory in that issue was that the runtime compiler in the JVM would
optimize some code path based on statistics collected when populating the test
table. The changes in how the table was populated would change these statistics
slightly, possibly making the optimizer make different choices than before.
Although these choices may be reasonable when populating the table, they may
not be that good when the test starts doing selects instead of inserts, and
it's the performance of the selects we're measuring in the test.
I think this is a theory that would explain what we're seeing in this issue
too, and in any case, since the performance drop happened after a change in the
test itself, there shouldn't be anything to fix in the product code. So I'll
close the issue as invalid.
(After DERBY-4233 I was planning to change the nightly tests to create the test
tables in a separate process to avoid this noise. Still on my todo list...)
> Investigate reduced throughput in some of the nightly performance tests
> -----------------------------------------------------------------------
>
> Key: DERBY-5206
> URL: https://issues.apache.org/jira/browse/DERBY-5206
> Project: Derby
> Issue Type: Task
> Components: SQL
> Affects Versions: 10.8.1.2
> Environment: Solaris 10 5/08 s10x_u5wos_10 X86
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build 1.6.0-b105)
> Java HotSpot(TM) Server VM (build 1.6.0-b105, mixed mode)
> Reporter: Knut Anders Hatlen
>
> The last two months, the nightly performance regression tests
> (http://home.online.no/~olmsan/derby/perf/) have shown results that are
> slightly lower than normal for the following test:
> - Single Record Select, Non-primary non-indexed column -
> http://home.online.no/~olmsan/derby/perf/select_sec_noindex_1y.html
> - TPC-B like load (aka bank_tx) -
> http://home.online.no/~olmsan/derby/perf/tpcb_1y.html
> For example, when averaging the results from 2011-03-01 to 2011-04-26 and
> comparing them to the average for the two preceding months (2011-01-01 to
> 2011-03-01), I see these changes:
> Select, single-threaded: 17.05 tx/s -> 16.81 tx/s (down 1.4%)
> Select, 4 threads: 25.50 tx/s -> 24.70 tx/s (down 3.1%)
> Select, 16 threads: 25.17 tx/s -> 24.47 tx/s (down 2.8%)
> bank_tx, single-threaded: 1931 tx/s -> 1904 tx/s (down 1.4%)
> bank_tx, 4 threads: 2628 tx/s -> 2589 tx/s (down 1.5%)
> bank_tx, 16 threads: 3584 tx/s -> 3519 tx/s (down 1.8%)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
