Re: [Dev] 0.7 perf goals

[Bryan Stearns]

It's important to run the tests against a consistent "base" repository, whether it's empty or not, and it's important to build that repository using the same sources you're using for Chandler (so, saving off built repositories to use across Chandler versions would scare me). There's "dynamic state" in the repository that strongly influences performance: collection indexes and cached block trees come to mind immediately, and there's probably more; block tree caching has already caused confusion in interpreting performance measurements. It might mean that running the suite means building a specific-sized repository using current sources, then running Chandler against a fresh copy of it for each test (or batch of related tests where we'll accept the possibility of the tests influencing each other).  We might even build several test repositories this way (empty, 100, 1000, 10000 items?) and choose an appropriate-sized one for each test batch so that the execution time is acceptable (say, shooting for a test runtime of less than a minute to keep the whole suite manageable). ...Bryan Heikki Toivonen wrote: Andi Vajda wrote: I was thinking about all the perf tests that start from an empty repository. I considered the usefulness of these tests some while back, with the hope of being able to eliminate them, but decided to leave them. I think they help track down and fix the causes of the fixed perf costs, i.e. costs that are not dependent on the size of the repository. For example, creating new event even with empty repository takes about half a second (compared with up to 4 seconds with 3k events). For both of those cases the ideal goal is <0.1s, and to get there the fixed costs must still come down. If nobody else thinks those tests are useful we could stop running them. Opinions? _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "Dev" mailing list http://lists.osafoundation.org/mailman/listinfo/dev

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev