Rob, I agree with you. The test environment configuration would impact the performance test results. Therefore, we should keep the test configuration unchanged between build to build.
I checked performance bugs list in bugzilla. 5 performance bugs are found in automation performance test, such as memory leak and save performance issue. Lots of performance bugs are found in AOO daily usage. Therefore, for volunteers who are not willing to run automation performance, you can also report bugs for any performance issue and it will be helpful for us. On Mon, Feb 25, 2013 at 9:16 PM, Rob Weir <[email protected]> wrote: > On Sun, Feb 24, 2013 at 11:42 PM, Yi Xuan Liu <[email protected]> > wrote: > > Hi, all: > > > > AOO 4.0 will release. Performance plays an important role in software > > quality. Is there any volunteer who want to run performance test? > > > > I've run AOO performance test on my own machines. I've tried on 3 > > platforms: Windows XP, Ubuntu 10.04, Mac Mac OS 10.7.3. The test > configures > > are as follows: > > > > (1) W500; CPU:2.53 GHz; Mem: 3GB; OS: XP SP3 > > (2) Ubuntu; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Ubuntu 10.04 > > (3) Mac; CPU: Interl® Core™ 2 Duo 2 GHz; Mem: 3GB; OS: Mac OS 10.7.3 > > > > I assume the volunteer does not need to have exactly the same machine > type as you had. But they need some stability in the configuration. > A performance test might be run first on the AOO 3.4.1 release to > establish a baseline. Then re-test on a current 4.0 snapshot build. > And then re-run on new dev snapshot builds, maybe once a week. > > The goal is to detect performance regressions early, so developers can fix > it. > > The technical challenge here is to preserve a stable machine > configuration. If the machine changes, because of an OS upgrade, or a > changed hard drive, or a different network environment, or because of > a new anti-virus product, then that confuses things. We need to > "control all the variables". > > One approach to controlling all of the variables is to have a machine > that is used for nothing but performance testing. That way we know > the machine's base performance does not change. > > Another approach is re-run the baseline AOO 3.4.1 performance tests > each week. This is more tolerant of changes in machine configuration, > etc. > > -Rob > > > > The test scenario include: AOO startup, file open, and save. > > > > Volunteers could run performance test on other platforms. > > > > All the automation scripts could be downloaded in AOO project. And it is > > not difficult to set the automation environment. You could follow the > guide: > > http://wiki.openoffice.org/wiki/QA/test_automation_guide > > > > For any questions, be free to contact with me :) >
