It seems we have finally solved the stability problems that have plagued the automated performance tests (http://builds.osafoundation.org/tinderbox/Chandler/status.html). They have now been running continuously for the past couple of days without interruption. Big thanks to Donn for fixing the issues as we found them, and to bear for being there to restart the boxes over and over as we worked on the issues, and of course writing the framework to run these tests on Tbox
We'd like to keep the boxes running continuously without problems. What this means is that if you see any of the Tinderbox machines turn orange or red, or not produce any results in an overly long period, you should treat it as a tree breakage. If you checked in during a window that could have broken things, please investigate (check with the release team first). Some of the results from the automated tests are somewhat suspect, and we are working on those. For example, tests #7, #8 and switch and stamp with large calendar seem to report suspiciously long times. We suspect part of the reported time is actually the initial UI refresh after the 3k calendar import which we should not be measuring. On the other hand, test #9 reports suspiciously fast numbers (but this could be due to us being able to take advantage of caches). You may have noticed that we now have scrolling speed tests automated as well (thanks Katie!). The table scrolling case is explicitly out of 0.6. Calendar scrolling is almost within our acceptable range. The results table now shows the time at which the latest results were processed. The table itself is regenerated every 15 minutes, so new results should appear within that time after any of the p_* Tboxes finishes a cycle. In case you haven't noticed, you can click on the links in the results table, and it will take you to a page that shows all the results for the last day, and draws trend graphs for you. -- Heikki Toivonen
signature.asc
Description: OpenPGP digital signature
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "Dev" mailing list http://lists.osafoundation.org/mailman/listinfo/dev
