Thanks Drew. This is really awesome! Finally we can get some kind of comparison
going on, just makes our lives a lot easier.
Question: Do you have some sample output from the test run? How are the
diffs/sucs represented?
Thanks,
Martin
On 8/11/2011 3:06 PM, Drew Fisher wrote:
Round 2:
https://cr.opensolaris.org/action/browse/caiman/drewfish/6987307_2/webrev/
I've incorporated Karen's request to make the test_map dynamic. We now use
os.walk() to traverse the source gate to find /test directories. I've also
added John's fixes as well.
Here's the output from --help:
[mox:tests] > sudo ./slim_regression_test.py --help
Usage: slim_regression_test.py [options] [test[,test]...]
Available tests:
group tests
all: libraries, commands
libraries: install_target, install_utils, install_doc,
install_logging_pymod, netif, liberrsvc, libict_pymod, install_ict, terminalui,
liberrsvc_pymod, install_boot, install_engine, install_manifest_input,
install_logging, install_common, install_manifest, install_transfer, libaimdns
commands: distro_const/checkpoints, js2ai/modules,
ai-webserver, system-config, system-config/profile, auto-install/checkpoints,
auto-install, installadm, text-install
individual tests
ai-webserver: cmd/ai-webserver/test
auto-install: cmd/auto-install/test
auto-install/checkpoints: cmd/auto-install/checkpoints/test
distro_const/checkpoints: cmd/distro_const/checkpoints/test
install_boot: lib/install_boot/test
install_common: lib/install_common/test
install_doc: lib/install_doc/test
install_engine: lib/install_engine/test
install_ict: lib/install_ict/test
install_logging: lib/install_logging/test
install_logging_pymod: lib/install_logging_pymod/test
install_manifest: lib/install_manifest/test
install_manifest_input: lib/install_manifest_input/test
install_target: lib/install_target/test
install_transfer: lib/install_transfer/test
install_utils: lib/install_utils/test
installadm: cmd/installadm/test
js2ai/modules: cmd/js2ai/modules/test
libaimdns: lib/libaimdns/test
liberrsvc: lib/liberrsvc/test
liberrsvc_pymod: lib/liberrsvc_pymod/test
libict_pymod: lib/libict_pymod/test
netif: lib/netif/test
system-config: cmd/system-config/test
system-config/profile: cmd/system-config/profile/test
terminalui: lib/terminalui/test
text-install: cmd/text-install/test
Options:
-h, --help show this help message and exit
-c CONFIG, --config=CONFIG
nose configuration file to use
--suppress-results suppress the printing of the results
--hudson=HUDSON_NUM hudson job number to use as a baseline
Some of the aliases are longer, but that can't be helped.
-Drew
On 8/11/11 9:53 AM, Drew Fisher wrote:
Good morning!
I was hoping to get a quick code review for
6987307 <http://monaco.us.oracle.com/detail.jsf?cr=6987307> Update slim_test to
allow better granularity of test selection
https://cr.opensolaris.org/action/browse/caiman/drewfish/6987307/webrev/
This impacts none of our packaged code, so I figured it was safe to send out
even though we're in a restricted build phase.
I added a new test script which allows two new things:
- the ability to specify subsets of tests to run rather than running every
single test in the gate
- regression testing against prior Hudson results.
The regression testing isn't the smartest algorithm so take that testing with a
grain of salt. By default, the new script looks at the latest Hudson
install_unit_tests job and compares against that. If somebody pushes something
that breaks 50 tests, that will be used as the baseline. I added a flag to the
script which allows you to specify which Hudson job you want to compare against
should something like that happen.
The change to tests.nose was to re-include the /lib/install_ict/test directory.
It went missing at some point ...
Thanks!
-Drew
_______________________________________________
caiman-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
_______________________________________________
caiman-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
_______________________________________________
caiman-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
