Something that's currently just far to manual to figure out is how testr is functioning in the parallel case, which tests are running on which process, and when does one of the pipelines run out of tests. The testr assumption that you don't want output unless there is a failure is a really bad assumption, because it's important to know that things are progressing, especially in complicated systems.
In tempest we actually have a wrapper that gives us a stream. But it's
got a couple of fundamental flaws.
First - it is very clear that the expected behavior of start / end of
tests doesn't work:
2014-01-24 00:49:59.005 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_image_with_min_ram[gate]
2014-01-24 00:49:59.005 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_image_with_min_ram[gate]
... ok
2014-01-24 00:49:59.121 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_remote_image[gate]
2014-01-24 00:49:59.122 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_remote_image[gate]
... ok
2014-01-24 00:49:59.932 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_then_upload[gate]
2014-01-24 00:49:59.933 |
tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_then_upload[gate]
... ok
These tests take > 1msec, all that's happening is the start message is
being printed with the end message, which is kind of useless.
Question #1: how do we either make start message be when things actually
start? or turn it off entirely?
Second - it's not entirely obvious how unbalanced the pipelines are, and
that would be extremely useful to see in the output. A starting point
for a UI would be to add...
" " x (5 * pipelinenumber)
To the test results. Then you'd have a visual waterfall to the front of
the tests, and could clearly see pipeline differences.
Question #2: where would this be injectable in the stream of testtools /
subunit / testr / unittest?
-Sean
--
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net
signature.asc
Description: OpenPGP digital signature
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
