+1 would be a good step up
> On May 22, 2015, at 3:19 PM, Karl Klashinsky <[email protected]> wrote: > > I find the way that CI displays test results to be quite challenging to > read/decipher, > for the following reasons: > > 1. Sanity results are displayed by using lynx to dump an html file into job > console > 2. Tempest results are displayed by using cat to dump out XML file > 3. In both cases, pipelines of grep, see, cut, etc are used to determine > failure counts > > Since the test results are available in XML files, I'm inclined to add code to > parse those XML files and display the test results in a more brief and > human-readable > format. What I'm proposing: > > A. If there are no failures, then display only a summary, perhaps something > like this: > > ================ > Sanity Test results: > Tests: 28 > Failed: 0 > Time: 3137.304 sec > URL: > http://10.204.216.50/Docs/logs/3.0-9736_2015-05-09_17:46:06/junit-noframes.html > ================ > Tempest Test summary results: > Tests: 165 > Failed: 0 > Time: 745.870 sec > URL: > http://10.204.216.50/Docs/logs/ci-tempest/result-10677,1-215201519543.xml > ================ > > [ The URL's provide full results in human-readable format ] > [ URL's likely on jenkins.opencontrail.org instead of our internal server ] > > B. Display the failed tests, if any. Like this: > > Sanity: > FAIL: TestBasicPolicyConfig.test_policy_to_deny (94.486 sec) > FAIL: TestBasicPolicyModify.test_policy_modify_vn_policy (124.770 sec) > > Tempest: > FAIL: test_add_remove_fixed_ip (39.939 sec) > FAIL: test_create_list_show_delete_interfaces (37.679 sec) > > C. Display the passed tests, if any. > Sanity: > PASS: TestBasicVMVN4.test_vm_add_delete (72.356 sec) > PASS: TestBasicVMVN4.test_vm_file_trf_scp_tests (125.162 sec) > > Tempest: > PASS: test_create_show_delete_security_group_rule (0.883 sec) > PASS: test_list_security_groups (0.054 sec) > > The PASS tests as per (C) are optional IMO because they aren't really of > interest, and are > easily available via URL in summary (A). However, perhaps the per-test time > info is useful > to have in the console log. > > Concerning (B), an additional option: for tempest tests the XML typically > contains > error message for each test, typically a Python traceback. This could be > added to > the output in (B). An example: > FAIL: test_create_list_show_update_delete_floating_ip (0.028 sec) > testcase error log: > Traceback (most recent call last): > testtools.testresult.real._StringException: Empty attachments: > stderr > stdout > > pythonlogging:'': {{{2015-05-21 23:13:07,611 Request > (FloatingIPTestJSON:test_create_list_show_update_delete_floating_ip): 400 > POST http://192.168.0.66:9696/v2.0/floatingips 0.017s}}} > > Traceback (most recent call last): > File "/root/tempest/tempest/api/network/test_floating_ips.py", line > 72, in test_create_list_show_update_delete_floating_ip > floating_network_id=self.ext_net_id, port_id=self.ports[0]['id']) > File "/root/tempest/tempest/services/network/network_client_base.py", > line 144, in _create > resp, body = self.post(uri, post_data) > File "/root/tempest/tempest/services/network/network_client_base.py", > line 71, in post > return self.rest_client.post(uri, body, headers) > File "/root/tempest/tempest/common/rest_client.py", line 209, in post > return self.request('POST', url, extra_headers, headers, body) > File "/root/tempest/tempest/common/rest_client.py", line 419, in > request > resp, resp_body) > File "/root/tempest/tempest/common/rest_client.py", line 468, in > _error_checker > raise exceptions.BadRequest(resp_body) > tempest.exceptions.BadRequest: Bad request > Details: {u'NeutronError': u"Invalid input for floating_network_id. > Reason: '' is not a valid UUID."} > > Please let me know your opinion as to whether this would be an improvement > to your leisurely console-reading time. > > -- > Klash > > > _______________________________________________ > Dev mailing list > [email protected] > http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org _______________________________________________ Dev mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
