+1 On Fri, May 22, 2015 at 3:33 PM, Hampapur Ajay <[email protected]> wrote:
> +1 would be a good step up > > > > > On May 22, 2015, at 3:19 PM, Karl Klashinsky <[email protected]> wrote: > > > > I find the way that CI displays test results to be quite challenging to > read/decipher, > > for the following reasons: > > > > 1. Sanity results are displayed by using lynx to dump an html file into > job console > > 2. Tempest results are displayed by using cat to dump out XML file > > 3. In both cases, pipelines of grep, see, cut, etc are used to determine > failure counts > > > > Since the test results are available in XML files, I'm inclined to add > code to > > parse those XML files and display the test results in a more brief and > human-readable > > format. What I'm proposing: > > > > A. If there are no failures, then display only a summary, perhaps > something like this: > > > > ================ > > Sanity Test results: > > Tests: 28 > > Failed: 0 > > Time: 3137.304 sec > > URL: > http://10.204.216.50/Docs/logs/3.0-9736_2015-05-09_17:46:06/junit-noframes.html > > ================ > > Tempest Test summary results: > > Tests: 165 > > Failed: 0 > > Time: 745.870 sec > > URL: > http://10.204.216.50/Docs/logs/ci-tempest/result-10677,1-215201519543.xml > > ================ > > > > [ The URL's provide full results in human-readable format ] > > [ URL's likely on jenkins.opencontrail.org instead of our internal > server ] > > > > B. Display the failed tests, if any. Like this: > > > > Sanity: > > FAIL: TestBasicPolicyConfig.test_policy_to_deny (94.486 sec) > > FAIL: TestBasicPolicyModify.test_policy_modify_vn_policy (124.770 sec) > > > > Tempest: > > FAIL: test_add_remove_fixed_ip (39.939 sec) > > FAIL: test_create_list_show_delete_interfaces (37.679 sec) > > > > C. Display the passed tests, if any. > > Sanity: > > PASS: TestBasicVMVN4.test_vm_add_delete (72.356 sec) > > PASS: TestBasicVMVN4.test_vm_file_trf_scp_tests (125.162 sec) > > > > Tempest: > > PASS: test_create_show_delete_security_group_rule (0.883 sec) > > PASS: test_list_security_groups (0.054 sec) > > > > The PASS tests as per (C) are optional IMO because they aren't really of > interest, and are > > easily available via URL in summary (A). However, perhaps the per-test > time info is useful > > to have in the console log. > > > > Concerning (B), an additional option: for tempest tests the XML > typically contains > > error message for each test, typically a Python traceback. This could be > added to > > the output in (B). An example: > > FAIL: test_create_list_show_update_delete_floating_ip (0.028 sec) > > testcase error log: > > Traceback (most recent call last): > > testtools.testresult.real._StringException: Empty attachments: > > stderr > > stdout > > > > pythonlogging:'': {{{2015-05-21 23:13:07,611 Request > (FloatingIPTestJSON:test_create_list_show_update_delete_floating_ip): 400 > POST http://192.168.0.66:9696/v2.0/floatingips 0.017s}}} > > > > Traceback (most recent call last): > > File "/root/tempest/tempest/api/network/test_floating_ips.py", > line 72, in test_create_list_show_update_delete_floating_ip > > floating_network_id=self.ext_net_id, > port_id=self.ports[0]['id']) > > File > "/root/tempest/tempest/services/network/network_client_base.py", line 144, > in _create > > resp, body = self.post(uri, post_data) > > File > "/root/tempest/tempest/services/network/network_client_base.py", line 71, > in post > > return self.rest_client.post(uri, body, headers) > > File "/root/tempest/tempest/common/rest_client.py", line 209, in > post > > return self.request('POST', url, extra_headers, headers, body) > > File "/root/tempest/tempest/common/rest_client.py", line 419, in > request > > resp, resp_body) > > File "/root/tempest/tempest/common/rest_client.py", line 468, in > _error_checker > > raise exceptions.BadRequest(resp_body) > > tempest.exceptions.BadRequest: Bad request > > Details: {u'NeutronError': u"Invalid input for > floating_network_id. Reason: '' is not a valid UUID."} > > > > Please let me know your opinion as to whether this would be an > improvement > > to your leisurely console-reading time. > > > > -- > > Klash > > > > > > _______________________________________________ > > Dev mailing list > > [email protected] > > > http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org > > _______________________________________________ > Dev mailing list > [email protected] > http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org >
_______________________________________________ Dev mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
