Oleksandr Terentiev <otere...@cisco.com> writes:
> Hi Abibal,
>
> In our project we need to analyze a total number of passed and failed
> tests for each packet. To distinguish packets we use lava-test-set
> feature.
> In order to implement that I modified ptest.py and send-to-lava.sh
> scripts. Could you please look at the patch and express your opinion?
> Maybe this code can be added to git.linaro.org/qa/test-definitions.git ?
>
> Best regards,
> Alex
>
> automated/linux/ptest: Analyze each test in package tests
>
> Currently ptest.py analyze only exit code of each package test
> to decide if it passed or not. However, ptest-runner can return
> success code even though some tests failed. So we need to parse
> test output and analyze it.
>
> It also quite useful to see exactly which tests failed. So results are
> recorded for each particular test, and lava-test-set feature is used
> to distinguish packages.
>
> Signed-off-by: Oleksandr Terentiev <otere...@cisco.com>
I gave this a quick test and found a minor problem below...
> diff --git a/automated/linux/ptest/ptest.py
> b/automated/linux/ptest/ptest.py
> index 13feb4d..a28d7f0 100755
> --- a/automated/linux/ptest/ptest.py
> +++ b/automated/linux/ptest/ptest.py
> @@ -84,20 +84,60 @@ def filter_ptests(ptests, requested_ptests, exclude):
>
> return filter_ptests
>
> +def parse_line(line):
> + test_status_list = {
> + 'pass': re.compile("^PASS:(.+)"),
> + 'fail': re.compile("^FAIL:(.+)"),
> + 'skip': re.compile("^SKIP:(.+)")
> + }
> +
> + for test_status, status_regex in test_status_list.items():
> + test_name = status_regex.search(line)
> + if test_name:
> + return [test_name.group(1), test_status]
>
> -def check_ptest(ptest_dir, ptest_name, output_log):
> - status = 'pass'
> + return None
>
> - try:
> - output = subprocess.check_call('ptest-runner -d %s %s' %
> - (ptest_dir, ptest_name),
> shell=True,
> - stderr=subprocess.STDOUT)
> - except subprocess.CalledProcessError:
> - status = 'fail'
> +def parse_ptest(log_file):
> + result = []
>
> - with open(output_log, 'a+') as f:
> - f.write("%s %s\n" % (ptest_name, status))
> + with open(log_file, 'r') as f:
> + for line in f:
> + result_tuple = parse_line(line)
> + if not result_tuple:
> + continue
> + print(result_tuple)
> + result.append(result_tuple)
> + continue
>
> + return result
> +
> +def run_command(command, log_file):
> + process = subprocess.Popen(command,
> + shell=True,
> + stdout=subprocess.PIPE,
> + stderr=subprocess.STDOUT)
> + with open(log_file, 'w') as f:
> + while True:
> + output = process.stdout.readline()
> + if output == '' and process.poll() is not None:
> + break
> + if output:
> + print output.strip()
This causes a syntax error in python. Don't you need to wrap this in ()
?
Kevin
_______________________________________________
linaro-validation mailing list
linaro-validation@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/linaro-validation
