Re: [OE-core] [PATCH] resulttool/merge: Enable merge results to one file
Hi RP,
Yes, you are right, I missed the current feature to merge to flat file. Thanks
for your inputs and sharing.
After we tested the merge to flat file features, these are our findings.
1) Existing merge could not support use case where both base and target are
directory.
Traceback (most recent call last):
File "/home/poky/scripts/resulttool", line 93, in
sys.exit(main())
File "/home/poky/scripts/resulttool", line 87, in main
ret = args.func(args, logger)
File "/home/poky/scripts/lib/resulttool/merge.py", line 22, in merge
resultutils.append_resultsdata(results, args.base_results,
configmap=resultutils.store_map)
File "/home/poky/scripts/lib/resulttool/resultutils.py", line 47, in
append_resultsdata
with open(f, "r") as filedata:
IsADirectoryError: [Errno 21] Is a directory: ''
2) Existing merge will always create testseries configuration where this
configuration has important implication to report and regression.
For current QA process, we need the merge to be more flexible where it will
allow merge where both base and target are directory, furthermore we need
control over when we want to create the testseries configuration.
To fulfill both of these requirements, we tried the below:
Previously the append_resultsdata function only allow append the results data
to the map_results data (where map_results data wrapped the results data with
configuration map as the key).
Initially, we tried to implement an extra function where it will enable append
one map_results to another map_results data. But we abandoned this alternative
as this new append function will be pretty much a duplicated function to the
original append_resultsdata, and these will create two append functions which
they might be both hard to maintain and confusing. Thus, we tried to refactor
the append function to enable a single append function to be used for most of
the situation. Furthermore, since the map_results were
only needed by report and regression, we pulled the instructions used to turn
results data to map_results data to another function. Finally, we renamed the
functions and arguments to clearly seperated the functions using results data
from the one using map_results data.
The new patches are at below:
http://lists.openembedded.org/pipermail/openembedded-core/2019-March/280547.html
http://lists.openembedded.org/pipermail/openembedded-core/2019-March/280548.html
Please let us know if you have any more inputs or questions. Thank you very
much for your attention and help!
Best regards,
Yeoh Ee Peng
-Original Message-
From: Richard Purdie [mailto:richard.pur...@linuxfoundation.org]
Sent: Tuesday, March 26, 2019 8:52 PM
To: Yeoh, Ee Peng ;
openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] resulttool/merge: Enable merge results to one
file
On Tue, 2019-03-26 at 10:02 +0800, Yeoh Ee Peng wrote:
> QA team execute extra testing that create multiple test result files,
> where these test result files need to be merged into a single file
> under certain use case.
>
> Enable merge to allow merging results into a single test result file.
>
> Signed-off-by: Yeoh Ee Peng
> ---
> scripts/lib/resulttool/merge.py | 29 -
> scripts/lib/resulttool/resultutils.py | 76
> +--
> 2 files changed, 82 insertions(+), 23 deletions(-)
>
> diff --git a/scripts/lib/resulttool/merge.py
> b/scripts/lib/resulttool/merge.py index 3e4b7a3..90b3cb3 100644
> --- a/scripts/lib/resulttool/merge.py
> +++ b/scripts/lib/resulttool/merge.py
> @@ -17,6 +17,26 @@ import json
> import resulttool.resultutils as resultutils
>
> def merge(args, logger):
> +if args.merge_to_one:
> +if os.path.isdir(args.target_results):
> +target_results = resultutils.load_results(args.target_results)
> +else:
> +target_results = resultutils.append_results({},
> + args.target_results)
Looking at load_resultsdata:
def load_resultsdata(source, configmap=store_map):
results = {}
if os.path.isfile(source):
append_resultsdata(results, source, configmap)
return results
The code above can therefore be simplified to:
target_results = resultutils.load_results(args.target_results)
?
> +if os.path.isdir(args.base_results):
> +base_results = resultutils.load_results(args.base_results)
> +results = resultutils.append_results(target_results,
> base_results)
> +else:
> +results = resultutils.append_results(target_results,
> + args.base_results)
Again, I'm not sure you need to differentiate between a file and a directory
given the way the code works internally?
> +
> +target_file_dir = os.path.join(os.path.dirname(args.target_results),
> 'merged_r
Re: [OE-core] [PATCH] resulttool/merge: Enable merge results to one file
On Tue, 2019-03-26 at 10:02 +0800, Yeoh Ee Peng wrote:
> QA team execute extra testing that create multiple test result files,
> where these test result files need to be merged into a single file
> under certain use case.
>
> Enable merge to allow merging results into a single test result file.
>
> Signed-off-by: Yeoh Ee Peng
> ---
> scripts/lib/resulttool/merge.py | 29 -
> scripts/lib/resulttool/resultutils.py | 76
> +--
> 2 files changed, 82 insertions(+), 23 deletions(-)
>
> diff --git a/scripts/lib/resulttool/merge.py b/scripts/lib/resulttool/merge.py
> index 3e4b7a3..90b3cb3 100644
> --- a/scripts/lib/resulttool/merge.py
> +++ b/scripts/lib/resulttool/merge.py
> @@ -17,6 +17,26 @@ import json
> import resulttool.resultutils as resultutils
>
> def merge(args, logger):
> +if args.merge_to_one:
> +if os.path.isdir(args.target_results):
> +target_results = resultutils.load_results(args.target_results)
> +else:
> +target_results = resultutils.append_results({},
> args.target_results)
Looking at load_resultsdata:
def load_resultsdata(source, configmap=store_map):
results = {}
if os.path.isfile(source):
append_resultsdata(results, source, configmap)
return results
The code above can therefore be simplified to:
target_results = resultutils.load_results(args.target_results)
?
> +if os.path.isdir(args.base_results):
> +base_results = resultutils.load_results(args.base_results)
> +results = resultutils.append_results(target_results,
> base_results)
> +else:
> +results = resultutils.append_results(target_results,
> args.base_results)
Again, I'm not sure you need to differentiate between a file and a
directory given the way the code works internally?
> +
> +target_file_dir = os.path.join(os.path.dirname(args.target_results),
> 'merged_results/testresults.json')
> +if os.path.isdir(args.target_results):
> +target_file_dir = os.path.join(args.target_results,
> 'merged_results/testresults.json')
> +if args.merge_to_one_dir:
> +target_file_dir = os.path.join(args.merge_to_one_dir,
> 'testresults.json')
> +resultutils.make_directory_and_write_json_file(target_file_dir,
> results)
> +logger.info('Merged results to %s' % target_file_dir)
> +return 0
Isn't this similar to calling load_resultsdata with the 'flatten'
mapping, i.e. resultutils.flatten_map?
That would match the code already in the merge command:
results = resultutils.load_resultsdata(args.base_results,
configmap=resultutils.flatten_map)
if os.path.exists(args.target_results):
resultutils.append_resultsdata(results, args.target_results,
configmap=resultutils.flatten_map)
resultutils.save_resultsdata(results,
os.path.dirname(args.target_results), fn=os.path.basename(args.target_results))
i.e. if you call:
"resulttool merge merged_results/testresults.json"
it should merge the results from into the results file specified?
In other words, doesn't the tool already support this?
Cheers,
Richard
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] resulttool/merge: Enable merge results to one file
QA team execute extra testing that create multiple test result files,
where these test result files need to be merged into a single file
under certain use case.
Enable merge to allow merging results into a single test result file.
Signed-off-by: Yeoh Ee Peng
---
scripts/lib/resulttool/merge.py | 29 -
scripts/lib/resulttool/resultutils.py | 76 +--
2 files changed, 82 insertions(+), 23 deletions(-)
diff --git a/scripts/lib/resulttool/merge.py b/scripts/lib/resulttool/merge.py
index 3e4b7a3..90b3cb3 100644
--- a/scripts/lib/resulttool/merge.py
+++ b/scripts/lib/resulttool/merge.py
@@ -17,6 +17,26 @@ import json
import resulttool.resultutils as resultutils
def merge(args, logger):
+if args.merge_to_one:
+if os.path.isdir(args.target_results):
+target_results = resultutils.load_results(args.target_results)
+else:
+target_results = resultutils.append_results({},
args.target_results)
+if os.path.isdir(args.base_results):
+base_results = resultutils.load_results(args.base_results)
+results = resultutils.append_results(target_results, base_results)
+else:
+results = resultutils.append_results(target_results,
args.base_results)
+
+target_file_dir = os.path.join(os.path.dirname(args.target_results),
'merged_results/testresults.json')
+if os.path.isdir(args.target_results):
+target_file_dir = os.path.join(args.target_results,
'merged_results/testresults.json')
+if args.merge_to_one_dir:
+target_file_dir = os.path.join(args.merge_to_one_dir,
'testresults.json')
+resultutils.make_directory_and_write_json_file(target_file_dir,
results)
+logger.info('Merged results to %s' % target_file_dir)
+return 0
+
if os.path.isdir(args.target_results):
results = resultutils.load_resultsdata(args.target_results,
configmap=resultutils.store_map)
resultutils.append_resultsdata(results, args.base_results,
configmap=resultutils.store_map)
@@ -26,7 +46,7 @@ def merge(args, logger):
if os.path.exists(args.target_results):
resultutils.append_resultsdata(results, args.target_results,
configmap=resultutils.flatten_map)
resultutils.save_resultsdata(results,
os.path.dirname(args.target_results), fn=os.path.basename(args.target_results))
-
+logger.info('Merged results to %s' % os.path.dirname(args.target_results))
return 0
def register_commands(subparsers):
@@ -39,4 +59,9 @@ def register_commands(subparsers):
help='the results file/directory to import')
parser_build.add_argument('target_results',
help='the target file or directory to merge the
base_results with')
-
+parser_build.add_argument('-o', '--merge-to-one', action='store_true',
+ help='merge results into one file only, does not
add any new configurations to results '
+ 'and does not create additional directory
structure which based on configurations')
+parser_build.add_argument('-d', '--merge-to-one-dir', default='',
+ help='target directory to merge results into one
file, default directory was based on '
+ 'target_results directory')
diff --git a/scripts/lib/resulttool/resultutils.py
b/scripts/lib/resulttool/resultutils.py
index 153f2b8..e76045b 100644
--- a/scripts/lib/resulttool/resultutils.py
+++ b/scripts/lib/resulttool/resultutils.py
@@ -39,18 +39,47 @@ store_map = {
"manual": ['TEST_TYPE', 'TEST_MODULE', 'MACHINE', 'IMAGE_BASENAME']
}
+def load_json_data(f):
+if type(f) is str:
+with open(f, "r") as filedata:
+return json.load(filedata)
+else:
+return f
+
+def validate_result(result):
+if "configuration" not in result or "result" not in result:
+raise ValueError("Test results data without configuration or result
section?")
+
+def delete_extra_ptest_data(result):
+if 'ptestresult.rawlogs' in result['result']:
+del result['result']['ptestresult.rawlogs']
+if 'ptestresult.sections' in result['result']:
+for i in result['result']['ptestresult.sections']:
+if 'log' in result['result']['ptestresult.sections'][i]:
+del result['result']['ptestresult.sections'][i]['log']
+
+def get_testresults_files(source):
+testresults_files = []
+for root, dirs, files in os.walk(source):
+for name in files:
+f = os.path.join(root, name)
+if name == "testresults.json":
+testresults_files.append(f)
+return testresults_files
+
+def make_directory_and_write_json_file(dst, results):
+os.makedirs(os.path.dirname(dst), exist_ok=True)
+with open(dst, 'w') as f:
+f.write(json.dumps(results, sort_keys=True, indent=4))
+
#
# Load
