fzoepffel opened a new pull request, #2063: URL: https://github.com/apache/systemds/pull/2063
In this PR incremental SliceLine was further developed. The following functionalities were added: - a params list parameter ensures consistent parameters over all runs - further parameters were added that are necessary for pruning - a function was added to determine the unchanged slices on each level - several pruning strategies were added or adjusted. e.g. score pruning based on the lowest (updated) score of the previous top k was added (basic slices and normal slices are pruned), unchanged slices under the minSup are removed right away. - consistent one hot encoded matrices are ensured - encoding and decoding functions with domain size scaling were added. If the parameter encodeLat is set to true the lattice is stored in an encoded form for reduced size. - the prevStats only include the slice sizes as the other columns are not needed for pruning - the code was cleaned up, comments were added, larger code snippets were replaced by functions - first experiments have indicated a strong decrease in execution time, especially if the number of added tuples is significantly smaller than the total number of slices in an incremental run. This is the case because a larger number of added tuples usually tends to change a larger portion of slices. Only unchanged slices can lead to significant efficiency increases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@systemds.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
