Kenny-Dean opened a new pull request, #2540: URL: https://github.com/apache/systemds/pull/2540
The method is adapted from the datamap-driven coreset approach of Hadar et al., "Datamap-Driven Tabular Coreset Selection for Classifier Training" (https://www.vldb.org/pvldb/vol18/p876-razmadze.pdf). This patch includes: - The coresetDT builtin, that selects a smaller training coreset via a depth-bound tree for classification datasets. Informative regions are kept whole and redundant ones are sampled down to a target fraction. - builtin-reference.md documentation. - A test on the wine dataset. The implementation works as intended, datasets are compressible down to a fraction of the full dataset while keeping the full datasets accuracy, however, parameter tweaking of frac, psi, tau, samp_ratio and max tree depth is possible, which allows someone to shift the focus from model hyperparameter tuning to coreset optimization. The current default parameters are results from some sweeps over the local Titanic and Wine dataset as well as the Adult and a subsampled version on Covertype dataset, which all are medium in row size and manageablein coloum size. Deviations from the default parameters of the paper include the psi (1 -> 0.9), tau (5 -> 10), datamap model (GBDT -> 1DT)(variation explained in paper), whereas samp_ratio stayed the same. This produces good results. Some statistics: | Dataset | config (psi / frac) | Realised frac (coreset % of Full dataset) | dtree: full -> core (delta) | logreg: full -> core (delta) | |--------------|-------------------------------|---------------|------------------------|-------------------------| | Wine | 0.90 / 0.15 | 0.14 | 97.97 -> (−2.43) | 98.44 -> (−0.24) | | Adult | 0.80 / 0.40 | 0.40 | 83.63 -> (−1.32) | 83.18 -> (−0.50) | | Titanic | 0.80 / 0.60 | 0.60 | 76.42 -> (−0.81) | 78.05 -> (−0.81) | | (subsampled) Covertype | 0.80 / 0.60 | 0.60 | 75.76 -> (−3.00) | 71.24 -> (−1.19) | Compressability and the floor are different for each dataset, so its advised to first run the function with verbose=True to get some insights over the region statistics and the dataset- (and parameter-) specific coreset floor. Future work might include the implementation of the opt_per algorithm that is described in the same paper. It returns the minimal coreset given an accuracy drop you are willing to accept, but the ability to perform parameter sweeps for the best parameters is still possible. PS: Since this Student Project was given to me without a Jira Issue present the title is only a placeholder for now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
