Thanks for working on this Sebastian. I will review & test this work tomorrow.
Thank you, Janardhan On Sat, Sep 26, 2020 at 5:39 PM Baunsgaard, Sebastian <[email protected]> wrote: > Hi Janardhan, > > > Thanks for pointing this out! > > They were not put into master because the examples does not work the same, > > I will try to make a reasonable extract of these documents in a PR, that > fit the current system. > > If you want to take a look on that PR it would be much appreciated! > > > thanks again > > Sebastian > > > ________________________________ > From: Janardhan <[email protected]> > Sent: Friday, September 25, 2020 10:26:00 PM > To: [email protected] > Subject: Re: [DISCUSS] Apache SystemDS 2.0 Release > > Hi Arnab, > > We do not seem to have algorithms documentation. > For example an equivalent of this[1] page. > > Ignore this message, if it exists already. > > [1] http://systemds.apache.org/docs/1.2.0/algorithms-reference.html > > Thank you, > Janardhan > > On Fri, Sep 25, 2020 at 3:53 PM arnab phani <[email protected]> wrote: > > > Hi All, > > > > Thanks for fixing the remaining issues. > > I will cut the first release candidates later in the afternoon (CET zone) > > today. > > > > Regards, > > Arnab.. > > > > On Tue, Sep 22, 2020 at 2:53 PM arnab phani <[email protected]> > wrote: > > > > > Hi All, > > > > > > Thanks for fixing the bugs. > > > A few tasks/bugs (see below) are still being worked on and hopefully > will > > > be closed in a couple of days. > > > And we should be all ready to distribute the release candidates by the > > end > > > of this week. > > > > > > - python tutorials, > > > - failing gpu tests, > > > - error in loading native BLAS. > > > > > > Regards, > > > Arnab.. > > > > > > On Thu, Sep 10, 2020 at 11:24 AM arnab phani <[email protected]> > > wrote: > > > > > >> Thank you all for the notes. > > >> Please find the consolidated release notes below, and please let me > know > > >> if anything major is missing. > > >> > > >> *Release notes for SystemDS 2.0.* > > >> > > >> SystemDS 2.0 is the first major release under the new name. This > release > > >> contains a major refactoring, a few major features, a large number of > > >> improvements and fixes, and some experimental features to better > support > > >> the end-to-end data science lifecycle. In addition to that, this > release > > >> also removes several features that are not up to the mark and > outdated. > > >> > > >> The major changes (compared to SystemML 1.2) include > > >> > > >> > > >> - New mechanism for DML-bodied (script-level) builtin functions, > and > > >> a wealth of new built-in functions for data preprocessing including > > data > > >> cleaning, augmentation and feature engineering techniques, new ML > > >> algorithms, and model debugging. > > >> - Several methods for data cleaning have been implemented including > > >> multiple imputations with multivariate imputation by chained > > equations > > >> (MICE) and other techniques, SMOTE, an oversampling technique for > > class > > >> imbalance, forward and backward NA filling, cleaning using schema > and > > >> length information, support for outlier detection using standard > > deviation > > >> and inter-quartile range, and functional dependency discovery. > > >> - A complete framework for lineage tracing and reuse including > > >> support for loop deduplication, full and partial reuse, compiler > > assisted > > >> reuse, several new rewrites to facilitate reuse. > > >> - New federated runtime backend including support for federated > > >> matrices and frames, federated builtins (transform-encode, decode > > etc.). > > >> - Refactor compression package and add functionalities including > > >> quantization for lossy compression, binary cell operations, left > > matrix > > >> multiplication. > > >> - New python bindings with supports for several builtins, matrix > > >> operations, federated tensors, and lineage traces. > > >> - Cuda implementation of cumulative aggregate operators (cumsum, > > >> cumprod etc.) > > >> - New model debugging technique with slice finder. > > >> - New tensor data model (basic tensors of different value types, > data > > >> tensors with schema) [experimental] > > >> - Cloud deployment scripts for AWS and scripts to set up and start > > >> federated operations. > > >> - Performance improvements with parallel sort, gpu cum agg, append > > >> cbind etc. > > >> - Various compiler and runtime improvements including new and > > >> improved rewrites, reduced Spark context creation, new eval > > framework, list > > >> operations, updated native kernel libraries to name a few. > > >> - New data reader/writer for json frames and support for sql as a > > >> data source. > > >> - Miscellaneous improvements: improved documentation, better > > >> testing, run/release scripts, improved packaging, Docker container > > for > > >> systemds, bug fixes. > > >> - Removed MapReduce compiler and runtime backend, pydml parser, > > >> Java-UDF framework, script-level debugger. > > >> > > >> > > >> Regards, > > >> Arnab. > > >> > > >> > > >> On Tue, Sep 8, 2020 at 4:10 AM Mark Dokter <[email protected]> > > >> wrote: > > >> > > >>> On 01.09.20 11:36, arnab phani wrote: > > >>> > While I will aggregate the notes from two SystemDS releases, it > will > > be > > >>> > great if you can update me with a few lines summarizing the > additions > > >>> to > > >>> > your features (including the external contributions), especially > > after > > >>> > March 24, 2020 (last SystemDS release). > > >>> > > >>> Hi Arnab! > > >>> > > >>> My contributions: > > >>> > > >>> - new run script > > >>> - improve/simplify release scripts > > >>> - various release related things (improve documentation, fix license > > >>> headers, clean up pom.xml, etc) > > >>> - cuda implementation of cumulative aggregate operators (cumsum, > > >>> cumprod, etc) > > >>> - bug fixes here and there > > >>> - maintain native blas support in a working state (now also > supporting > > >>> windows) > > >>> - kmeans builtin dml function > > >>> - builtins for image augmentation > > >>> > > >>> Best, > > >>> Mark > > >>> > > >> > > >
