Repository: systemml-website Updated Branches: refs/heads/master 5f0f7508a -> bd61b2065
Add release notes 1.2.0 and updated road map Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/bd61b206 Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/bd61b206 Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/bd61b206 Branch: refs/heads/master Commit: bd61b2065247ee9d6b953721395583dfa589b4c7 Parents: 5f0f750 Author: Berthold Reinwald <reinw...@us.ibm.com> Authored: Fri Aug 24 21:53:51 2018 -0700 Committer: Berthold Reinwald <reinw...@us.ibm.com> Committed: Fri Aug 24 21:53:51 2018 -0700 ---------------------------------------------------------------------- .../systemml-release-notes-1.2.0.md | 105 +++++++++++++++++++ _src/roadmap.html | 68 ++++++------ 2 files changed, 141 insertions(+), 32 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/bd61b206/_src/release-notes/systemml-release-notes-1.2.0.md ---------------------------------------------------------------------- diff --git a/_src/release-notes/systemml-release-notes-1.2.0.md b/_src/release-notes/systemml-release-notes-1.2.0.md new file mode 100644 index 0000000..fc8c2a7 --- /dev/null +++ b/_src/release-notes/systemml-release-notes-1.2.0.md @@ -0,0 +1,105 @@ +--- +layout: page +title: SystemML 1.2.0 Release Notes +description: Project Release Notes +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +<section class="full-stripe full-stripe--subpage-header clear-header"> + <div class="ml-container ml-container--horizontally-center"> + <div class="col col-12 content-group content-group--center-content content-group--center-align"> + <h1>{{ site.data.project.name }} 1.2.0 Release Notes</h1> + </div> + </div> +</section> + +<section class="full-stripe full-stripe--alternate"> + <div class="ml-container"> + <div class="col col-12 content-group content-group--medium-bottom-margin" markdown="1"> + +The Apache SystemML 1.2.0 release was approved on Aug. 24th, 2018. The release includes enhancements, features, and additions as listed below. + + +### New Builtin Functions/Operations/Scripts/Features +- Factorization Machines +- Support for functions with default parameters +- `exists()` for checking of existing variables +- Triangular matrix functions: `lower.tri()` and `upper.tri()` +- New nary min/max operations +- `as.matrix()` over list of scalars +- Function calls with named function arguments +- Convolution operations (forward/backward) +- Maxpooling operations (forward) +- Support for bias_add and bias_mult +- Global constants +- Added support for rowProd/colProd +- DML eval function +- New data type list for lists and structs + +### API +- JMLC API extension for passing multiple dml scripts + +### Compiler & Runtime +- Code motion framework +- Global subexpression elimination +- Optional rewrite for hoisting loop-invariant operations +- Improved IPA constant propagation and replacement +- ParFor Data Partitioning Rewrite on Hops instead of Statements +- New rewrites for chains of comparisons +- Extended rewrite framework for codegen plans +- Improved parfor optimizer rewrite for in-place-update +- Rework function block recompilation +- Support rowMeans in codegen row templates + +### Performance Improvements +- Improved multi-threading of unary aggregates +- Reuse of fair scheduler pools in local parfor workers +- Performance issues Spark ctable(X,Y) w/ large num distinct +- Performance instruction generation +- Performance sample operations +- Performance ultra-sparse block operations +- New native tsmm operator and its integration +- Multi-threaded unary operations (e.g., exp, log, sigmoid) + +### Bug Fixes +- Memory leak buffer pool due to missing variable cleanup +- Missing buffer pool serialization of compressed matrices +- Compilation failure on inferring size of reshapes w/ zero rows/columns +- Incorrect result for min/max over matrices with NaNs +- Missing support for external functions with variable number of outputs +- Reblock ultra-sparse matrix fails with index out of bounds +- Performance issue CSE on DAGs w/ many root nodes (e.g., resnet200) +- Non-fused bias_add builtin creates incorrect results over sparse inputs +- Inconsistent namespace naming depending on OS +- Codegen failing on three-way multi-aggregate +- Codegen optimizer failing for MLogreg special cases +- Failing matrix market to binary reblock with zero rows/columns + +### Deprecated/Removed/Cleanup +- Opt level 4 +- File-based removeEmpty() +- Cleanup exception handling apis/compiler/runtime + +### Experimental +- Parameter server: local backend and distributed +- Sparsity estimators + +### [JIRA release notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342959&projectId=12319522) http://git-wip-us.apache.org/repos/asf/systemml-website/blob/bd61b206/_src/roadmap.html ---------------------------------------------------------------------- diff --git a/_src/roadmap.html b/_src/roadmap.html index 9dc3044..f247051 100644 --- a/_src/roadmap.html +++ b/_src/roadmap.html @@ -36,15 +36,17 @@ limitations under the License. <section class="full-stripe full-stripe--alternate"> <div class="ml-container ml-container--narrow content-group"> <div class="col col-12"> - <h2>Planned for Future SystemML 1.2</h2> + <h2>Planned for Future Releases</h2> <ul> + <li>Completion of Prior Experimental Features</li> <li>Algorithms & builtin functions <ul> <li>NN layers-based factorization machines with regression & classification capabilities</li> <li>NN optimization test suite with well known optimization test functions</li> <li>Model selection & hyper parameter tuning</li> <li>Additional distribution functions, e.g. weibull, gamma</li> - <li>Generalization of operations, such as xor, and other operations</li> + <li>Time Series Analysis and Anomaly Detection</li> + <li>Scalable Decomposition Algorithms</li> </ul> </li> <li>Enhanced Deep Learning support @@ -73,6 +75,7 @@ limitations under the License. <li>Ultra-sparse datasets</li> </ul> </li> + <li>Model Serving</li> <li>Misc Runtime <ul> <li>NUMA-awareness (thread pools, matrix partitioning)</li> @@ -83,43 +86,50 @@ limitations under the License. </li> <li>Misc Compiler <ul> - <li>Consolidate replicated compilation chain (e.g., diff APIs)</li> <li>Holistic sum-product optimization and operator fusion</li> - <li>Extended sparsity estimators</li> - <li>Rewrites and compiler improvements for mini-batching including prefetching</li> - <li>Parfor optimizer support for shared reads</li> <li>SPOOF compiler improvement</li> + <li>Tree-based Data Structures</li> + <li>Global Dataflow Optimizations</li> </ul> </li> - <li>APIs - <ul> - <li>Python Binding for JMLC API</li> - </ul> - - </li> - - + <li>Tooling</li> + <li>Common DSL Architecture</li> </ul> </div> <div class="col col-12"> - <h2>Feature Candidates for Future Releases</h2> + <h2>Current Release</h2> <ul> - <li>Completion of Prior Experimental Features</li> - <li>Python DSL</li> - <li>New Algorithms -- e.g. Decomposition Algorithms</li> - <li>Common DSL Architecture</li> - <li>R Interfaces: R DSL and R Wrappers</li> - <li>Native Zeppelin Notebook Support</li> - <li>Sum Product Optimizations</li> - <li>Tree-based Data Structures</li> - <li>Global Dataflow Optimizations</li> - <li>Tooling</li> + <li> + <strong>SystemML 1.2.0 (<a href="release-notes/systemml-release-notes-1.2.0.html">released</a> in Aug, 2018) + <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342959&projectId=12319522">details</a></strong> + <ul> + <li>New data type list for lists and structs</li> + <li>Built in deep learning functions, e.g. convolution, maxpooling, bias</li> + <li>DML eval function</li> + <li>Multiple new builtin functions, e.g. exists, lower.tri, upper.tri</li> + <li>Code motion, global subexpression elimination, rewrites for codegen</li> + <li>Performance improvements for ultra-sparse data and instruction generation + </ul> + </li> + <li> + Experimental Features + <ul> + <li>Parameter server</li> + <li>Sparsity estimators</li> + </ul> + </li> + <li> + Removals + <ul> + <li>Opt level 4</li> + </ul> + </li> </ul> </div> <div class="col col-12"> - <h2>Current Release</h2> + <h2>Prior Releases</h2> <ul> <li> <strong>SystemML 1.1.0 (<a href="release-notes/systemml-release-notes-1.1.0.html">released</a> in March, 2018) @@ -133,12 +143,6 @@ limitations under the License. <li>Performance improvements</li> </ul> </li> - </ul> - </div> - - <div class="col col-12"> - <h2>Prior Releases</h2> - <ul> <li> <strong>SystemML 1.0.0 (<a href="release-notes/systemml-release-notes-1.0.0.html">released</a> in December, 2017) <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319522&version=12338328">details</a></strong>