Repository: systemml-website Updated Branches: refs/heads/master 733cbd741 -> 2c1e7dfca
Add 1.0.0 release notes, and updated roadmap Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/2c1e7dfc Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/2c1e7dfc Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/2c1e7dfc Branch: refs/heads/master Commit: 2c1e7dfca536dc2ae885ac527950cfebf07b84c9 Parents: 733cbd7 Author: Berthold Reinwald <[email protected]> Authored: Thu Dec 21 11:42:24 2017 -0800 Committer: Berthold Reinwald <[email protected]> Committed: Thu Dec 21 11:42:24 2017 -0800 ---------------------------------------------------------------------- .../systemml-release-notes-1.0.0.md | 94 ++++++++++++ _src/roadmap.html | 143 +++++++++++++++---- 2 files changed, 209 insertions(+), 28 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/2c1e7dfc/_src/release-notes/systemml-release-notes-1.0.0.md ---------------------------------------------------------------------- diff --git a/_src/release-notes/systemml-release-notes-1.0.0.md b/_src/release-notes/systemml-release-notes-1.0.0.md new file mode 100644 index 0000000..6e0ad0d --- /dev/null +++ b/_src/release-notes/systemml-release-notes-1.0.0.md @@ -0,0 +1,94 @@ +--- +layout: page +title: SystemML 1.0.0 Release Notes +description: Project Release Notes +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +<section class="full-stripe full-stripe--subpage-header clear-header"> + <div class="ml-container ml-container--horizontally-center"> + <div class="col col-12 content-group content-group--center-content content-group--center-align"> + <h1>{{ site.data.project.name }} 1.0.0 Release Notes</h1> + </div> + </div> +</section> + +<section class="full-stripe full-stripe--alternate"> + <div class="ml-container"> + <div class="col col-12 content-group content-group--medium-bottom-margin" markdown="1"> + +The Apache SystemML 1.0.0 release was approved on December 12, 2017. The release includes enhancements, features, and additions as listed below. + +### API Enhancements +- JMLC/MLContext parameter passing w/ and w/o meta data. + +### Deep Learning +- Improved CPU convolution function performance, including sparsity. +- `Caffe2DML`. +- Enhanced/additional NN layers, e.g. conv2d transpose and depthwise convolution +- NN architecture summary table. +- [Experimental] `Keras2DML`. + +### New Scripts / Algorithms +- Autoencoder. +- Enhanced PCA. + +### Features & new Functions +- `sinh`, `cosh`, and `tanh`. +- `transformcolmap`. +- n-ary `rbind`/`cbind`. +- `order` with multiple order-by columns. + +### Compiler +- Improved rewrites, e.g. merging of statement block sequences. +- IPA effectiveness. +- Automatic vectorization of indexing pairs. +- Instruction generation for memory efficiency. +- Instruction code organization. +- [Experimental] Code generation, code gen optimizer, and multi-threaded codegen operators. + +### Robustness & Performance +- ParFor/HOP Memory budgets for Spark cluster configurations. +- JMLC prepared scripts (thread affinity for outputs and configs, script cloning, configuration management). +- Sparse-dense binary cell wise operations. + +### [Experimental] GPU +- GPU conv2d and memory estimates. +- Additional kernels,e.g. right indexing. +- Single precision backend. + +### Additional Packages +- SystemML Lite artifact, a minimumm-size uber JAR for embeddability (w/o Hadoop or Spark dependencies). + +### Environment +- Compression on by default. +- Exploitation of native BLAS libraries. +- JDK 8 (dropped JDK 7). +- jCUDA for Windows & Linux (x86_64, ppc64le) included. +- Fine-grained runtime statistics. +- Refactored configurations parameters to have sysml prefix +- Performance test suite. + +### Other +- Refreshed/new examples and notebooks (DML examples), tutorial. + +### [JIRA release notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319522&version=12338328) +- Features, Improvements, Bug fixes, etc. http://git-wip-us.apache.org/repos/asf/systemml-website/blob/2c1e7dfc/_src/roadmap.html ---------------------------------------------------------------------- diff --git a/_src/roadmap.html b/_src/roadmap.html index b0a2681..45ebca9 100644 --- a/_src/roadmap.html +++ b/_src/roadmap.html @@ -36,19 +36,75 @@ limitations under the License. <section class="full-stripe full-stripe--alternate"> <div class="ml-container ml-container--narrow content-group"> <div class="col col-12"> - <h2>Planned for Future SystemML 1.0</h2> + <h2>Planned for Future SystemML 1.1</h2> <ul> - <li>Compression (additional operations)</li> - <li>Experimental Features: + <li>Algorithms & builtin functions <ul> - <li>Deep Learning</li> - <li>Single GPU support</li> - <li>Native BLAS support</li> - <li>Code Generation</li> + <li>NN layers-based factorization machines with regression & classification capabilities</li> + <li>NN optimization test suite with well known optimization test functions</li> + <li>Model selection & hyper parameter tuning</li> + <li>Additional distribution functions, e.g. weibull, gamma</li> + <li>Generalization of operations, such as xor, and other operations</li> + </ul> + </li> + <li>Enhanced Deep Learning support + <ul> + <li>Coherent sparse operations on CPU/GPU</li> + <li>Coherent single-precision support on CPU/GPU</li> + <li>Distributed DL operations</li> + </ul> + </li> + <li>GPU Support + <ul> + <li>Full compiler integration (cost-based, automatic placement)</li> + <li>Multi GPUs</li> + <li>Distributed GPUs</li> + </ul> + </li> + <li>Code generation + <ul> + <li>Deep learning operationss</li> + <li>Heterogeneous HW, incl GPUs</li> </ul> </li> - <li>Rigorous Performance and Scalability Testing (Bug Fixes)</li> - <li>Remove Deprecated APIs/Functions</li> + <li>Compressed Linear Algebra + <ul> + <li>Matrix-matrix multiplications</li> + <li>Deep learning operations</li> + <li>Ultra-sparse datasets</li> + </ul> + </li> + <li>Misc Runtime + <ul> + <li>Large dense matrix blocks > 16GB</li> + <li>NUMA-awareness (thread pools, matrix partitioning)</li> + <li>Unified memory management (ops, bufferpool, RDDs/broadcasts)</li> + <li>Support additional external formats such as feather format for matrices and frames</li> + <li>Parfor support for broadcasts</li> + <li>Extended support for multi-threaded operations</li> + <li>Boolean matrices</li> + </ul> + </li> + <li>Misc Compiler + <ul> + <li>Support single-output UDFs in expressions</li> + <li>Consolidate replicated compilation chain (e.g., diff APIs)</li> + <li>Holistic sum-product optimization and operator fusion</li> + <li>Extended sparsity estimators</li> + <li>Rewrites and compiler improvements for mini-batching including prefetching</li> + <li>Parfor optimizer support for shared reads</li> + <li>SPOOF compiler improvement</li> + </ul> + </li> + <li>APIs + <ul> + <li>Python Binding for JMLC API</li> + <li>Consistency Python/Java APIs</li> + </ul> + + </li> + + </ul> </div> @@ -72,6 +128,40 @@ limitations under the License. <h2>Current Release</h2> <ul> <li> + <strong>SystemML 1.0.0 (<a href="release-notes/systemml-release-notes-1.0.0.html">released</a> in December, 2017) + <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319522&version=12338328";>details</a></strong> + <ul> + <li>Enhanced Deep Learning support with enhanced NN layers and functions, Caffe2DML, and operator implementation</li> + <li>Native BLAS support</li> + <li>Additional algorithms: autoencoder, enhanced PCA</li> + <li>Enhanced rewrites, IPA, vectorization, and instruction generation</li> + <li>Enhanced JMLC API, e.g. prepared scripts with thread affinity for outputs and configs, script cloning, configuration management</li> + <li>SystemML Lite artifact</li> + <li>Compression on by default</li> + </ul> + </li> + <li> + Experimental Features + <ul> + <li>Keras2DML.</li> + <li>Enhanced code generation, code gen optimizer, and multi-threaded codegen operators/li> + <li>Enhanced GPU support</li> + </ul> + </li> + <li> + Removals + <ul> + <li>Dropped JDK 7 support</li> + </ul> + </li> + </ul> + </div> + + <div class="col col-12"> + <h2>Prior Releases</h2> + + <ul> + <li> <strong>SystemML 0.15.0 (<a href="release-notes/systemml-release-notes-0.15.0.html">released</a> in September, 2017) <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319522&version=12341587";>details</a></strong> <ul> @@ -94,10 +184,6 @@ limitations under the License. </ul> </li> </ul> - </div> - - <div class="col col-12"> - <h2>Prior Releases</h2> <ul> <li> @@ -111,8 +197,9 @@ limitations under the License. <li> Experimental Features <ul> - <li>New Code Generation capabilities for automatic operator fusion (basic code generator, compiler integration, runtime integration, in-memory source code compilation, extended explain tool, support for right indexing and replace in cellwise and row aggregate templates, support for row, column, or no aggregation in rowwise template). - Note code generation provides significant performance gains with fewer read/write intermediates, reduced scans of inputs and intermediates, and enhanced sparsity exploitation. To enable this feature, set codegen.enabled property to true in SystemML-config.xml file.</li> + <li>New Code Generation capabilities for automatic operator fusion (basic code generator, compiler integration, runtime integration, in-memory source code compilation, extended explain tool, support for right indexing and replace in cellwise and + row aggregate templates, support for row, column, or no aggregation in rowwise template). Note code generation provides significant performance gains with fewer read/write intermediates, reduced scans of inputs and intermediates, and enhanced + sparsity exploitation. To enable this feature, set codegen.enabled property to true in SystemML-config.xml file.</li> <li>New instructions and operators for GPU support (relu_maxpooling, conv2d_bias_add, bias_multiply)</li> </ul> </li> @@ -131,30 +218,30 @@ limitations under the License. <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319522&version=12339247";>details</a></strong> <ul> <li>Updated build for Spark 2.1.0</li> - <li>New simplification rewrites for stratstats</li> - <li>New fused operator tack+* in CP and Spark</li> + <li>New simplification rewrites for stratstats</li> + <li>New fused operator tack+* in CP and Spark</li> <li>New dmlFromResource capability in Python (equivalent to Scala)</li> - <li>Add input float support to MLContext</li> + <li>Add input float support to MLContext</li> </ul> </li> <li> Documentation Enhancements <ul> <li>Deploy versioned documentation to main project website</li> - <li>Add python mlcontext example to engine dev guide</li> - <li>Add MLContext info functionality to docs</li> - <li>Update DML Language Reference for write description parameter</li> + <li>Add python mlcontext example to engine dev guide</li> + <li>Add MLContext info functionality to docs</li> + <li>Update DML Language Reference for write description parameter</li> </ul> </li> <li> Deprecations, Removals <ul> <li>Deprecate old MLContext API</li> - <li>Deprecate parfor perftesttool</li> - <li>Deprecate SQLContext methods</li> - <li>Replace deprecated Accumulator with AccumulatorV2</li> - <li>Replace append with cbind for matrices</li> - <li>Migrate Vector and LabeledPoint classes from mllib to ml</li> + <li>Deprecate parfor perftesttool</li> + <li>Deprecate SQLContext methods</li> + <li>Replace deprecated Accumulator with AccumulatorV2</li> + <li>Replace append with cbind for matrices</li> + <li>Migrate Vector and LabeledPoint classes from mllib to ml</li> </ul> </li> <li> @@ -173,8 +260,8 @@ limitations under the License. <li>Support pip install of new python package</li> <li>Allow NumPy arrays, Pandas DataFrame and SciPy matrices as input to MLContext</li> <li>Improve SystemML Python DSL for NumPy</li> - <li>Updated build for Spark 1.6.0</li> - <li>DML utility script to shuffle input dataset</li> + <li>Updated build for Spark 1.6.0</li> + <li>DML utility script to shuffle input dataset</li> </ul> </li> <li>
