This is an automated email from the ASF dual-hosted git repository.
mboehm7 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/systemml.git
The following commit(s) were added to refs/heads/master by this push:
new 0dae427 [SYSTEMDS-263] Initial design ONNX graph importer
0dae427 is described below
commit 0dae42705f91b00abc03be09d810b3a9286338c5
Author: Lukas Timpl <[email protected]>
AuthorDate: Sun Apr 12 20:43:47 2020 +0200
[SYSTEMDS-263] Initial design ONNX graph importer
Since ONNX does support conditional operators (loop, if), I've tailored
the design towards a command-line tool that generates a DML script as
discussed.
AMLS project SS2020.
Closes #885.
---
docs/Tasks.txt | 1 +
docs/onnx-systemds-design.md | 46 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/docs/Tasks.txt b/docs/Tasks.txt
index cfcab1a..316971d 100644
--- a/docs/Tasks.txt
+++ b/docs/Tasks.txt
@@ -205,6 +205,7 @@ SYSTEMDS-250 Extended Slice Finding
SYSTEMDS-260 Misc Tools
* 261 Stable marriage algorithm OK
* 262 Data augmentation tool for data cleaning OK
+ * 263 ONNX graph importer/exporter
SYSTEMDS-270 Compressed Matrix Blocks
* 271 Reintroduce compressed matrix blocks from SystemML OK
diff --git a/docs/onnx-systemds-design.md b/docs/onnx-systemds-design.md
new file mode 100644
index 0000000..9650f9c
--- /dev/null
+++ b/docs/onnx-systemds-design.md
@@ -0,0 +1,46 @@
+# onnx-systemds
+
+A tool for importing/exporting
[ONNX](https://github.com/onnx/onnx/blob/master/docs/IR.md) graphs into/from
SystemDS DML scripts.
+
+
+## Goals
+
+* Support for importing [operators of the ONNX base
definition](https://github.com/onnx/onnx/blob/master/docs/Operators.md)
+
+* Support for importing [operators defined by
ONNX-ML](https://github.com/onnx/onnx/blob/master/docs/Operators-ml.md)
+
+* Support for exporting DML script to ONNX graphs
+
+## Limitations
+
+* Not able to support all data types / operators as they are not currently
supported by SystemDS
+
+
+
+## Suggested Implementation
+
+Since the ONNX specification includes the conditional operators
[loop](https://github.com/onnx/onnx/blob/master/docs/Operators.md#Loop) and
[if](https://github.com/onnx/onnx/blob/master/docs/Operators.md#If), a direct
conversion from ONNX to the internal HOP might not be ideal.
+
+Hence my suggested implementation is a dedicated tool invoked from command
line which generates DML scripts. This also enables optimizations performed by
the compiler at both graph and program level.
+
+### Example Call
+
+```bash
+onnx-systemds model.onx --out model_script.dml
+```
+
+
+### Tooling
+
+* Due to the availability of a [Python
API](https://github.com/onnx/onnx/blob/master/docs/PythonAPIOverview.md) for
ONNX, I would suggest implementing the tool in Python
+* Another advantage of Python is good support for template engines e.g.
[Jinja](https://jinja.palletsprojects.com/en/2.11.x/)
+* An implementation could use templates for various operators which are then
combined into a script
+
+### Implementation Details
+
+ONNX is a [serialized
graph](https://github.com/onnx/onnx/blob/master/docs/IR.md#graphs) structured
as a sorted list of nodes that form a DAG (directed acyclic graph).
+
+1. Loading in the serialized structure
+2.
[Checking](https://github.com/onnx/onnx/blob/master/docs/PythonAPIOverview.md#checking-an-onnx-model)
model and
[converting](https://github.com/onnx/onnx/blob/master/docs/PythonAPIOverview.md#converting-version-of-an-onnx-model-within-default-domain-aionnx)
models to a common version
+3. Building a simple internal graph structure (for arbitrary operators)
+4. Generating the DML script while traversing this graph (provided information
in doc_strings and other description variables are added as comments to improve
human-readability of the generated script)
