Repository: incubator-systemml Updated Branches: refs/heads/gh-pages 591f7ffbe -> 6b4cd55ed
[SYSTEMML-467] Update script invocation of Beginner's Guide Closes #220. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/6b4cd55e Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/6b4cd55e Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/6b4cd55e Branch: refs/heads/gh-pages Commit: 6b4cd55ed8ba523495c6819ae7ee949497993d7d Parents: 591f7ff Author: Deron Eriksson <[email protected]> Authored: Fri Aug 26 13:24:34 2016 -0700 Committer: Deron Eriksson <[email protected]> Committed: Fri Aug 26 13:24:34 2016 -0700 ---------------------------------------------------------------------- _layouts/global.html | 5 +- beginners-guide-to-dml-and-pydml.md | 46 +++---------------- engine-dev-guide.md | 28 +++++++++++ ...bug-configuration-hello-world-main-class.png | Bin 95393 -> 0 bytes ...figuration-hello-world-program-arguments.png | Bin 85243 -> 0 bytes ...bug-configuration-hello-world-main-class.png | Bin 0 -> 95393 bytes ...figuration-hello-world-program-arguments.png | Bin 0 -> 85243 bytes 7 files changed, 38 insertions(+), 41 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/6b4cd55e/_layouts/global.html ---------------------------------------------------------------------- diff --git a/_layouts/global.html b/_layouts/global.html index e650cf8..aebd204 100644 --- a/_layouts/global.html +++ b/_layouts/global.html @@ -47,9 +47,10 @@ <ul class="dropdown-menu" role="menu"> <li><b>Running SystemML:</b></li> <li><a href="https://github.com/apache/incubator-systemml";>SystemML GitHub README</a></li> - <li><a href="standalone-guide.html">Standalone Guide</a></li> - <li><a href="spark-mlcontext-programming-guide.html">Spark MLContext Programming Guide</a></li> + <li><a href="spark-mlcontext-programming-guide.html">Spark MLContext</a></li> + <li><a href="spark-batch-mode.html">Spark Batch Mode</a> <li><a href="hadoop-batch-mode.html">Hadoop Batch Mode</a> + <li><a href="standalone-guide.html">Standalone Guide</a></li> <li><a href="jmlc.html">Java Machine Learning Connector (JMLC)</a> <li class="divider"></li> <li><b>Language Guides:</b></li> http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/6b4cd55e/beginners-guide-to-dml-and-pydml.md ---------------------------------------------------------------------- diff --git a/beginners-guide-to-dml-and-pydml.md b/beginners-guide-to-dml-and-pydml.md index fe96829..a5374e7 100644 --- a/beginners-guide-to-dml-and-pydml.md +++ b/beginners-guide-to-dml-and-pydml.md @@ -35,11 +35,9 @@ through the specification of a high-level declarative machine learning language that comes in two flavors, one with an R-like syntax (DML) and one with a Python-like syntax (PyDML). -Algorithm scripts written in DML and PyDML can be run on Hadoop, on Spark, or -in Standalone mode. No script modifications are required to change between modes. -SystemML automatically performs advanced -optimizations based on data and cluster characteristics, so much of the need to manually -tweak algorithms is largely reduced or eliminated. +Algorithm scripts written in DML and PyDML can be run on Spark, on Hadoop, or +in Standalone mode. SystemML also features an MLContext API that allows SystemML +to be accessed via Scala or Python from a Spark Shell, a Jupyter Notebook, or a Zeppelin Notebook. This Beginner's Guide serves as a starting point for writing DML and PyDML scripts. @@ -52,43 +50,14 @@ DML and PyDML scripts can be invoked in a variety of ways. Suppose that we have print('hello ' + $1) -One way to begin working with SystemML is to build the project and unpack the standalone distribution, -which features the `runStandaloneSystemML.sh` and `runStandaloneSystemML.bat` scripts. The name of the DML or PyDML script +One way to begin working with SystemML is to [download a standalone tar.gz or zip distribution of SystemML](http://systemml.apache.org/download.html) +and use the `runStandaloneSystemML.sh` and `runStandaloneSystemML.bat` scripts to run SystemML in standalone +mode. The name of the DML or PyDML script is passed as the first argument to these scripts, along with a variety of arguments. ./runStandaloneSystemML.sh hello.dml -args world ./runStandaloneSystemML.sh hello.pydml -python -args world -For DML and PyDML script invocations that take multiple arguments, a common technique is to create -a standard script that invokes `runStandaloneSystemML.sh` or `runStandaloneSystemML.bat` with the arguments specified. - -SystemML itself is written in Java and is managed using Maven. As a result, SystemML can readily be -imported into a standard development environment such as Eclipse. -The `DMLScript` class serves as the main entrypoint to SystemML. Executing -`DMLScript` with no arguments displays usage information. A script file can be specified using the `-f` argument. - -In Eclipse, a Debug Configuration can be created with `DMLScript` as the Main class and any arguments specified as -Program arguments. A PyDML script requires the addition of a `-python` switch. - -<div class="codetabs2"> - -<div data-lang="Eclipse Debug Configuration - Main" markdown="1"> - -</div> - -<div data-lang="Eclipse Debug Configuration - Arguments" markdown="1"> - -</div> - -</div> - -SystemML contains a default set of configuration information. In addition to this, SystemML looks for a default `./SystemML-config.xml` file in the working directory, where overriding configuration information can be specified. Furthermore, a config file can be specified using the `-config` argument, as in this example: - - -f hello.dml -config=src/main/standalone/SystemML-config.xml -args world - -When operating in a distributed environment, it is *highly recommended* that cluster-specific configuration information -is provided to SystemML via a configuration file for optimal performance. - # Data Types @@ -913,6 +882,5 @@ for (i in 0:numRowsToPrint-1): The [Language Reference](dml-language-reference.html) contains highly detailed information regarding DML. -In addition, many excellent examples of DML and PyDML can be found in the `system-ml/scripts` and -`system-ml/test/scripts/applications` directories. +In addition, many excellent examples of DML and PyDML can be found in the [`scripts`](https://github.com/apache/incubator-systemml/tree/master/scripts) directory. http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/6b4cd55e/engine-dev-guide.md ---------------------------------------------------------------------- diff --git a/engine-dev-guide.md b/engine-dev-guide.md index 634846a..4240408 100644 --- a/engine-dev-guide.md +++ b/engine-dev-guide.md @@ -54,3 +54,31 @@ If required, please install the following packages in R: install.packages(c("batch", "bitops", "boot", "caTools", "data.table", "doMC", "doSNOW", "ggplot2", "glmnet", "lda", "Matrix", "matrixStats", "moments", "plotrix", "psych", "reshape", "topicmodels", "wordcloud"), dependencies=TRUE) * * * + +## Development Environment + +SystemML itself is written in Java and is managed using Maven. As a result, SystemML can readily be +imported into a standard development environment such as Eclipse and IntelliJ IDEA. +The `DMLScript` class serves as the main entrypoint to SystemML. Executing +`DMLScript` with no arguments displays usage information. A script file can be specified using the `-f` argument. + +In Eclipse, a Debug Configuration can be created with `DMLScript` as the Main class and any arguments specified as +Program arguments. A PyDML script requires the addition of a `-python` switch. + +Suppose that we have a `hello.dml` script containing the following: + + print('hello ' + $1) + +This SystemML script can be debugged in Eclipse using a Debug Configuration such as the following: + +<div class="codetabs2"> + +<div data-lang="Eclipse Debug Configuration - Main" markdown="1"> + +</div> + +<div data-lang="Eclipse Debug Configuration - Arguments" markdown="1"> + +</div> + +</div> http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/6b4cd55e/img/beginners-guide-to-dml-and-pydml/dmlscript-debug-configuration-hello-world-main-class.png ---------------------------------------------------------------------- diff --git a/img/beginners-guide-to-dml-and-pydml/dmlscript-debug-configuration-hello-world-main-class.png b/img/beginners-guide-to-dml-and-pydml/dmlscript-debug-configuration-hello-world-main-class.png deleted file mode 100644 index 2ee98b9..0000000 Binary files a/img/beginners-guide-to-dml-and-pydml/dmlscript-debug-configuration-hello-world-main-class.png and /dev/null differ http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/6b4cd55e/img/beginners-guide-to-dml-and-pydml/dmlscript-debug-configuration-hello-world-program-arguments.png ---------------------------------------------------------------------- diff --git a/img/beginners-guide-to-dml-and-pydml/dmlscript-debug-configuration-hello-world-program-arguments.png b/img/beginners-guide-to-dml-and-pydml/dmlscript-debug-configuration-hello-world-program-arguments.png deleted file mode 100644 index b1e344a..0000000 Binary files a/img/beginners-guide-to-dml-and-pydml/dmlscript-debug-configuration-hello-world-program-arguments.png and /dev/null differ http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/6b4cd55e/img/engine-dev-guide/dmlscript-debug-configuration-hello-world-main-class.png ---------------------------------------------------------------------- diff --git a/img/engine-dev-guide/dmlscript-debug-configuration-hello-world-main-class.png b/img/engine-dev-guide/dmlscript-debug-configuration-hello-world-main-class.png new file mode 100644 index 0000000..2ee98b9 Binary files /dev/null and b/img/engine-dev-guide/dmlscript-debug-configuration-hello-world-main-class.png differ http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/6b4cd55e/img/engine-dev-guide/dmlscript-debug-configuration-hello-world-program-arguments.png ---------------------------------------------------------------------- diff --git a/img/engine-dev-guide/dmlscript-debug-configuration-hello-world-program-arguments.png b/img/engine-dev-guide/dmlscript-debug-configuration-hello-world-program-arguments.png new file mode 100644 index 0000000..b1e344a Binary files /dev/null and b/img/engine-dev-guide/dmlscript-debug-configuration-hello-world-program-arguments.png differ
