Modified: incubator/systemml/site/get-started.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/get-started.html?rev=1795636&r1=1795635&r2=1795636&view=diff ============================================================================== --- incubator/systemml/site/get-started.html (original) +++ incubator/systemml/site/get-started.html Fri May 19 21:48:49 2017 @@ -155,243 +155,60 @@ </div> </section> --> -<!-- Overview--> -<section class="full-stripe clear-header"> +<!-- Install SystemML --> +<section class="full-stripe full-stripe--alternate"> <div class="ml-container ml-container--vertically-centered ml-container--reverse-order"> <div class="col col-6 content-group content-group--more-padding"> <img src="/assets/img/robotTutorialSm.png" alt="What is Apache SystemML?"> </div> - <div class="col col-6 content-group content-group--more-padding"> - <h2>SystemML Beginner Tutorial</h2> + <div class="col col-6 content-group content-group--more-padding button-group"> + <h2>Install SystemML</h2> <h4><strong>Level:</strong> Beginner | <strong>Time:</strong> 20 minutes</h4><br> - <p><bdi>How to set up and run Apache SystemML locally.</bdi></p> + <p>New to Apache SystemML? Try our guick install guide that will walk you through setting up your environment and getting you up and going with SystemML.</p> <a class="button button-secondary" href="https://apache.github.io/incubator-systemml"; target="_blank">Docs</a> + <a class="button button-primary" href="install-systemml.html">Install SystemML</a> </div> </div> -</section> - -<!-- Tutorial Instructions --> -<section class="full-stripe full-stripe--alternate"> - - <!-- Section 1 --> - <div class="ml-container content-group content-group--tutorial border"> - <!-- Section Header --> - <div class="col col-12 content-group--medium-bottom-margin"> - <h2>Assumptions</h2> - <p>If you havenât run Apache SystemML before, make sure to set up your environment first.</p> - </div> - - <!-- Step 1 Instructions --> - <div class="col col-12"> - <h3><span class="circle">1</span>Install Homebrew</h3> + <!-- Sample Notebooks --> + <h2 class="text-center">Sample Notebooks</h2> + <div class="flex-container"> + <div class="nb-card"> + <h3>SystemML LinearRegCG</h3> + <p>SystemML Linear Regression in Zeppelin Notebook.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/SystemML_LinearRegCG.json"; target="_blank"><span class="icon zeppelin-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>Deep Learning Image Classification</h3> + <p>This notebook shows SystemML Deep Learning functionality to map images of single digit numbers to their corresponding numeric representations.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>Linear Regression Algorithms Demo</h3> + <p>This notebook shows: Install SystemML Python package and jar file, pip, SystemML 'Hello World'.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>SystemML PySpark Recommendation Demo</h3> + <p>This demonstrates using SystemML for product recommendation using Poisson NonNegative Matrix Factorization (PNMF) with PNMF algorithm written using R like syntax.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/SystemML-PySpark-Recommendation-Demo.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>SystemML Scala Tutorial</h3> + <p>This tutorial includes simple example to run DML script and display output.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/tutorial1.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>Autoencoder</h3> + <p>This notebook demonstrates the invocation of the SystemML autoencoder script, and alternative ways of passing in/out data.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Autoencoder.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>SystemML Zeppelin Tutorial</h3> + <p>SystemML Zeppelin tutorial.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/tutorial1_zeppelin.json"; target="_blank"><span class="icon zeppelin-logo"></span><span>View on Github</span></a> </div> - <!-- Step 1 Code --> - <div class="col col-12"> - - - <figure class="highlight"><pre><code class="language-bash" data-lang="bash">/usr/bin/ruby -e <span class="s2">"</span><span class="k">$(</span>curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install<span class="k">)</span><span class="s2">"</span> -<span class="c"># Linux</span> -ruby -e <span class="s2">"</span><span class="k">$(</span>curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install<span class="k">)</span><span class="s2">"</span></code></pre></figure> - - </div> - - <!-- Step 2 Instructions --> - <div class="col col-12"> - <h3><span class="circle">2</span>Install Java</h3> - </div> - - <!-- Step 2 Code --> - <div class="col col-12"> - <figure class="highlight"><pre><code class="language-bash" data-lang="bash">brew tap caskroom/cask -brew install Caskroom/cask/java - -brew install python -pip install jupyter matplotlib numpy</code></pre></figure> - </div> </div> - - <!-- Section 2 --> - <div class="ml-container content-group content-group--tutorial border"> - <!-- Section Header --> - <div class="col col-12 content-group--medium-bottom-margin"> - <h2>Downloads</h2> - <p>Download Apache Spark and Apache SystemML.</p> - </div> - - <!-- Step 3 Instructions --> - <div class="col col-12"> - <h3><span class="circle">3</span>Download and Install Apache Spark</h3> - </div> - - <!-- Step 3 Code --> - <div class="col col-12"> - <figure class="highlight"><pre><code class="language-bash" data-lang="bash">brew tap homebrew/versions -brew install apache-spark</code></pre></figure> - - <p> Alternatively, you can <a href="http://spark.apache.org/downloads.html";>download Apache Spark</a> directly. </p> - </div> - - <!-- Step 4 Instructions --> - <div class="col col-12"> - <h3><span class="circle">4</span>Download and Install Apache SystemML</h3> - </div> - - <!-- Step 4 Code --> - <div class="col col-12"> - - <p> - If you are a python user, we recommend that you download and install Apache SystemML via pip: - </p> - <figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Python 2</span> -pip install systemml -<span class="c"># Bleeding edge: pip install git+git://github.com/apache/incubator-systemml.git#subdirectory=src/main/python</span></code></pre></figure> - - <figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Python 3:</span> -pip3 install systemml -<span class="c"># Bleeding edge: pip3 install git+git://github.com/apache/incubator-systemml.git#subdirectory=src/main/python</span></code></pre></figure> - - - - <p> - Alternatively, if you intend to use SystemML via spark-shell (or spark-submit), you only need systemml-0.14.0-incubating.jar, which is packaged into our official binary release (<a href="http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip"; target="_blank">systemml-0.14.0-incubating-bin.zip</a>). - Note: If you have installed SystemML via pip, you can get the location of this jar by executing following command: - </p> - <figure class="highlight"><pre><code class="language-bash" data-lang="bash">python -c <span class="s1">'import imp; import os; print os.path.join(imp.find_module("systemml")[1], "systemml-java")'</span></code></pre></figure> - - <p> - Note - For Spark 1.6 users only, include a version specifier to download and install compatible Apache SystemML via pip: - </p> - <figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># For Spark 1.6 users with Python 2:</span> -pip install <span class="s2">"systemml<0.13.0"</span></code></pre></figure> - - </div> - - <!-- Section 3 --> - <div class="ml-container content-group content-group--tutorial"> - <!-- Section Header --> - <div class="col col-12 content-group--medium-bottom-margin"> - <h2>Ways to Use</h2> - <p>You can use SystemML in one of the following ways:</p> - <ol> - <li>On Cluster (using our programmatic APIs): - <ul> - <li>Using pyspark: Please see our <a href="http://apache.github.io/incubator-systemml/beginners-guide-python";>beginner's guide for python users</a>.</li> - <li>Using Jupyter: Described below in step 5.</li> - <li>Using spark-shell: Described below in step 6.</li> - </ul> - </li> - - <li>On Cluster (command-line batch mode): - <ul> - <li>Using spark-submit: Please see our <a href="http://apache.github.io/incubator-systemml/spark-batch-mode";>spark batch mode tutorial</a>.</li> - <li>Using hadoop: Please see our <a href="http://apache.github.io/incubator-systemml/hadoop-batch-mode";>hadoop batch model tutorial</a>.</li> - </ul> - </li> - - <li>On laptop (command-line batch mode) without installing Spark or Hadoop: Please see our <a href="http://apache.github.io/incubator-systemml/standalone-guide";>standalone mode tutorial</a>.</li> - - <li>In-memory mode (as part of another Java application for scoring): Please see our <a href="http://apache.github.io/incubator-systemml/jmlc";>JMLC tutorial</a>.</li> - </ol> - - <p> - Note that you can also run pyspark, spark-shell, spark-submit on you laptop using "--master local[*]" parameter. - </p> - </div> - - <!-- Step 5 Instructions --> - <div class="col col-12"> - <h3><span class="circle">5</span>In Jupyter Notebook</h3> - </div> - - <!-- Step 5 Code --> - <div class="col col-12"> - <h4>Get Started</h4> - <p>Start up your Jupyter notebook by moving to the folder where you saved the notebook. Then copy and paste the line below:</p> - <figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># Python 2:</span> -<span class="n">PYSPARK_DRIVER_PYTHON</span><span class="o">=</span><span class="n">jupyter</span> <span class="n">PYSPARK_DRIVER_PYTHON_OPTS</span><span class="o">=</span><span class="s">"notebook"</span> <span class="n">pyspark</span> <span class="o">--</span><span class="n">master</span> <span class="n">local</span><span class="p">[</span><span class="o">*</span><span class="p">]</span> <span class="o">--</span><span class="n">driver</span><span class="o">-</span><span class="n">class</span><span class="o">-</span><span class="n">path</span> <span class="n">SystemML</span><span class="o">.</span><span class="n">jar</span> <span class="o">--</span><span class="n">jars</span> <span class="n">SystemML</span><span class="o">.</span><span class="n">jar</span><span class="o">--</span><span class="n">conf</span> <span class="s">"spark.driver.memory=12g"</span> <span class="o">--</span><span class="n">conf</span> <span class="n">spark</span><span class="o">.</span><span class="n">driver< /span><span class="o">.</span><span class="n">maxResultSize</span><span class="o">=</span><span class="mi">0</span> <span class="o">--</span><span class="n">conf</span> <span class="n">spark</span><span class="o">.</span><span class="n">akka</span><span class="o">.</span><span class="n">frameSize</span><span class="o">=</span><span class="mi">128</span> <span class="o">--</span><span class="n">conf</span> <span class="n">spark</span><span class="o">.</span><span class="n">default</span><span class="o">.</span><span class="n">parallelism</span><span class="o">=</span><span class="mi">100</span></code></pre></figure> - <figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># Python 3:</span> -<span class="n">PYSPARK_PYTHON</span><span class="o">=</span><span class="n">python3</span> <span class="n">PYSPARK_DRIVER_PYTHON</span><span class="o">=</span><span class="n">jupyter</span> <span class="n">PYSPARK_DRIVER_PYTHON_OPTS</span><span class="o">=</span><span class="s">"notebook"</span> <span class="n">pyspark</span> <span class="o">--</span><span class="n">master</span> <span class="n">local</span><span class="p">[</span><span class="o">*</span><span class="p">]</span> <span class="o">--</span><span class="n">driver</span><span class="o">-</span><span class="n">class</span><span class="o">-</span><span class="n">path</span> <span class="n">SystemML</span><span class="o">.</span><span class="n">jar</span> <span class="o">--</span><span class="n">jars</span> <span class="n">SystemML</span><span class="o">.</span><span class="n">jar</span> <span class="o">--</span><span class="n">conf</span> <span class="s">"spark.driver.memory=12g"</span> <span class="o">--</span><span clas s="n">conf</span> <span class="n">spark</span><span class="o">.</span><span class="n">driver</span><span class="o">.</span><span class="n">maxResultSize</span><span class="o">=</span><span class="mi">0</span> <span class="o">--</span><span class="n">conf</span> <span class="n">spark</span><span class="o">.</span><span class="n">akka</span><span class="o">.</span><span class="n">frameSize</span><span class="o">=</span><span class="mi">128</span> <span class="o">--</span><span class="n">conf</span> <span class="n">spark</span><span class="o">.</span><span class="n">default</span><span class="o">.</span><span class="n">parallelism</span><span class="o">=</span><span class="mi">100</span></code></pre></figure> - - </div> - - <!-- Step 6 Instructions --> - <div class="col col-12"> - <h3><span class="circle">6</span>To Run SystemML in the Spark Shell</h3> - </div> - - <!-- Step 6 Code --> - <div class="col col-12"> - <h4>Start Spark Shell with SystemML</h4> - <p> To use SystemML with Spark Shell, the SystemML jar can be referenced using Spark Shellâs --jars option. Start the Spark Shell with SystemML with the following line of code in your terminal:</p> - <figure class="highlight"><pre><code class="language-bash" data-lang="bash">spark-shell --executor-memory 4G --driver-memory 4G --jars SystemML.jar</code></pre></figure> - <!-- <pre><code>spark-shell --executor-memory 4G --driver-memory 4G --jars SystemML.jar</code></pre> --> - <h4>Create the MLContext</h4> - <p>To begin, start an MLContext by typing the code below. Once successful, you should see a âWelcome to Apache SystemML!â message.</p> - <figure class="highlight"><pre><code class="language-bash" data-lang="bash">import org.apache.sysml.api.mlcontext._ -import org.apache.sysml.api.mlcontext.ScriptFactory._ -val ml <span class="o">=</span> new MLContext<span class="o">(</span>sc<span class="o">)</span></code></pre></figure> - - <h4>Hello World</h4> - <p>The ScriptFactory class allows DML and PYDML scripts to be created from Strings, Files, URLs, and InputStreams. Here, weâll use the dmlmethod to create a DML âhello worldâ script based on a String. We execute the script using MLContextâs execute method, which displays âhello worldâ to the console. The execute method returns an MLResults object, which contains no results since the script has no outputs.</p> - <figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">val</span> <span class="n">helloScript</span> <span class="o">=</span> <span class="n">dml</span><span class="p">(</span><span class="s">"print('hello world')"</span><span class="p">)</span> -<span class="n">ml</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">helloScript</span><span class="p">)</span></code></pre></figure> - <h4>DataFrame Example</h4> - <p>As an example of how to use SystemML, weâll first use Spark to create a DataFrame called df of random doubles from 0 to 1 consisting of 10,000 rows and 1,000 columns.</p> - <figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="kn">import</span> <span class="nn">org.apache.spark.sql._</span> -<span class="kn">import</span> <span class="nn">org.apache.spark.sql.types.</span><span class="p">{</span><span class="n">StructType</span><span class="p">,</span><span class="n">StructField</span><span class="p">,</span><span class="n">DoubleType</span><span class="p">}</span> -<span class="kn">import</span> <span class="nn">scala.util.Random</span> -<span class="n">val</span> <span class="n">numRows</span> <span class="o">=</span> <span class="mi">10000</span> -<span class="n">val</span> <span class="n">numCols</span> <span class="o">=</span> <span class="mi">1000</span> -<span class="n">val</span> <span class="n">data</span> <span class="o">=</span> <span class="n">sc</span><span class="o">.</span><span class="n">parallelize</span><span class="p">(</span><span class="mi">0</span> <span class="n">to</span> <span class="n">numRows</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="nb">map</span> <span class="p">{</span> <span class="n">_</span> <span class="o">=></span> <span class="n">Row</span><span class="o">.</span><span class="n">fromSeq</span><span class="p">(</span><span class="n">Seq</span><span class="o">.</span><span class="n">fill</span><span class="p">(</span><span class="n">numCols</span><span class="p">)(</span><span class="n">Random</span><span class="o">.</span><span class="n">nextDouble</span><span class="p">))</span> <span class="p">}</span> -<span class="n">val</span> <span class="n">schema</span> <span class="o">=</span> <span class="n">StructType</span><span class="p">((</span><span class="mi">0</span> <span class="n">to</span> <span class="n">numCols</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="nb">map</span> <span class="p">{</span> <span class="n">i</span> <span class="o">=></span> <span class="n">StructField</span><span class="p">(</span><span class="s">"C"</span> <span class="o">+</span> <span class="n">i</span><span class="p">,</span> <span class="n">DoubleType</span><span class="p">,</span> <span class="n">true</span><span class="p">)</span> <span class="p">}</span> <span class="p">)</span> -<span class="n">val</span> <span class="n">df</span> <span class="o">=</span> <span class="n">sqlContext</span><span class="o">.</span><span class="n">createDataFrame</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span></code></pre></figure> - - <p>Weâll create a DML script using the ScriptFactory dml method to find the minimum, maximum, and mean values in a matrix. This script has one input variable, matrix Xin, and three output variables, minOut, maxOut, and meanOut. -For performance, weâll specify metadata indicating that the matrix has 10,000 rows and 1,000 columns. -We execute the script and obtain the results as a Tuple by calling getTuple on the results, specifying the types and names of the output variables.</p> - <figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">val</span> <span class="n">minMaxMean</span> <span class="o">=</span> -<span class="s">""" -minOut = min(Xin) -maxOut = max(Xin) -meanOut = mean(Xin) -"""</span> -<span class="n">val</span> <span class="n">mm</span> <span class="o">=</span> <span class="n">new</span> <span class="n">MatrixMetadata</span><span class="p">(</span><span class="n">numRows</span><span class="p">,</span> <span class="n">numCols</span><span class="p">)</span> -<span class="n">val</span> <span class="n">minMaxMeanScript</span> <span class="o">=</span> <span class="n">dml</span><span class="p">(</span><span class="n">minMaxMean</span><span class="p">)</span><span class="o">.</span><span class="ow">in</span><span class="p">(</span><span class="s">"Xin"</span><span class="p">,</span> <span class="n">df</span><span class="p">,</span> <span class="n">mm</span><span class="p">)</span><span class="o">.</span><span class="n">out</span><span class="p">(</span><span class="s">"minOut"</span><span class="p">,</span> <span class="s">"maxOut"</span><span class="p">,</span> <span class="s">"meanOut"</span><span class="p">)</span> -<span class="n">val</span> <span class="p">(</span><span class="nb">min</span><span class="p">,</span> <span class="nb">max</span><span class="p">,</span> <span class="n">mean</span><span class="p">)</span> <span class="o">=</span> <span class="n">ml</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">minMaxMeanScript</span><span class="p">)</span><span class="o">.</span><span class="n">getTuple</span><span class="p">[</span><span class="n">Double</span><span class="p">,</span> <span class="n">Double</span><span class="p">,</span> <span class="n">Double</span><span class="p">](</span><span class="s">"minOut"</span><span class="p">,</span> <span class="s">"maxOut"</span><span class="p">,</span> <span class="s">"meanOut"</span><span class="p">)</span></code></pre></figure> - <p>Many different types of input and output variables are automatically allowed. These types include Boolean, Long, Double, String, Array[Array[Double]], RDD<String> and JavaRDD<String> in CSV (dense) and IJV (sparse) formats, DataFrame, BinaryBlockMatrix,Matrix, and Frame. RDDs and JavaRDDs are assumed to be CSV format unless MatrixMetadata is supplied indicating IJV format.</p> - <h4>RDD Example:</h4> - <p>Letâs take a look at an example of input matrices as RDDs in CSV format. Weâll create two 2x2 matrices and input these into a DML script. This script will sum each matrix and create a message based on which sum is greater. We will output the sums and the message.</p> - <figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">val</span> <span class="n">rdd1</span> <span class="o">=</span> <span class="n">sc</span><span class="o">.</span><span class="n">parallelize</span><span class="p">(</span><span class="n">Array</span><span class="p">(</span><span class="s">"1.0,2.0"</span><span class="p">,</span> <span class="s">"3.0,4.0"</span><span class="p">))</span> -<span class="n">val</span> <span class="n">rdd2</span> <span class="o">=</span> <span class="n">sc</span><span class="o">.</span><span class="n">parallelize</span><span class="p">(</span><span class="n">Array</span><span class="p">(</span><span class="s">"5.0,6.0"</span><span class="p">,</span> <span class="s">"7.0,8.0"</span><span class="p">))</span> -<span class="n">val</span> <span class="n">sums</span> <span class="o">=</span> <span class="s">""" -s1 = sum(m1); -s2 = sum(m2); -if (s1 > s2) { -message = "s1 is greater" -} else if (s2 > s1) { -message = "s2 is greater" -} else { -message = "s1 and s2 are equal" -} -"""</span> -<span class="n">scala</span><span class="o">.</span><span class="n">tools</span><span class="o">.</span><span class="n">nsc</span><span class="o">.</span><span class="n">io</span><span class="o">.</span><span class="n">File</span><span class="p">(</span><span class="s">"sums.dml"</span><span class="p">)</span><span class="o">.</span><span class="n">writeAll</span><span class="p">(</span><span class="n">sums</span><span class="p">)</span> -<span class="n">val</span> <span class="n">sumScript</span> <span class="o">=</span> <span class="n">dmlFromFile</span><span class="p">(</span><span class="s">"sums.dml"</span><span class="p">)</span><span class="o">.</span><span class="ow">in</span><span class="p">(</span><span class="n">Map</span><span class="p">(</span><span class="s">"m1"</span><span class="o">-></span> <span class="n">rdd1</span><span class="p">,</span> <span class="s">"m2"</span><span class="o">-></span> <span class="n">rdd2</span><span class="p">))</span><span class="o">.</span><span class="n">out</span><span class="p">(</span><span class="s">"s1"</span><span class="p">,</span> <span class="s">"s2"</span><span class="p">,</span> <span class="s">"message"</span><span class="p">)</span> -<span class="n">val</span> <span class="n">sumResults</span> <span class="o">=</span> <span class="n">ml</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">sumScript</span><span class="p">)</span> -<span class="n">val</span> <span class="n">s1</span> <span class="o">=</span> <span class="n">sumResults</span><span class="o">.</span><span class="n">getDouble</span><span class="p">(</span><span class="s">"s1"</span><span class="p">)</span> -<span class="n">val</span> <span class="n">s2</span> <span class="o">=</span> <span class="n">sumResults</span><span class="o">.</span><span class="n">getDouble</span><span class="p">(</span><span class="s">"s2"</span><span class="p">)</span> -<span class="n">val</span> <span class="n">message</span> <span class="o">=</span> <span class="n">sumResults</span><span class="o">.</span><span class="n">getString</span><span class="p">(</span><span class="s">"message"</span><span class="p">)</span> -<span class="n">val</span> <span class="n">rdd1Metadata</span> <span class="o">=</span> <span class="n">new</span> <span class="n">MatrixMetadata</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span> -<span class="n">val</span> <span class="n">rdd2Metadata</span> <span class="o">=</span> <span class="n">new</span> <span class="n">MatrixMetadata</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span> -<span class="n">val</span> <span class="n">sumScript</span> <span class="o">=</span> <span class="n">dmlFromFile</span><span class="p">(</span><span class="s">"sums.dml"</span><span class="p">)</span><span class="o">.</span><span class="ow">in</span><span class="p">(</span><span class="n">Seq</span><span class="p">((</span><span class="s">"m1"</span><span class="p">,</span> <span class="n">rdd1</span><span class="p">,</span> <span class="n">rdd1Metadata</span><span class="p">),</span> <span class="p">(</span><span class="s">"m2"</span><span class="p">,</span> <span class="n">rdd2</span><span class="p">,</span> <span class="n">rdd2Metadata</span><span class="p">)))</span><span class="o">.</span><span class="n">out</span><span class="p">(</span><span class="s">"s1"</span><span class="p">,</span> <span class="s">"s2"</span><span class="p">,</span> <span class="s">"message"</span><span class="p">)</span> -<span class="n">val</span> <span class="p">(</span><span class="n">firstSum</span><span class="p">,</span> <span class="n">secondSum</span><span class="p">,</span> <span class="n">sumMessage</span><span class="p">)</span> <span class="o">=</span> <span class="n">ml</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">sumScript</span><span class="p">)</span><span class="o">.</span><span class="n">getTuple</span><span class="p">[</span><span class="n">Double</span><span class="p">,</span> <span class="n">Double</span><span class="p">,</span> <span class="n">String</span><span class="p">](</span><span class="s">"s1"</span><span class="p">,</span> <span class="s">"s2"</span><span class="p">,</span> <span class="s">"message"</span><span class="p">)</span></code></pre></figure> - <p>Congratulations! Youâve now run examples in Apache SystemML!</p> - </div> - </div> - - - - </section> </div>
Modified: incubator/systemml/site/index.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/index.html?rev=1795636&r1=1795635&r2=1795636&view=diff ============================================================================== --- incubator/systemml/site/index.html (original) +++ incubator/systemml/site/index.html Fri May 19 21:48:49 2017 @@ -175,7 +175,7 @@ <!-- About --> <section class="full-stripe"> <div class="ml-container ml-container--vertically-top ml-container--reverse-order"> - <div class="col col-6 content-group content-group--more-padding"> + <div class="col col-6 content-group"> <!--<img src="/assets/img/diagramAnim-v4.gif" alt="What is Apache SystemML?">--> <div class="video-wrapper"> <iframe src="https://www.youtube.com/embed/r-kFYkYoD_4"; frameborder="0" allowfullscreen></iframe> @@ -184,14 +184,12 @@ <div class="col col-6 content-group content-group--more-padding"> <!-- bdi tag prevents reverse punctuation when rtl direction property is applied --> <h2><bdi>What is SystemML?</bdi></h2> - <p><bdi>Apache SystemML provides declarative, large-scalable machine learning and deep learning. Data scientists are able to implement algorithms and neural network architectures in a high-level language without knowledge of distributed programming or Apache Spark. Depending on data characteristics such as data size/shape and data sparsity (dense/sparse), and cluster characteristics such as cluster size and memory configurations, SystemML's cost-based optimizing compiler automatically generates hybrid runtime execution plans that are composed of single-node and distributed operations on a Apache Spark cluster for best performance.</bdi></p> - - <p><bdi>Very soon, additional deep learning capabilities will allow for importing and running popular neural network architectures and pre-trained models from Caffe for training and scoring in SystemML.</bdi></p> + <p>Apache SystemML provides an optimal workplace for machine learning using big data. It can be run on top of Apache Spark, where it automatically scales your data, line by line, determining whether your code should be run on the driver or an Apache Spark cluster. Future SystemML developments include additional deep learning with GPU capabilities such as importing and running neural network architectures and pre-trained models for training.</p> </div> </div> </section> -<!-- Beginner Tutorial --> +<!-- BGet Started --> <section class="full-stripe full-stripe--alternate"> <div class="ml-container ml-container--vertically-centered"> <div class="col col-6 content-group content-group--more-padding"> @@ -200,24 +198,27 @@ <div class="col col-6 content-group content-group--more-padding button-group"> <h2>Get Started</h2> <p>New to Apache SystemML? Try our Get Started tutorial that will walk you through setting up your environment and getting you up and going with SystemML.</p> - <a class="button button-primary" href="get-started.html">Begin Tutorial</a> + <a class="button button-primary" href="install-systemml.html">Install SystemML</a> <a class="button button-secondary" href="documentation.html" target="_blank">Docs</a> </div> </div> + + <h4 class="text-center"><a href="get-started.html">View Sample Notebooks</a></h4> + </section> <!-- Contact Us --> <section class="full-stripe"> <div class="ml-container ml-container--horizontally-center"> <div class="col col-12 content-group "> - <h2>Subscribe to Our Mailing Lists</h2> - <p>Subscribe to our development mailing list for SystemML updates and news. Once subscribed, <a href="mailto:[email protected]";>join the conversation</a>. - As SystemML grows, so will our community. Check out <a href="community.html#mailing-list">All Mailing Lists</a>. + <h2>Contribute</h2> + <p>Contribute to Apache SystemML<sup>TM</sup> by subscribing to our developer mailing list for updates and news. Check out <a href="community.html#mailing-list">All Mailing Lists</a>. </p> </div> <div class="col col-12 content-group button-group"> <a href="mailto:[email protected]?subject=send this email to subscribe" class="button button-primary">Subscribe</a> <a href="http://mail-archives.apache.org/mod_mbox/incubator-systemml-dev/"; target="_blank" class="button button-secondary">View Archive</a> + <a href="https://systemml.apache.org/roadmap"; target="_blank" class="button button-secondary">Roadmap</a> </div> </div> </section> Added: incubator/systemml/site/install-systemml.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/install-systemml.html?rev=1795636&view=auto ============================================================================== --- incubator/systemml/site/install-systemml.html (added) +++ incubator/systemml/site/install-systemml.html Fri May 19 21:48:49 2017 @@ -0,0 +1,325 @@ +<!-- + +--> +<!-- + +--> +<!DOCTYPE html> +<html lang="en"> + <!-- + +--> +<head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + + <title>Get Started</title> + + <meta name="description" content="Get-Started Page"> + + <meta name="author" content="Apache SystemML"> + + <!-- Enable responsive viewport --> + <meta name="HandheldFriendly" content="True"> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + + <!-- You can use Open Graph tags to customize link previews. + Learn more: https://developers.facebook.com/docs/sharing/webmasters --> + <meta property="og:url" content="https://systemml.apache.org/"; /> + <meta property="og:type" content="website" /> + <meta property="og:title" content="Get Started" /> + <meta property="og:description" content="Apache SystemML provides an optimal workplace for Machine Learning using big data" /> + <meta property="og:image" content="" /> + + <!-- Le HTML5 shim, for IE6-8 support of HTML elements --> + <!--[if lt IE 9]> + <script src="http://html5shim.googlecode.com/svn/trunk/html5.js";></script> + <![endif]--> + + <!-- Le styles --> + <link rel="stylesheet" href="/assets/css/main.css"> + + <!-- favicons --> + <link rel="shortcut icon" href="/assets/img/favicon.png"> +</head> + <!-- META --> + <body class="vcard"> + <!-- + +--> +<header class="site-header site-header--not-home"> + <h1 class="logo"><a class="url" href="/"><i class="logo-mark"></i><span class="fn org">Apache SystemML<sup id="trademark">™</sup></span></a></h1> + <nav class="main-nav"> + <ul> + + <li role="presentation"> + + + <a href="/download" target="_self">Download</a> + + </li> + + <li role="presentation"> + + + <a href="/get-started" target="_self">Get Started</a> + + </li> + + <li role="presentation"> + + + <a href="/documentation" target="_self">Docs</a> + + </li> + + <li role="presentation"> + + + <a href="/roadmap" target="_self">Roadmap</a> + + </li> + + <li role="presentation"> + + <a class="nav-link--hover">Community <i class="icon icon-chevron-down"></i></a> + <ul> + + + <li><a href="/community" target="_self">Get Involved</a></li> + + + <li><a href="https://issues.apache.org/jira/browse/SYSTEMML"; target="_blank">Issue Tracker</a></li> + + + <li><a href="https://github.com/apache/incubator-systemml"; target="_blank">Source Code</a></li> + + + <li><a href="https://github.com/apache/incubator-systemml-website"; target="_blank">Website Source Code</a></li> + + + <li><a href="/roadmap" target="_self">Roadmap</a></li> + + </ul> + + </li> + + <li role="presentation"> + + <a class="nav-link--hover">Apache <i class="icon icon-chevron-down"></i></a> + <ul> + + + <li><a href="http://www.apache.org/foundation/how-it-works.html"; target="_blank">Apache Software Foundation</a></li> + + + <li><a href="http://www.apache.org/licenses/"; target="_blank">Apache License</a></li> + + + <li><a href="http://www.apache.org/foundation/sponsorship"; target="_blank">Sponsorship</a></li> + + + <li><a href="http://www.apache.org/foundation/thanks.html"; target="_blank">Thanks</a></li> + + + <li><a href="/privacy-policy" target="_self">Privacy Policy</a></li> + + + <li><a href="/security" target="_self">Security</a></li> + + </ul> + + </li> + + </ul> + </nav> +</header> + <!-- GLOBAL HEADER --> + <!-- + +--> +<!-- + +--> +<div> + <!-- + +--> + +<!-- Hero --> +<!-- <section class="full-stripe full-stripe--subpage-header clear-header"> + <div class="ml-container ml-container--horizontally-center"> + <div class="col col-12 content-group"> + <h1>Tutorials</h1> + </div> + </div> +</section> --> + + +<!-- Tutorial Instructions --> +<section class="full-stripe full-stripe--alternate"> + + <!-- Section 1 --> + <div class="ml-container content-group content-group--tutorial border"> + <!-- Section Header --> + <div class="col col-12 content-group--medium-bottom-margin"> + <h2>Install SystemML</h2> + </div> + + <!-- Step 1 Instructions --> + <div class="col col-12"> + <h3><span class="circle">1</span>Pre-requisite</h3> + </div> + + <!-- Step 1 Code --> + <div class="col col-12"> + + <p class="indent">Apache Spark 2.x</p> + <p class="indent">Set SPARK_HOME to a location where Spark 2.x has installed.</p> + + </div> + + <!-- Step 2 Instructions --> + <div class="col col-12"> + <h3><span class="circle">2</span>Setup</h3> + </div> + + <!-- Step 2 Code --> + <ul class="ml-tabs"> + <li class="tab-link current" data-tab="tab-1">Python</li> + <li class="tab-link" data-tab="tab-2">Scala</li> + <li class="tab-link" data-tab="tab-3">Dev Python (Latest code)</li> + <li class="tab-link" data-tab="tab-4">Dev Scala (Latest code)</li> + </ul> + + <div id="tab-1" class="col col-12 tab-content current"> + <figure class="highlight"><pre><code class="language-bash" data-lang="bash"> <span class="c"># Install SystemML</span> + pip install systemml + </code></pre></figure> + </code> + </div> + <div id="tab-2" class="col col-12 tab-content"> + <pre>Download and extract SystemML jar (systemml-0.14.0-incubating-SNAPSHOT.jar) file from systemml-0.14.0-incubating-bin.tgz or systemml-0.14.0-incubating-bin.tgz file located on <a href="https://dist.apache.org/repos/dist/release/incubator/systemml/0.14.0-incubating/";>https://dist.apache.org/repos/dist/release/incubator/systemml/0.14.0-incubating/</a></pre> + </div> + <div id="tab-3" class="col col-12 tab-content"> + <figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># Install latest SystemML</span> +pip install https://sparktc.ibmcloud.com/repo/latest/systemml-1.0.0-incubating-SNAPSHOT-python.tgz</code></pre></figure> + </div> + <div id="tab-4" class="col col-12 tab-content"> + <pre>Download SystemML jar (systemml-1.0.0-incubating-SNAPSHOT.jar) from <a href="https://sparktc.ibmcloud.com/repo/latest/";>https://sparktc.ibmcloud.com/repo/latest/</a></pre> + </div> + + <!-- Step 3 Instructions --> + <div class="col col-12"> + <h3><span class="circle">3</span>Configure Jupyter Notebook (optional)</h3> + </div> + + <!-- Step 3 Code --> + <div class="col col-12"> + <h4 class="indent">3.1 Toree Kernel Setup (Required for Scala Kernel)</h4> + <p class="indent">Toree installation</p> + <figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># For detail instructions visit https://github.com/apache/incubator-toree</span> +pip install https://dist.apache.org/repos/dist/dev/incubator/toree/0.2.0/snapshots/dev1/toree-pip/toree-0.2.0.dev1.tar.gz</code></pre></figure> + + <p class="indent">Installation of Toree component in Jupyter</p> + <figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># For detail instructions visit https://toree.apache.org/docs/current/user/installation/</span> +jupyter toree install â-replace â-interpreters<span class="o">=</span>Scala,PySpark --spark_opts<span class="o">=</span><span class="s2">"--master=local --jars <SystemML JAR File>â --spark_home=</span><span class="k">${</span><span class="nv">SPARK_HOME</span><span class="k">}</span></code></pre></figure> + <h4 class="indent">3.2 Start Jupyter Notebook Server</h4> + <figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">PYSPARK_DRIVER_PYTHON</span><span class="o">=</span>jupyter <span class="nv">PYSPARK_DRIVER_PYTHON_OPTS</span><span class="o">=</span><span class="s2">"notebook"</span> pyspark --master <span class="nb">local</span><span class="o">[</span><span class="k">*</span><span class="o">]</span> --conf <span class="s2">"spark.driver.memory=12g"</span> --conf spark.driver.maxResultSize<span class="o">=</span>0 --conf spark.akka.frameSize<span class="o">=</span>128 --conf spark.default.parallelism<span class="o">=</span>100</code></pre></figure> + <p>This will a default browser with contents from current directory where above command has run. +You can create your own notebook example or download sample notebooks from SystemML resository <a href="https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks";>https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks</a></p> + <figure class="img-border"><img src="/assets/img/systemml-juypter-install.jpeg" alt="Start Jupyter Notebook Server"></figure> + <figure class="img-border"><img src="/assets/img/systemml-juypter-install-2.jpeg" alt="Start Jupyter Notebook Server"></figure> + </div> + + <!-- Step 4 Instructions --> + <div class="col col-12"> + <h3><span class="circle">4</span>Run SystemML in batch mode</h3> + </div> + + <!-- Step 4 Code --> + <div class="col col-12"> + + <prev>Download systemml-0.14.0-incubating-bin.tgz or systemml-0.14.0-incubating-bin.tgz file located on <a href="https://dist.apache.org/repos/dist/release/incubator/systemml/0.14.0-incubating/";>https://dist.apache.org/repos/dist/release/incubator/systemml/0.14.0-incubating/</a> and extract into a directory say SYSTEMML_HOME +Once you extract zip.tgz file you will have files required to run steps outlined in instructions link: <a href="http://apache.github.io/incubator-systemml/spark-batch-mode";>http://apache.github.io/incubator-systemml/spark-batch-mode</a></pre> + + </div> + </div> + + <h4 class="text-center"><a href="get-started.html">View Sample Notebooks</a></h4> + + <h2 class="text-center">Sample Notebooks</h2> + <div class="flex-container"> + <div class="nb-card"> + <h3>SystemML LinearRegCG</h3> + <p>SystemML Linear Regression in Zeppelin Notebook.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/SystemML_LinearRegCG.json"; target="_blank"><span class="icon zeppelin-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>Deep Learning Image Classification</h3> + <p>This notebook shows SystemML Deep Learning functionality to map images of single digit numbers to their corresponding numeric representations.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>Linear Regression Algorithms Demo</h3> + <p>This notebook shows: Install SystemML Python package and jar file, pip, SystemML 'Hello World'.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>SystemML PySpark Recommendation Demo</h3> + <p>This demonstrates using SystemML for product recommendation using Poisson NonNegative Matrix Factorization (PNMF) with PNMF algorithm written using R like syntax.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/SystemML-PySpark-Recommendation-Demo.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>SystemML Scala Tutorial</h3> + <p>This tutorial includes simple example to run DML script and display output.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/tutorial1.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>Autoencoder</h3> + <p>This notebook demonstrates the invocation of the SystemML autoencoder script, and alternative ways of passing in/out data.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Autoencoder.ipynb"; target="_blank"><span class="icon jupyter-logo"></span><span>View on Github</span></a> + </div> + <div class="nb-card"> + <h3>SystemML Zeppelin Tutorial</h3> + <p>SystemML Zeppelin tutorial.</p> + <a class="nb-link" href="https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/tutorial1_zeppelin.json"; target="_blank"><span class="icon zeppelin-logo"></span><span>View on Github</span></a> + </div> + + </div> + +</section> + +</div> + + + <!-- + +--> +<footer class="site-footer"> + <div class="ml-container"> + <div class="footer-col-wrapper"> + <div style="margin-top: 20px; margin-bottom: 20px;"> + <a href="http://incubator.apache.org";><img width="350px;" src="/assets/img/apache-foundation-logo.svg" alt="Apache Incubator"/></a> + </div> + <div> + <p>Copyright © 2015-2017 <a href="http://www.apache.org";>The Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a>. Apache and the Apache Feather logo are trademarks of The Apache Software Foundation.</p> + </div> + </div> + </div> + <script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); + + ga('create', 'UA-71553733-1', 'auto'); + ga('send', 'pageview'); +</script> + +</footer> + <!-- GLOBAL FOOTER--> + <script src="assets/js/bundle.min.js"></script> + </body> +</html> +
