Repository: incubator-systemml Updated Branches: refs/heads/master 700b08094 -> fc612f2e0
[SYSTEMML-1606] Update notebook samples with latest code Updated Deep Learning notebook compatible with 0.14 release code Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/fc612f2e Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/fc612f2e Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/fc612f2e Branch: refs/heads/master Commit: fc612f2e08f8bab9daa74a451c0b618c2d92543d Parents: 700b080 Author: Arvind Surve <[email protected]> Authored: Sun May 21 17:38:56 2017 -0700 Committer: Arvind Surve <[email protected]> Committed: Sun May 21 17:38:56 2017 -0700 ---------------------------------------------------------------------- .../Deep_Learning_Image_Classification.ipynb | 96 ++++++-------------- 1 file changed, 27 insertions(+), 69 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/fc612f2e/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb ---------------------------------------------------------------------- diff --git a/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb b/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb index 4285acd..996c5d0 100644 --- a/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb +++ b/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb @@ -4,13 +4,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Deep Learning Image Classification\n", + "# Deep Learning Image Classification using Apache SystemML\n", "\n", "This notebook shows SystemML Deep Learning functionality to map images of single digit numbers to their corresponding numeric representations. See [Getting Started with Deep Learning and Python](http://www.pyimagesearch.com/2014/09/22/getting-started-deep-learning-python/) for an explanation of the used deep learning concepts and assumptions.\n", "\n", "The downloaded MNIST dataset contains labeled images of handwritten digits, where each example is a 28x28 pixel image of grayscale values in the range [0,255] stretched out as 784 pixels, and each label is one of 10 possible digits in [0,9]. We download 60,000 training examples, and 10,000 test examples, where the format is \"label, pixel_1, pixel_2, ..., pixel_n\". We train a SystemML LeNet model. The results of the learning algorithms have an accuracy of 98 percent.\n", "\n", - "1. [Install and load SystemML and other libraries](#load_systemml)\n", "1. [Download and Access MNIST data](#access_data)\n", "1. [Train a CNN classifier for MNIST handwritten digits](#train)\n", "1. [Detect handwritten Digits](#predict)\n" @@ -32,39 +31,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "<a id=\"load_systemml\"></a>\n", - "## Install and load SystemML and other libraries\n", - "\n", - "You can install SystemML Python tgz file either from \n", - " 1. Distribution location (https://dist.apache.org/repos/dist/release/incubator/systemml/) or \n", - " 2. Latest daily built code (https://sparktc.ibmcloud.com/repo/latest/) or\n", - " 3. From your local system if you have extracted recent code and built locally. \n", - " (e.g. ~/git/incubator-systemml/target/systemml-1.0.0-incubating-SNAPSHOT-python.tgz)\n", - " \n", - "### This notebook is supported with master branch (SystemML 1.0.0) as of 05/15/2017 and later code." + "### This notebook is supported with SystemML 0.14.0 and above." ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "collapsed": true, - "scrolled": false - }, - "outputs": [], - "source": [ - "# !pip install --user systemml>=1.0.0\n", - "\n", - "!pip install https://sparktc.ibmcloud.com/repo/latest/systemml-1.0.0-incubating-SNAPSHOT-python.tgz\n";, - " \n", - "# !pip install ~/git/incubator-systemml/target/systemml-1.0.0-incubating-SNAPSHOT-python.tgz" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "collapsed": true, "scrolled": false }, "outputs": [], @@ -75,9 +48,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "from systemml import MLContext, dml\n", @@ -93,7 +64,6 @@ "cell_type": "code", "execution_count": null, "metadata": { - "collapsed": true, "scrolled": true }, "outputs": [], @@ -144,7 +114,6 @@ "cell_type": "code", "execution_count": null, "metadata": { - "collapsed": true, "scrolled": true }, "outputs": [], @@ -172,21 +141,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Alternatively get the data from here." + "Alternatively get the data from here. (Uncomment curl commands from following cell if you want to download using following approach)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "collapsed": true, "scrolled": true }, "outputs": [], "source": [ "%%sh\n", - "curl -O https://pjreddie.com/media/files/mnist_train.csv\n";, - "curl -O https://pjreddie.com/media/files/mnist_test.csv\n";, + "cd data/mnist\n", + "# curl -O https://pjreddie.com/media/files/mnist_train.csv\n";, + "# curl -O https://pjreddie.com/media/files/mnist_test.csv\n";, "wc -l mnist*" ] }, @@ -200,9 +169,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "trainData = np.genfromtxt('data/mnist/mnist_train.csv', delimiter=\",\")\n", @@ -215,9 +182,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "pd.set_option('display.max_columns', 200)\n", @@ -228,6 +193,22 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "### Following command is not required for code above SystemML 0.14 (master branch dated 05/15/2017 or later)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!svn --force export https://github.com/apache/incubator-systemml/trunk/scripts/nn"; + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ "<a id=\"train\"></a>\n", "## Develop LeNet CNN classifier on Training Data" ] @@ -308,9 +289,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "scriptPredict = \"\"\"\n", @@ -382,7 +361,6 @@ "cell_type": "code", "execution_count": null, "metadata": { - "collapsed": true, "scrolled": true }, "outputs": [], @@ -399,32 +377,12 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "collapsed": true - }, + "metadata": {}, "outputs": [], "source": [ "pd.set_option('display.max_columns', 28)\n", "pd.DataFrame((testData[i,1:]).reshape(img_size, img_size),dtype='uint')" ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Uninstall/Clean up SystemML Python package and jar file" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "!yes | pip uninstall systemml" - ] } ], "metadata": {
