sebwrede commented on a change in pull request #1061: URL: https://github.com/apache/systemds/pull/1061#discussion_r493347509
########## File path: src/main/python/docs/source/guide/algorithms.rst ########## @@ -26,18 +26,176 @@ Prerequisite: - :doc:`/getting_started/install` +This example goes through an algorithm from the list of builtin algorithms that can be applied to a dataset. +For simplicity the dataset used for this is `MNist <http://yann.lecun.com/exdb/mnist/>`_, +since it is commonly known and explored. + +If one want to skip the explanation then the full script is available in to bottom of this page. Review comment: > If one want* to *wants > available (in to)* bottom *at the ########## File path: src/main/python/docs/source/guide/algorithms.rst ########## @@ -26,18 +26,176 @@ Prerequisite: - :doc:`/getting_started/install` +This example goes through an algorithm from the list of builtin algorithms that can be applied to a dataset. +For simplicity the dataset used for this is `MNist <http://yann.lecun.com/exdb/mnist/>`_, +since it is commonly known and explored. + +If one want to skip the explanation then the full script is available in to bottom of this page. Step 1: Get Dataset ------------------- -TODO +Systemds provide builtin for downloading and setup of the mnist dataset. Review comment: > Systemds provide* *provides ########## File path: src/main/python/docs/source/guide/algorithms.rst ########## @@ -26,18 +26,176 @@ Prerequisite: - :doc:`/getting_started/install` +This example goes through an algorithm from the list of builtin algorithms that can be applied to a dataset. +For simplicity the dataset used for this is `MNist <http://yann.lecun.com/exdb/mnist/>`_, +since it is commonly known and explored. + +If one want to skip the explanation then the full script is available in to bottom of this page. Step 1: Get Dataset ------------------- -TODO +Systemds provide builtin for downloading and setup of the mnist dataset. +To setup this simply use:: -Step 2: Train model -------------------- + from systemds.examples.tutorials.mnist import DataManager + d = DataManager() + X = d.get_train_data() + Y = d.get_train_labels() + +Here the DataManager contains the code for downloading and setting up numpy arrays containing the data. + +Step 2: Reshape & Format +------------------------ + +Usually data does not come in formats that perfectly fits the algorithms, to make this tutorial more +realistic some data preprocessing is required to change the input to fit. + +First the Training data, X, has multiple dimensions resulting in a shape (60000, 28, 28). +these dimensions corresponds to first the number of images 60000, then the number of row pixels, 28, Review comment: *correspond ########## File path: src/main/python/docs/source/guide/algorithms.rst ########## @@ -26,18 +26,176 @@ Prerequisite: - :doc:`/getting_started/install` +This example goes through an algorithm from the list of builtin algorithms that can be applied to a dataset. +For simplicity the dataset used for this is `MNist <http://yann.lecun.com/exdb/mnist/>`_, +since it is commonly known and explored. + +If one want to skip the explanation then the full script is available in to bottom of this page. Step 1: Get Dataset ------------------- -TODO +Systemds provide builtin for downloading and setup of the mnist dataset. +To setup this simply use:: -Step 2: Train model -------------------- + from systemds.examples.tutorials.mnist import DataManager + d = DataManager() + X = d.get_train_data() Review comment: I get: `>>> X = d.get_train_data() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/.local/lib/python3.8/site-packages/systemds/examples/tutorials/mnist.py", line 56, in get_train_data self._get_data(self._train_data_url, self._train_data_loc) File "/.local/lib/python3.8/site-packages/systemds/examples/tutorials/mnist.py", line 102, in _get_data os.mkdir(folder) FileNotFoundError: [Errno 2] No such file or directory: 'systemds/examples/tutorials/mnist' ` ########## File path: src/main/python/docs/source/guide/algorithms.rst ########## @@ -26,18 +26,176 @@ Prerequisite: - :doc:`/getting_started/install` +This example goes through an algorithm from the list of builtin algorithms that can be applied to a dataset. +For simplicity the dataset used for this is `MNist <http://yann.lecun.com/exdb/mnist/>`_, +since it is commonly known and explored. + +If one want to skip the explanation then the full script is available in to bottom of this page. Step 1: Get Dataset ------------------- -TODO +Systemds provide builtin for downloading and setup of the mnist dataset. +To setup this simply use:: -Step 2: Train model -------------------- + from systemds.examples.tutorials.mnist import DataManager + d = DataManager() + X = d.get_train_data() + Y = d.get_train_labels() + +Here the DataManager contains the code for downloading and setting up numpy arrays containing the data. + +Step 2: Reshape & Format +------------------------ + +Usually data does not come in formats that perfectly fits the algorithms, to make this tutorial more +realistic some data preprocessing is required to change the input to fit. + +First the Training data, X, has multiple dimensions resulting in a shape (60000, 28, 28). +these dimensions corresponds to first the number of images 60000, then the number of row pixels, 28, Review comment: *These ########## File path: src/main/python/docs/source/guide/algorithms.rst ########## @@ -26,18 +26,176 @@ Prerequisite: - :doc:`/getting_started/install` +This example goes through an algorithm from the list of builtin algorithms that can be applied to a dataset. +For simplicity the dataset used for this is `MNist <http://yann.lecun.com/exdb/mnist/>`_, +since it is commonly known and explored. + +If one want to skip the explanation then the full script is available in to bottom of this page. Step 1: Get Dataset ------------------- -TODO +Systemds provide builtin for downloading and setup of the mnist dataset. +To setup this simply use:: -Step 2: Train model -------------------- + from systemds.examples.tutorials.mnist import DataManager + d = DataManager() + X = d.get_train_data() + Y = d.get_train_labels() + +Here the DataManager contains the code for downloading and setting up numpy arrays containing the data. + +Step 2: Reshape & Format +------------------------ + +Usually data does not come in formats that perfectly fits the algorithms, to make this tutorial more +realistic some data preprocessing is required to change the input to fit. + +First the Training data, X, has multiple dimensions resulting in a shape (60000, 28, 28). +these dimensions corresponds to first the number of images 60000, then the number of row pixels, 28, +and finally the column pixels, 28. + +To use this data for Logistic Regression we have to reduce the dimensions. +The input, X, of this algorithm require the data, to have two dimensions, and the first resemble the Review comment: Check commas ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
