[SYSTEMML-1239] DML Language Reference write description parameter Give example of write description parameter in DML Language Reference. Describe author and created parameters. Update metadata examples in DML Language Reference and Beginner's Guide to DML and PyDML.
Closes #385. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/452a41a0 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/452a41a0 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/452a41a0 Branch: refs/heads/gh-pages Commit: 452a41a0257956b3b0d20cf894f37f2d3e20a90f Parents: 1695060 Author: Deron Eriksson <[email protected]> Authored: Mon Feb 13 12:44:10 2017 -0800 Committer: Deron Eriksson <[email protected]> Committed: Mon Feb 13 12:44:10 2017 -0800 ---------------------------------------------------------------------- beginners-guide-to-dml-and-pydml.md | 59 +++++++++++++++++--------------- dml-language-reference.md | 56 +++++++++++++++++++++++------- 2 files changed, 74 insertions(+), 41 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/452a41a0/beginners-guide-to-dml-and-pydml.md ---------------------------------------------------------------------- diff --git a/beginners-guide-to-dml-and-pydml.md b/beginners-guide-to-dml-and-pydml.md index 67f1c96..8479ef7 100644 --- a/beginners-guide-to-dml-and-pydml.md +++ b/beginners-guide-to-dml-and-pydml.md @@ -283,14 +283,15 @@ is 0-based.* </div> <div data-lang="m.txt.mtd" markdown="1"> - { - "data_type": "matrix" - ,"value_type": "double" - ,"rows": 4 - ,"cols": 3 - ,"nnz": 6 - ,"format": "text" - ,"description": { "author": "SystemML" } + { + "data_type": "matrix", + "value_type": "double", + "rows": 4, + "cols": 3, + "nnz": 6, + "format": "text", + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } </div> @@ -313,16 +314,17 @@ is 0-based.* </div> <div data-lang="m.csv.mtd" markdown="1"> - { - "data_type": "matrix" - ,"value_type": "double" - ,"rows": 4 - ,"cols": 3 - ,"nnz": 6 - ,"format": "csv" - ,"header": false - ,"sep": "," - ,"description": { "author": "SystemML" } + { + "data_type": "matrix", + "value_type": "double", + "rows": 4, + "cols": 3, + "nnz": 6, + "format": "csv", + "header": false, + "sep": ",", + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } </div> @@ -331,16 +333,17 @@ is 0-based.* </div> <div data-lang="m.binary.mtd" markdown="1"> - { - "data_type": "matrix" - ,"value_type": "double" - ,"rows": 4 - ,"cols": 3 - ,"rows_in_block": 1000 - ,"cols_in_block": 1000 - ,"nnz": 6 - ,"format": "binary" - ,"description": { "author": "SystemML" } + { + "data_type": "matrix", + "value_type": "double", + "rows": 4, + "cols": 3, + "rows_in_block": 1000, + "cols_in_block": 1000, + "nnz": 6, + "format": "binary", + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } </div> http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/452a41a0/dml-language-reference.md ---------------------------------------------------------------------- diff --git a/dml-language-reference.md b/dml-language-reference.md index 05625fd..22ec0d9 100644 --- a/dml-language-reference.md +++ b/dml-language-reference.md @@ -933,7 +933,8 @@ Below, we have examples of this matrix in the CSV, Matrix Market, IJV, and Binar "format": "csv", "header": false, "sep": ",", - "description": { "author": "SystemML" } + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } </div> @@ -965,7 +966,8 @@ Below, we have examples of this matrix in the CSV, Matrix Market, IJV, and Binar "cols": 3, "nnz": 6, "format": "text", - "description": { "author": "SystemML" } + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } </div> @@ -983,7 +985,8 @@ Below, we have examples of this matrix in the CSV, Matrix Market, IJV, and Binar "cols_in_block": 1000, "nnz": 6, "format": "binary", - "description": { "author": "SystemML" } + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } </div> @@ -992,12 +995,13 @@ Below, we have examples of this matrix in the CSV, Matrix Market, IJV, and Binar As another example, here we see the content of the MTD file `scalar.mtd` associated with a scalar data file `scalar` that contains the scalar value 2.0. - { - "data_type": "scalar", - "value_type": "double", - "format": "text", - "description": { "author": "SystemML" } - } + { + "data_type": "scalar", + "value_type": "double", + "format": "text", + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" + } Metadata is represented as an MTD file that contains a single JSON object with the attributes described below. @@ -1015,6 +1019,8 @@ Parameter Name | Description | Optional | Permissible values | Data type valid f `nnz` | Number of non-zero values | Yes | any integer > `0` | `matrix` `format` | Data file format | Yes. Default value is `text` | `csv`, `mm`, `text`, `binary` | `matrix`, `scalar`. Formats `csv` and `mm` are applicable only to matrices `description` | Description of the data | Yes | Any valid JSON string or object | `matrix`, `scalar` +`author` | User that created the metadata file, defaults to `SystemML` | N/A | N/A | N/A +`created` | Date/time when metadata file was written | N/A | N/A | N/A In addition, when reading or writing CSV files, the metadata may contain one or more of the following five attributes. @@ -1126,7 +1132,8 @@ Example content of `out/file.ijv.mtd`: "cols": 8, "nnz": 4, "format": "text", - "description": { "author": "SystemML" } + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } Write `V` to `out/file` in `binary` format: @@ -1144,7 +1151,8 @@ Example content of `out/file.mtd`: "rows_in_block": 1000, "cols_in_block": 1000, "format": "binary", - "description": { "author": "SystemML" } + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } Write `V` to `n.csv` in `csv` format with column headers, `";"` as delimiter, and zero values are not written. @@ -1162,7 +1170,8 @@ Example content of `n.csv.mtd`: "format": "csv", "header": true, "sep": ";", - "description": { "author": "SystemML" } + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } Write `x` integer value to file `out/scalar_i` @@ -1175,7 +1184,8 @@ Example content of `out/scalar_i.mtd`: "data_type": "scalar", "value_type": "int", "format": "text", - "description": { "author": "SystemML" } + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" } Unlike `read`, the `write` function does not need a constant string expression, so the following example will work: @@ -1186,6 +1196,26 @@ Unlike `read`, the `write` function does not need a constant string expression, file = "A" + i + ".mtx"; write(A, dir + file, format="csv"); +The `description` parameter can be used to attach a description to the metadata: + + A = matrix("1 2 3 4", rows=2, cols=2) + write(A, "mymatrix.csv", format="csv", description="my matrix") + +This will generate the following `mymatrix.csv.mtd` metadata file: + + { + "data_type": "matrix", + "value_type": "double", + "rows": 2, + "cols": 2, + "nnz": 4, + "format": "csv", + "header": false, + "sep": ",", + "description": "my matrix", + "author": "SystemML", + "created": "2017-01-01 00:00:01 PST" + } ### Data Pre-Processing Built-In Functions
