incubator-systemml git commit: [SYSTEMML-757] Removed abego, ANTLR, SLF4J from NOTICE Closes #180.
Repository: incubator-systemml Updated Branches: refs/heads/master bd0f71159 -> ba260e947 [SYSTEMML-757] Removed abego, ANTLR, SLF4J from NOTICE Closes #180. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/ba260e94 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/ba260e94 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/ba260e94 Branch: refs/heads/master Commit: ba260e94784f77fe8122d060a73465e2d75e0d94 Parents: bd0f711 Author: Glenn Weidner Authored: Mon Jun 20 15:03:31 2016 -0700 Committer: Glenn Weidner Committed: Mon Jun 20 15:03:31 2016 -0700 -- src/assembly/distrib/NOTICE| 9 - src/assembly/inmemory/NOTICE | 12 src/assembly/jar/NOTICE| 9 - src/assembly/standalone-jar/NOTICE | 12 src/assembly/standalone/NOTICE | 12 5 files changed, 54 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/ba260e94/src/assembly/distrib/NOTICE -- diff --git a/src/assembly/distrib/NOTICE b/src/assembly/distrib/NOTICE index 919b3c6..4ac75b5 100644 --- a/src/assembly/distrib/NOTICE +++ b/src/assembly/distrib/NOTICE @@ -3,12 +3,3 @@ Copyright [2015-2016] The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). - -This product includes abego software (http://www.abego.org) -Copyright (c) 2011, abego Software GmbH, Germany -All rights reserved. - -This product includes ANTLR software (http://www.antlr.org/) -Copyright (c) 2012 Terence Parr and Sam Harwell -All rights reserved. - http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/ba260e94/src/assembly/inmemory/NOTICE -- diff --git a/src/assembly/inmemory/NOTICE b/src/assembly/inmemory/NOTICE index 067f1bc..4ac75b5 100644 --- a/src/assembly/inmemory/NOTICE +++ b/src/assembly/inmemory/NOTICE @@ -3,15 +3,3 @@ Copyright [2015-2016] The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). - -This product includes abego software (http://www.abego.org) -Copyright (c) 2011, abego Software GmbH, Germany -All rights reserved. - -This product includes ANTLR software (http://www.antlr.org/) -Copyright (c) 2012 Terence Parr and Sam Harwell -All rights reserved. - -This product includes SLF4J software (http://www.slf4j.org/) -Copyright (c) 2004-2008 QOS.ch -All rights reserved. http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/ba260e94/src/assembly/jar/NOTICE -- diff --git a/src/assembly/jar/NOTICE b/src/assembly/jar/NOTICE index 919b3c6..4ac75b5 100644 --- a/src/assembly/jar/NOTICE +++ b/src/assembly/jar/NOTICE @@ -3,12 +3,3 @@ Copyright [2015-2016] The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). - -This product includes abego software (http://www.abego.org) -Copyright (c) 2011, abego Software GmbH, Germany -All rights reserved. - -This product includes ANTLR software (http://www.antlr.org/) -Copyright (c) 2012 Terence Parr and Sam Harwell -All rights reserved. - http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/ba260e94/src/assembly/standalone-jar/NOTICE -- diff --git a/src/assembly/standalone-jar/NOTICE b/src/assembly/standalone-jar/NOTICE index 067f1bc..4ac75b5 100644 --- a/src/assembly/standalone-jar/NOTICE +++ b/src/assembly/standalone-jar/NOTICE @@ -3,15 +3,3 @@ Copyright [2015-2016] The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). - -This product includes abego software (http://www.abego.org) -Copyright (c) 2011, abego Software GmbH, Germany -All rights reserved. - -This product includes ANTLR software (http://www.antlr.org/) -Copyright (c) 2012 Terence Parr and Sam Harwell -All rights reserved. - -This product includes SLF4J software (http://www.slf4j.org/) -Copyright (c) 2004-2008 QOS.ch -All rights reserved. http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/ba260e94/src/assembly/standalone/NOTICE -- diff --git a/src/assembly/standalone/NOTICE b/src/assembly/standalone/NOTICE index 067f1bc..4ac75b5 100644 --- a/src/assembly/standalone/NOTICE +++ b/src/assembly/standalone/NOTICE @@ -3,15 +3,3 @@ Copyright [2015-2016] The Apache Software Foundation This product includes s
incubator-systemml git commit: [SYSTEMML-770] Remove warning from GLM-predict.dml execution
Repository: incubator-systemml Updated Branches: refs/heads/master 640f65463 -> 7d4acddc0 [SYSTEMML-770] Remove warning from GLM-predict.dml execution Closes #181. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/7d4acddc Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/7d4acddc Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/7d4acddc Branch: refs/heads/master Commit: 7d4acddc0354b3afa2021844a049dd7808364807 Parents: 640f654 Author: Glenn Weidner Authored: Thu Jun 23 13:53:39 2016 -0700 Committer: Glenn Weidner Committed: Thu Jun 23 13:53:39 2016 -0700 -- scripts/algorithms/GLM-predict.dml| 3 +++ src/test/scripts/functions/jmlc/reuse-glm-predict.dml | 3 +++ 2 files changed, 6 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/7d4acddc/scripts/algorithms/GLM-predict.dml -- diff --git a/scripts/algorithms/GLM-predict.dml b/scripts/algorithms/GLM-predict.dml index 355928b..f58d8d0 100644 --- a/scripts/algorithms/GLM-predict.dml +++ b/scripts/algorithms/GLM-predict.dml @@ -164,6 +164,9 @@ if (fileY != " ") G2_scaled= 0.0 / 0.0; G2_scaled_pValue = 0.0 / 0.0; +# set Y_counts to avoid 'Initialization of Y_counts depends on if-else execution' warning +Y_counts = matrix(0.0, rows=1, cols=1); + if (dist_type == 1 & link_type == 1) { # # POWER DISTRIBUTIONS (GAUSSIAN, POISSON, GAMMA, ETC.) http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/7d4acddc/src/test/scripts/functions/jmlc/reuse-glm-predict.dml -- diff --git a/src/test/scripts/functions/jmlc/reuse-glm-predict.dml b/src/test/scripts/functions/jmlc/reuse-glm-predict.dml index 6ff0b68..01d2101 100644 --- a/src/test/scripts/functions/jmlc/reuse-glm-predict.dml +++ b/src/test/scripts/functions/jmlc/reuse-glm-predict.dml @@ -103,6 +103,9 @@ if (fileY != " ") G2_scaled= 0.0 / 0.0; G2_scaled_pValue = 0.0 / 0.0; +# set Y_counts to avoid 'Initialization of Y_counts depends on if-else execution' warning +Y_counts = matrix(0.0, rows=1, cols=1); + if (dist_type == 1 & link_type == 1) { # # POWER DISTRIBUTIONS (GAUSSIAN, POISSON, GAMMA, ETC.)
incubator-systemml git commit: [HOTFIX} Exclude new test file from license check
Repository: incubator-systemml Updated Branches: refs/heads/master 293c81c6c -> 464600e3e [HOTFIX} Exclude new test file from license check Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/464600e3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/464600e3 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/464600e3 Branch: refs/heads/master Commit: 464600e3e6a9b223897b33256afc2734cc1483d9 Parents: 293c81c Author: Glenn Weidner Authored: Fri Jun 24 09:05:02 2016 -0700 Committer: Glenn Weidner Committed: Fri Jun 24 09:05:02 2016 -0700 -- pom.xml | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/464600e3/pom.xml -- diff --git a/pom.xml b/pom.xml index 857566e..0c94376 100644 --- a/pom.xml +++ b/pom.xml @@ -601,6 +601,7 @@ src/test/scripts/functions/jmlc/tfmtd_example/Bin/sqft.bin src/test/scripts/functions/jmlc/tfmtd_example/column.names src/test/scripts/functions/jmlc/tfmtd_example/dummycoded.column.names + src/test/scripts/functions/jmlc/tfmtd_example2/column.names src/test/scripts/functions/jmlc/tfmtd_frame_example/tfmtd_frame src/test/scripts/functions/transform/input/homes/homesNAN.csv.colnames
incubator-systemml git commit: [SYSTEMML-771] Fix warnings in CsplineCG.dml and CsplineDS.dml
Repository: incubator-systemml Updated Branches: refs/heads/master be2632127 -> d8a8c3b69 [SYSTEMML-771] Fix warnings in CsplineCG.dml and CsplineDS.dml Closes #182. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/d8a8c3b6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/d8a8c3b6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/d8a8c3b6 Branch: refs/heads/master Commit: d8a8c3b69fdbd9e80d0ccef6c366f5b060ef9d45 Parents: be26321 Author: Glenn Weidner Authored: Fri Jun 24 15:28:04 2016 -0700 Committer: Glenn Weidner Committed: Fri Jun 24 15:28:04 2016 -0700 -- scripts/algorithms/CsplineCG.dml | 2 +- scripts/algorithms/CsplineDS.dml | 2 +- src/test/scripts/applications/cspline/CsplineCG.dml | 2 +- src/test/scripts/applications/cspline/CsplineCG.pydml | 2 +- src/test/scripts/applications/cspline/CsplineDS.dml | 2 +- src/test/scripts/applications/cspline/CsplineDS.pydml | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d8a8c3b6/scripts/algorithms/CsplineCG.dml -- diff --git a/scripts/algorithms/CsplineCG.dml b/scripts/algorithms/CsplineCG.dml index c5372d7..2281270 100644 --- a/scripts/algorithms/CsplineCG.dml +++ b/scripts/algorithms/CsplineCG.dml @@ -102,7 +102,7 @@ y_mat = matrix(y, 1, 1) if (fileO != " ") { write (y_mat, fileO); } else { - print(y_mat) + print(y) } print ("END CUBIC SPLINE REGRESSION SCRIPT"); http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d8a8c3b6/scripts/algorithms/CsplineDS.dml -- diff --git a/scripts/algorithms/CsplineDS.dml b/scripts/algorithms/CsplineDS.dml index 72508be..ff34b5e 100644 --- a/scripts/algorithms/CsplineDS.dml +++ b/scripts/algorithms/CsplineDS.dml @@ -82,7 +82,7 @@ y_mat = matrix(y, 1, 1) if (fileO != " ") { write (y_mat, fileO); } else { - print(y_mat) + print(y) } http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d8a8c3b6/src/test/scripts/applications/cspline/CsplineCG.dml -- diff --git a/src/test/scripts/applications/cspline/CsplineCG.dml b/src/test/scripts/applications/cspline/CsplineCG.dml index c5372d7..2281270 100644 --- a/src/test/scripts/applications/cspline/CsplineCG.dml +++ b/src/test/scripts/applications/cspline/CsplineCG.dml @@ -102,7 +102,7 @@ y_mat = matrix(y, 1, 1) if (fileO != " ") { write (y_mat, fileO); } else { - print(y_mat) + print(y) } print ("END CUBIC SPLINE REGRESSION SCRIPT"); http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d8a8c3b6/src/test/scripts/applications/cspline/CsplineCG.pydml -- diff --git a/src/test/scripts/applications/cspline/CsplineCG.pydml b/src/test/scripts/applications/cspline/CsplineCG.pydml index e7ef6bb..8e84f14 100644 --- a/src/test/scripts/applications/cspline/CsplineCG.pydml +++ b/src/test/scripts/applications/cspline/CsplineCG.pydml @@ -96,7 +96,7 @@ y_mat = full(y, 1, 1) if (fileO != " "): save (y_mat, fileO) else: -print(y_mat) +print(y) print ("END CUBIC SPLINE REGRESSION SCRIPT") http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d8a8c3b6/src/test/scripts/applications/cspline/CsplineDS.dml -- diff --git a/src/test/scripts/applications/cspline/CsplineDS.dml b/src/test/scripts/applications/cspline/CsplineDS.dml index 72508be..ff34b5e 100644 --- a/src/test/scripts/applications/cspline/CsplineDS.dml +++ b/src/test/scripts/applications/cspline/CsplineDS.dml @@ -82,7 +82,7 @@ y_mat = matrix(y, 1, 1) if (fileO != " ") { write (y_mat, fileO); } else { - print(y_mat) + print(y) } http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d8a8c3b6/src/test/scripts/applications/cspline/CsplineDS.pydml -- diff --git a/src/test/scripts/applications/cspline/CsplineDS.pydml b/src/test/scripts/applications/cspline/CsplineDS.pydml index 363df7d..dc16661 100644 --- a/src/test/scripts/applications/cspline/CsplineDS.pydml +++ b/src/test/scripts/applications/cspline/CsplineDS.pydml @@ -77,7 +77,7 @@ y_mat = full(y, 1, 1) if (fileO != " "): save (y_mat, fileO) else: -print(y_mat) +print(y) print ("END CUBIC SPLINE REGRESSION SCRIPT")
incubator-systemml git commit: [SYSTEMML-764] Add Univar-Stats.dml labeled console output
Repository: incubator-systemml Updated Branches: refs/heads/master b584aecf6 -> 1035699c3 [SYSTEMML-764] Add Univar-Stats.dml labeled console output Added console output. Removed the existing table listing number and name of univariate statistics to avoid redundancy. Closes #192. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/1035699c Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/1035699c Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/1035699c Branch: refs/heads/master Commit: 1035699c3b23bef916eab4738a9d2d64e98d9d6e Parents: b584aec Author: Sandeep Narayanaswami Authored: Tue Jul 19 15:36:28 2016 -0700 Committer: Glenn Weidner Committed: Tue Jul 19 15:36:28 2016 -0700 -- docs/quick-start-guide.md | 103 ++--- 1 file changed, 66 insertions(+), 37 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/1035699c/docs/quick-start-guide.md -- diff --git a/docs/quick-start-guide.md b/docs/quick-start-guide.md index ed6bd3a..f05db25 100644 --- a/docs/quick-start-guide.md +++ b/docs/quick-start-guide.md @@ -145,25 +145,78 @@ for each feature column using the algorithm `Univar-Stats.dml` which requires 3 * `X`: location of the input data file to analyze * `TYPES`: location of the file that contains the feature column types encoded by integer numbers: `1` = scale, `2` = nominal, `3` = ordinal -* `STATS`: location of the output matrix of computed statistics will be stored +* `STATS`: location where the output matrix of computed statistics is to be stored We need to create a file `types.csv` that describes the type of each column in -the data along with it's metadata file `types.csv.mtd`. +the data along with its metadata file `types.csv.mtd`. $ echo '1,1,1,2' > data/types.csv $ echo '{"rows": 1, "cols": 4, "format": "csv"}' > data/types.csv.mtd -To run the `Univar-Stats.dml` algorithm, issue the following command: - -$ ./runStandaloneSystemML.sh scripts/algorithms/Univar-Stats.dml -nvargs X=data/haberman.data TYPES=data/types.csv STATS=data/univarOut.mtx - -The resulting matrix has one row per each univariate statistic and one column -per input feature. The output file `univarOut.mtx` describes that -matrix. The elements of the first column denote the number of the statistic, -the elements of the second column refer to the number of the feature column in -the input data, and the elements of the third column show the value of the -univariate statistic. +To run the `Univar-Stats.dml` algorithm, issue the following command (we set the optional argument `CONSOLE_OUTPUT` to `TRUE` to print the statistics to the console): + +$ ./runStandaloneSystemML.sh scripts/algorithms/Univar-Stats.dml -nvargs X=data/haberman.data TYPES=data/types.csv STATS=data/univarOut.mtx CONSOLE_OUTPUT=TRUE + +[...] +- +Feature [1]: Scale + (01) Minimum | 30.0 + (02) Maximum | 83.0 + (03) Range | 53.0 + (04) Mean| 52.45751633986928 + (05) Variance| 116.71458266366658 + (06) Std deviation | 10.803452349303281 + (07) Std err of mean | 0.6175922641866753 + (08) Coeff of variation | 0.20594669940735139 + (09) Skewness| 0.1450718616532357 + (10) Kurtosis| -0.6150152487211726 + (11) Std err of skewness | 0.13934809593495995 + (12) Std err of kurtosis | 0.277810485320835 + (13) Median | 52.0 + (14) Interquartile mean | 52.16013071895425 +- +Feature [2]: Scale + (01) Minimum | 58.0 + (02) Maximum | 69.0 + (03) Range | 11.0 + (04) Mean| 62.85294117647059 + (05) Variance| 10.558630665380907 + (06) Std deviation | 3.2494046632238507 + (07) Std err of mean | 0.18575610076612029 + (08) Coeff of variation | 0.051698529971741194 + (09) Skewness| 0.07798443581479181 + (10) Kurtosis| -1.1324380182967442 + (11) Std err of skewness | 0.13934809593495995 + (12) Std err of kurtosis | 0.277810485320835 + (13) Median | 63.0 + (14) Interquartile mean | 62.80392156862745 +- +Feature [3]: Scale + (01) Minimum | 0.0 + (02) Maximum | 52.0 + (03) Range | 52.0 + (04) Mean| 4.026143790849673 + (05) Variance| 51.691117539912135
incubator-systemml git commit: [SYSTEMML-764] Add Univar-Stats.dml labeled console output
Repository: incubator-systemml Updated Branches: refs/heads/gh-pages a11a73efd -> 9f325cc7d [SYSTEMML-764] Add Univar-Stats.dml labeled console output Added console output. Removed the existing table listing number and name of univariate statistics to avoid redundancy. Closes #192. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/9f325cc7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/9f325cc7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/9f325cc7 Branch: refs/heads/gh-pages Commit: 9f325cc7dee826c005142e4829e186a4c8838347 Parents: a11a73e Author: Sandeep Narayanaswami Authored: Tue Jul 19 15:36:28 2016 -0700 Committer: Glenn Weidner Committed: Tue Jul 19 15:36:28 2016 -0700 -- quick-start-guide.md | 103 +- 1 file changed, 66 insertions(+), 37 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/9f325cc7/quick-start-guide.md -- diff --git a/quick-start-guide.md b/quick-start-guide.md index ed6bd3a..f05db25 100644 --- a/quick-start-guide.md +++ b/quick-start-guide.md @@ -145,25 +145,78 @@ for each feature column using the algorithm `Univar-Stats.dml` which requires 3 * `X`: location of the input data file to analyze * `TYPES`: location of the file that contains the feature column types encoded by integer numbers: `1` = scale, `2` = nominal, `3` = ordinal -* `STATS`: location of the output matrix of computed statistics will be stored +* `STATS`: location where the output matrix of computed statistics is to be stored We need to create a file `types.csv` that describes the type of each column in -the data along with it's metadata file `types.csv.mtd`. +the data along with its metadata file `types.csv.mtd`. $ echo '1,1,1,2' > data/types.csv $ echo '{"rows": 1, "cols": 4, "format": "csv"}' > data/types.csv.mtd -To run the `Univar-Stats.dml` algorithm, issue the following command: - -$ ./runStandaloneSystemML.sh scripts/algorithms/Univar-Stats.dml -nvargs X=data/haberman.data TYPES=data/types.csv STATS=data/univarOut.mtx - -The resulting matrix has one row per each univariate statistic and one column -per input feature. The output file `univarOut.mtx` describes that -matrix. The elements of the first column denote the number of the statistic, -the elements of the second column refer to the number of the feature column in -the input data, and the elements of the third column show the value of the -univariate statistic. +To run the `Univar-Stats.dml` algorithm, issue the following command (we set the optional argument `CONSOLE_OUTPUT` to `TRUE` to print the statistics to the console): + +$ ./runStandaloneSystemML.sh scripts/algorithms/Univar-Stats.dml -nvargs X=data/haberman.data TYPES=data/types.csv STATS=data/univarOut.mtx CONSOLE_OUTPUT=TRUE + +[...] +- +Feature [1]: Scale + (01) Minimum | 30.0 + (02) Maximum | 83.0 + (03) Range | 53.0 + (04) Mean| 52.45751633986928 + (05) Variance| 116.71458266366658 + (06) Std deviation | 10.803452349303281 + (07) Std err of mean | 0.6175922641866753 + (08) Coeff of variation | 0.20594669940735139 + (09) Skewness| 0.1450718616532357 + (10) Kurtosis| -0.6150152487211726 + (11) Std err of skewness | 0.13934809593495995 + (12) Std err of kurtosis | 0.277810485320835 + (13) Median | 52.0 + (14) Interquartile mean | 52.16013071895425 +- +Feature [2]: Scale + (01) Minimum | 58.0 + (02) Maximum | 69.0 + (03) Range | 11.0 + (04) Mean| 62.85294117647059 + (05) Variance| 10.558630665380907 + (06) Std deviation | 3.2494046632238507 + (07) Std err of mean | 0.18575610076612029 + (08) Coeff of variation | 0.051698529971741194 + (09) Skewness| 0.07798443581479181 + (10) Kurtosis| -1.1324380182967442 + (11) Std err of skewness | 0.13934809593495995 + (12) Std err of kurtosis | 0.277810485320835 + (13) Median | 63.0 + (14) Interquartile mean | 62.80392156862745 +- +Feature [3]: Scale + (01) Minimum | 0.0 + (02) Maximum | 52.0 + (03) Range | 52.0 + (04) Mean| 4.026143790849673 + (05) Variance| 51.691117539912135 + (06) Std deviat
incubator-systemml git commit: [SYSTEMML-847] Remove LogisticRegression and LogisticRegressionModel in api/javaml
Repository: incubator-systemml Updated Branches: refs/heads/master 77363c0c6 -> d7b9cc467 [SYSTEMML-847] Remove LogisticRegression and LogisticRegressionModel in api/javaml Removed Java classes in api/javaml since corresponding Scala versions exist in api/ml. Closes #205. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/d7b9cc46 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/d7b9cc46 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/d7b9cc46 Branch: refs/heads/master Commit: d7b9cc4676e04992235a63669ee06f31fa1d14b5 Parents: 77363c0 Author: Glenn Weidner Authored: Fri Aug 5 16:25:33 2016 -0700 Committer: Glenn Weidner Committed: Fri Aug 5 16:25:33 2016 -0700 -- .../sysml/api/javaml/LogisticRegression.java| 473 --- .../api/javaml/LogisticRegressionModel.java | 179 --- 2 files changed, 652 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d7b9cc46/src/main/java/org/apache/sysml/api/javaml/LogisticRegression.java -- diff --git a/src/main/java/org/apache/sysml/api/javaml/LogisticRegression.java b/src/main/java/org/apache/sysml/api/javaml/LogisticRegression.java deleted file mode 100644 index dbcc118..000 --- a/src/main/java/org/apache/sysml/api/javaml/LogisticRegression.java +++ /dev/null @@ -1,473 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ - -package org.apache.sysml.api.javaml; - -import java.io.File; -import java.util.HashMap; - -import org.apache.spark.SparkContext; -import org.apache.spark.api.java.JavaPairRDD; -import org.apache.spark.api.java.JavaRDD; -import org.apache.spark.api.java.JavaSparkContext; -import org.apache.spark.ml.classification.LogisticRegressionParams; -import org.apache.spark.ml.classification.ProbabilisticClassifier; -import org.apache.spark.ml.param.BooleanParam; -import org.apache.spark.ml.param.DoubleParam; -import org.apache.spark.ml.param.IntParam; -import org.apache.spark.ml.param.StringArrayParam; -import org.apache.spark.mllib.linalg.Vector; -import org.apache.spark.sql.DataFrame; -import org.apache.spark.sql.SQLContext; -import org.apache.sysml.api.MLContext; -import org.apache.sysml.api.MLOutput; -import org.apache.sysml.api.javaml.LogisticRegressionModel; -import org.apache.sysml.api.ml.functions.ConvertSingleColumnToString; -import org.apache.sysml.runtime.DMLRuntimeException; -import org.apache.sysml.runtime.instructions.spark.utils.RDDConverterUtilsExt; -import org.apache.sysml.runtime.matrix.MatrixCharacteristics; -import org.apache.sysml.runtime.matrix.data.MatrixBlock; -import org.apache.sysml.runtime.matrix.data.MatrixIndexes; - -/** - * - * This class shows how SystemML can be integrated into MLPipeline. Note, it has not been optimized for performance and - * is implemented as a proof of concept. An optimized pipeline can be constructed by usage of DML's 'parfor' construct. - * - * TODO: - * - Please note that this class expects 1-based labels. To run below example, - * please set environment variable 'SYSTEMML_HOME' and create folder 'algorithms' - * and place atleast two scripts in that folder 'MultiLogReg.dml' and 'GLM-predict.dml' - * - It is not yet optimized for performance. - * - Also, it needs to be extended to surface all the parameters of MultiLogReg.dml - * - * Example usage: - * - * // Code to demonstrate usage of pipeline - * import org.apache.spark.ml.Pipeline - * import org.apache.sysml.api.ml.LogisticRegression - * import org.apache.spark.ml.feature.{HashingTF, Tokenizer} - * import org.apache.spark.mllib.linalg.Vector - * case class LabeledDocument(id: Long, text: String, label: Double) - * case class Document(id: Long, text: String) - * val training = sc.parallelize(Seq( - * LabeledDocument(0L, "a b c d e spark", 1.0), - * LabeledDocument(1L, "b d", 2.0), - * LabeledDocument(2L, "spark f g h", 1.0), - * LabeledDocument(3L, "
incubator-systemml git commit: [SYSTEMML-868] Switch default to scala 2.10
Repository: incubator-systemml Updated Branches: refs/heads/master 97dee8fba -> 90f5cbba4 [SYSTEMML-868] Switch default to scala 2.10 Changed default scala properties to 2.10 and added scala-2.11 profile. Closes #219 Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/90f5cbba Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/90f5cbba Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/90f5cbba Branch: refs/heads/master Commit: 90f5cbba406af6f95ac011f88d62f0e2955d4384 Parents: 97dee8f Author: Glenn Weidner Authored: Thu Aug 25 15:13:42 2016 -0700 Committer: Glenn Weidner Committed: Thu Aug 25 15:13:42 2016 -0700 -- pom.xml | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/90f5cbba/pom.xml -- diff --git a/pom.xml b/pom.xml index a4c66a1..b6e51e2 100644 --- a/pom.xml +++ b/pom.xml @@ -66,8 +66,8 @@ 2.4.1 4.3 1.4.1 - 2.11.8 - 2.11 + 2.10.5 + 2.10 2.2.6 @@ -490,6 +490,14 @@ 2.10 + + + scala-2.11 + + 2.11.8 + 2.11 + +
incubator-systemml git commit: [SYSTEMML-1400] Include pandas in required python packages
Repository: incubator-systemml Updated Branches: refs/heads/master d2172dcf3 -> d8390d4d2 [SYSTEMML-1400] Include pandas in required python packages Closes #430. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/d8390d4d Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/d8390d4d Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/d8390d4d Branch: refs/heads/master Commit: d8390d4d228c1c53ee25a01aab3979f6eeb7b598 Parents: d2172dc Author: Glenn Weidner Authored: Tue Mar 14 12:27:38 2017 -0700 Committer: Glenn Weidner Committed: Tue Mar 14 12:27:38 2017 -0700 -- src/main/python/setup.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/d8390d4d/src/main/python/setup.py -- diff --git a/src/main/python/setup.py b/src/main/python/setup.py index b0fb1a8..635dad7 100644 --- a/src/main/python/setup.py +++ b/src/main/python/setup.py @@ -40,7 +40,8 @@ numpy_version = '1.8.2' scipy_version = '0.15.1' REQUIRED_PACKAGES = [ 'numpy >= %s' % numpy_version, -'scipy >= %s' % scipy_version +'scipy >= %s' % scipy_version, +'pandas' ]
incubator-systemml-website git commit: [MINOR] Add security page under Apache
Repository: incubator-systemml-website Updated Branches: refs/heads/master 0c170c572 -> b2f52ca65 [MINOR] Add security page under Apache Closes #42. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/b2f52ca6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/b2f52ca6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/b2f52ca6 Branch: refs/heads/master Commit: b2f52ca653077a67815e5e79b0b9342ece043412 Parents: 0c170c5 Author: Glenn Weidner Authored: Fri Mar 17 21:03:48 2017 -0700 Committer: Glenn Weidner Committed: Fri Mar 17 21:03:48 2017 -0700 -- _src/_data/navigation.yml | 2 + .../themes/apache/partials/_navigation.html | 1 + _src/security.html | 43 3 files changed, 46 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/b2f52ca6/_src/_data/navigation.yml -- diff --git a/_src/_data/navigation.yml b/_src/_data/navigation.yml index 87b7b2b..b9ac7dd 100644 --- a/_src/_data/navigation.yml +++ b/_src/_data/navigation.yml @@ -51,3 +51,5 @@ topnav: url: http://www.apache.org/foundation/thanks.html - title: Privacy Policy url: /privacy-policy + - title: Security +url: /security \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/b2f52ca6/_src/_includes/themes/apache/partials/_navigation.html -- diff --git a/_src/_includes/themes/apache/partials/_navigation.html b/_src/_includes/themes/apache/partials/_navigation.html index 08c2317..2c12ff3 100644 --- a/_src/_includes/themes/apache/partials/_navigation.html +++ b/_src/_includes/themes/apache/partials/_navigation.html @@ -31,6 +31,7 @@ limitations under the License. http://www.apache.org/foundation/sponsorship"; target="_blank">Sponsorship http://www.apache.org/foundation/thanks.html"; target="_blank">Thanks https://systemml.apache.org/privacy-policy"; target="_blank">Privacy Policy +https://systemml.apache.org/security"; target="_blank">Security http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/b2f52ca6/_src/security.html -- diff --git a/_src/security.html b/_src/security.html new file mode 100644 index 000..2b7c446 --- /dev/null +++ b/_src/security.html @@ -0,0 +1,43 @@ +--- +layout: page +title: Apache SystemML Security +description: Apache SystemML security +group: nav-right +--- + + + + + {{ site.data.project.name }} Security + + The Apache Software Foundation takes a very active stance in eliminating security problems with its projects. + If you have any security concerns regarding Apache SystemML™, or believe you have uncovered a vulnerability or potential threat, contact the Apache Security Team at mailto:secur...@apache.org?Subject=[SECURITY][SYSTEMML] My security issue">secur...@apache.org. + Include a description of the issue and try to provide steps to reproduce the problem. + The Apache SystemML™ Community and the Apache Security Team take security issues very seriously and will respond. + + + Note the general Apache Security mailing address mailto:secur...@apache.org";>secur...@apache.org is private and should be used only for undisclosed vulnerabilities. + Please report any security problems to Apache Security Team before disclosing it publicly. + + + The ASF Security team maintains a page with a description of how vulnerabilities and potential threats are handled. Refer to their http://www.apache.org/security/";>security page for more information. + + +
svn commit: r1787577 - /incubator/systemml/site/security.html
Author: gweidner Date: Sat Mar 18 16:54:47 2017 New Revision: 1787577 URL: http://svn.apache.org/viewvc?rev=1787577&view=rev Log: [MINOR] Add security.html page only Added: incubator/systemml/site/security.html Added: incubator/systemml/site/security.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/security.html?rev=1787577&view=auto == --- incubator/systemml/site/security.html (added) +++ incubator/systemml/site/security.html Sat Mar 18 16:54:47 2017 @@ -0,0 +1,205 @@ + + + + + + + + + + Apache SystemML Security + + + + + + + + + + + https://systemml.apache.org/"; /> + + + + + + + + + + + + + + + + + + + Apache SystemML™ (incubating) + + + + + + +Download + + + + + + +Get Started + + + + + + +Documentation + + + + + +Community + + + + Get Involved + + + https://issues.apache.org/jira/browse/SYSTEMML"; target="_blank">Issue Tracker + + + https://github.com/apache/incubator-systemml"; target="_blank">Source Code + + + https://github.com/apache/incubator-systemml-website"; target="_blank">Website Source Code + + + Roadmap + + + + + + + +Apache + + + + http://www.apache.org/foundation/how-it-works.html"; target="_blank">Apache Software Foundation + + + http://www.apache.org/licenses/"; target="_blank">Apache License + + + http://www.apache.org/foundation/sponsorship"; target="_blank">Sponsorship + + + http://www.apache.org/foundation/thanks.html"; target="_blank">Thanks + + + Privacy Policy + + + Security + + + + + + + + + + + + + + + + + Apache SystemML™ Security + + The Apache Software Foundation takes a very active stance in eliminating security problems with its projects. + If you have any security concerns regarding Apache SystemML™, or believe you have uncovered a vulnerability or potential threat, contact the Apache Security Team at mailto:secur...@apache.org?Subject=[SECURITY][SYSTEMML] My security issue">secur...@apache.org. + Include a description of the issue and try to provide steps to reproduce the problem. + The Apache SystemML™ Community and the Apache Security Team take security issues very seriously and will respond. + + + Note the general Apache Security mailing address mailto:secur...@apache.org";>secur...@apache.org is private and should be used only for undisclosed vulnerabilities. + Please report any security problems to Apache Security Team before disclosing it publicly. + + + The ASF Security team maintains a page with a description of how vulnerabilities and potential threats are handled. Refer to their http://www.apache.org/security/";>security page for more information. + + + + + + + + + + + + + Apache SystemML™ is an effort undergoing https://incubator.apache.org/index.html";>Incubation at The Apache Software Foundation (ASF), sponsored by the Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. + + + + +http://incubator.apache.org";> + + + +Copyright © 2015-2017 http://www.apache.org";>The Apache Software Foundation. +Licensed under the http://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0. + + +Apache, the Apache Feather logo, and the Apache Incubator project logo are trademarks of The Apache Software Foundation. + + + + + + + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ +
svn commit: r1787582 - in /incubator/systemml/site: community.html documentation.html download.html get-started.html index.html privacy-policy.html roadmap.html tutorials.html
Author: gweidner Date: Sat Mar 18 17:52:01 2017 New Revision: 1787582 URL: http://svn.apache.org/viewvc?rev=1787582&view=rev Log: [MINOR] Update other html pages apache menu section with security link Modified: incubator/systemml/site/community.html incubator/systemml/site/documentation.html incubator/systemml/site/download.html incubator/systemml/site/get-started.html incubator/systemml/site/index.html incubator/systemml/site/privacy-policy.html incubator/systemml/site/roadmap.html incubator/systemml/site/tutorials.html Modified: incubator/systemml/site/community.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/community.html?rev=1787582&r1=1787581&r2=1787582&view=diff == --- incubator/systemml/site/community.html (original) +++ incubator/systemml/site/community.html Sat Mar 18 17:52:01 2017 @@ -117,6 +117,9 @@ Privacy Policy + + Security + Modified: incubator/systemml/site/documentation.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/documentation.html?rev=1787582&r1=1787581&r2=1787582&view=diff == --- incubator/systemml/site/documentation.html (original) +++ incubator/systemml/site/documentation.html Sat Mar 18 17:52:01 2017 @@ -117,6 +117,9 @@ Privacy Policy + + Security + Modified: incubator/systemml/site/download.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/download.html?rev=1787582&r1=1787581&r2=1787582&view=diff == --- incubator/systemml/site/download.html (original) +++ incubator/systemml/site/download.html Sat Mar 18 17:52:01 2017 @@ -117,6 +117,9 @@ Privacy Policy + + Security + Modified: incubator/systemml/site/get-started.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/get-started.html?rev=1787582&r1=1787581&r2=1787582&view=diff == --- incubator/systemml/site/get-started.html (original) +++ incubator/systemml/site/get-started.html Sat Mar 18 17:52:01 2017 @@ -117,6 +117,9 @@ Privacy Policy + + Security + Modified: incubator/systemml/site/index.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/index.html?rev=1787582&r1=1787581&r2=1787582&view=diff == --- incubator/systemml/site/index.html (original) +++ incubator/systemml/site/index.html Sat Mar 18 17:52:01 2017 @@ -117,6 +117,9 @@ Privacy Policy + + Security + Modified: incubator/systemml/site/privacy-policy.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/privacy-policy.html?rev=1787582&r1=1787581&r2=1787582&view=diff == --- incubator/systemml/site/privacy-policy.html (original) +++ incubator/systemml/site/privacy-policy.html Sat Mar 18 17:52:01 2017 @@ -117,6 +117,9 @@ Privacy Policy + + Security + Modified: incubator/systemml/site/roadmap.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/roadmap.html?rev=1787582&r1=1787581&r2=1787582&view=diff == --- incubator/systemml/site/roadmap.html (original) +++ incubator/systemml/site/roadmap.html Sat Mar 18 17:52:01 2017 @@ -117,6 +117,9 @@ Privacy Policy + + Security + Modified: incubator/systemml/site/tutorials.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/tutorials.html?rev=1787582&r1=1787581&r2=1787582&view=diff == --- incubator/systemml/site/tutorials.html (original) +++ incubator/systemml/site/tutorials.html Sat Mar 18 17:52:01 2017 @@ -117,6 +117,9 @@ Privacy Policy + + Security +
incubator-systemml-website git commit: [MINOR] Update 0.13 roadmap and release notes for CLA v2
Repository: incubator-systemml-website Updated Branches: refs/heads/master b2f52ca65 -> cb489926b [MINOR] Update 0.13 roadmap and release notes for CLA v2 Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/cb489926 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/cb489926 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/cb489926 Branch: refs/heads/master Commit: cb489926ba848eebe252b0ee21751f7e830c51bb Parents: b2f52ca Author: Glenn Weidner Authored: Sun Mar 26 12:32:27 2017 -0700 Committer: Glenn Weidner Committed: Sun Mar 26 12:32:27 2017 -0700 -- 0.13.0-incubating/release_notes.md | 3 +-- _src/maturity.html | Bin 0 -> 392918 bytes _src/roadmap.html | 3 +-- 3 files changed, 2 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/cb489926/0.13.0-incubating/release_notes.md -- diff --git a/0.13.0-incubating/release_notes.md b/0.13.0-incubating/release_notes.md index 3e7848b..65560e7 100644 --- a/0.13.0-incubating/release_notes.md +++ b/0.13.0-incubating/release_notes.md @@ -37,8 +37,7 @@ became an incubator project on November 2nd, 2015. Updates have been made to the - Versioned documentation ### Experimental Features / Algorithms -- Compressed Linear Algebra - Hardening Sample-Based Estimators (special cases) -- Compressed Linear Algebra - Debugging Tools (compression plans, stats) +- Compressed Linear Algebra v2 (new DDC encoding format, hardened sample-based estimators, debugging tools, new column grouping algorithm, additional operations) ### Various Fixes * Bug fixes for diverse issues. See [JIRA release notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319522&version=12339247) for complete list. http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/cb489926/_src/maturity.html -- diff --git a/_src/maturity.html b/_src/maturity.html new file mode 100644 index 000..4b776c2 Binary files /dev/null and b/_src/maturity.html differ http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/cb489926/_src/roadmap.html -- diff --git a/_src/roadmap.html b/_src/roadmap.html index ff4e377..0a133ea 100644 --- a/_src/roadmap.html +++ b/_src/roadmap.html @@ -96,8 +96,7 @@ limitations under the License. Experimental Features / Algorithms -Compressed Linear Algebra - Hardening Sample-Based Estimators (special cases) - Compressed Linear Algebra - Debugging Tools (compression plans, stats) +Compressed Linear Algebra v2 (new DDC encoding format, hardened sample-based estimators, debugging tools, new column grouping algorithm, additional operations)
svn commit: r1788775 - in /incubator/systemml/site: maturity.html roadmap.html
Author: gweidner Date: Sun Mar 26 20:08:55 2017 New Revision: 1788775 URL: http://svn.apache.org/viewvc?rev=1788775&view=rev Log: [MINOR] Update 0.13 roadmap and release notes for CLA v2 Added: incubator/systemml/site/maturity.html (with props) Modified: incubator/systemml/site/roadmap.html Added: incubator/systemml/site/maturity.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/maturity.html?rev=1788775&view=auto == Binary file - no diff available. Propchange: incubator/systemml/site/maturity.html -- svn:mime-type = application/octet-stream Modified: incubator/systemml/site/roadmap.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/roadmap.html?rev=1788775&r1=1788774&r2=1788775&view=diff == --- incubator/systemml/site/roadmap.html (original) +++ incubator/systemml/site/roadmap.html Sun Mar 26 20:08:55 2017 @@ -212,8 +212,7 @@ Experimental Features / Algorithms -Compressed Linear Algebra - Hardening Sample-Based Estimators (special cases) - Compressed Linear Algebra - Debugging Tools (compression plans, stats) +Compressed Linear Algebra v2 (new DDC encoding format, hardened sample-based estimators, debugging tools, new column grouping algorithm, additional operations)
incubator-systemml git commit: [SYSTEMML-1529] Improve accuracy score in test_svm of test_mllearn_numpy.py
Repository: incubator-systemml Updated Branches: refs/heads/master 0d625a05e -> 499428072 [SYSTEMML-1529] Improve accuracy score in test_svm of test_mllearn_numpy.py Adjusted tolerance parameter in test_svm to help prevent occasional failure due to accuracy not meeting test criteria of greater than 95%. Closes #460. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/49942807 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/49942807 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/49942807 Branch: refs/heads/master Commit: 499428072b53fbc63547d190a85e551c11c4d474 Parents: 0d625a0 Author: Glenn Weidner Authored: Sun Apr 16 10:53:49 2017 -0700 Committer: Glenn Weidner Committed: Sun Apr 16 10:53:49 2017 -0700 -- src/main/python/tests/test_mllearn_numpy.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/49942807/src/main/python/tests/test_mllearn_numpy.py -- diff --git a/src/main/python/tests/test_mllearn_numpy.py b/src/main/python/tests/test_mllearn_numpy.py index faa4d32..2a1bc3c 100644 --- a/src/main/python/tests/test_mllearn_numpy.py +++ b/src/main/python/tests/test_mllearn_numpy.py @@ -149,12 +149,14 @@ class TestMLLearn(unittest.TestCase): y_train = y_digits[:int(.9 * n_samples)] X_test = X_digits[int(.9 * n_samples):] y_test = y_digits[int(.9 * n_samples):] -svm = SVM(sparkSession, is_multi_class=True) +svm = SVM(sparkSession, is_multi_class=True, tol=0.0001) mllearn_predicted = svm.fit(X_train, y_train).predict(X_test) from sklearn import linear_model, svm clf = svm.LinearSVC() sklearn_predicted = clf.fit(X_train, y_train).predict(X_test) -self.failUnless(accuracy_score(sklearn_predicted, mllearn_predicted) > 0.95 ) +accuracy = accuracy_score(sklearn_predicted, mllearn_predicted) +evaluation = 'test_svm accuracy_score(sklearn_predicted, mllearn_predicted) was {}'.format(accuracy) +self.failUnless(accuracy > 0.95, evaluation) def test_naive_bayes(self): digits = datasets.load_digits()
incubator-systemml git commit: [SYSTEMML-1571] New Jupyter Python Notebook to showcase SystemML 2-layer autoencoder for acoustic signal modeling
Repository: incubator-systemml Updated Branches: refs/heads/master e8fbc7539 -> 5aa6fb75b [SYSTEMML-1571] New Jupyter Python Notebook to showcase SystemML 2-layer autoencoder for acoustic signal modeling Closes #478. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/5aa6fb75 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/5aa6fb75 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/5aa6fb75 Branch: refs/heads/master Commit: 5aa6fb75b066228d723dfc88bc0ae24551812e92 Parents: e8fbc75 Author: fmakari Authored: Mon May 1 11:50:23 2017 -0700 Committer: Glenn Weidner Committed: Mon May 1 11:50:24 2017 -0700 -- samples/jupyter-notebooks/Autoencoder.ipynb | 188 +++ 1 file changed, 188 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/5aa6fb75/samples/jupyter-notebooks/Autoencoder.ipynb -- diff --git a/samples/jupyter-notebooks/Autoencoder.ipynb b/samples/jupyter-notebooks/Autoencoder.ipynb new file mode 100644 index 000..ec9c2aa --- /dev/null +++ b/samples/jupyter-notebooks/Autoencoder.ipynb @@ -0,0 +1,188 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ +"# Autoencoder\n", +"This notebook demonstrates the invocation of the SystemML autoencoder script, and alternative ways of passing in/out data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { +"collapsed": false + }, + "outputs": [], + "source": [ +"#!pip install --user systemml>0.13.0\n", +"!pip show systemml" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { +"collapsed": false + }, + "outputs": [], + "source": [ +"import pandas as pd\n", +"from systemml import MLContext, dml\n", +"ml = MLContext(sc)\n", +"print(ml.info())\n", +"sc.version" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ +"## SystemML Read/Write data from local file system" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { +"collapsed": true + }, + "outputs": [], + "source": [ +"FsPath = \"/tmp/data/\"\n", +"inp = FsPath + \"Input/\"\n", +"outp = FsPath + \"Output/\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ +"Generate Data and write out to file." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { +"collapsed": false + }, + "outputs": [], + "source": [ +"X_pd = pd.DataFrame(range(1, 2001,1),dtype=float).values.reshape(100,20)\n", +"script =\"\"\"\n", +"write(X, $Xfile)\n", +"\"\"\"\n", +"prog = dml(script).input(X=X_pd).input(**{\"$Xfile\":inp+\"X.csv\"})\n", +"ml.execute(prog)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { +"collapsed": false + }, + "outputs": [], + "source": [ +"!ls -l /tmp/data/Input" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { +"collapsed": false + }, + "outputs": [], + "source": [ +"autoencoderURL = \"https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/staging/autoencoder-2layer.dml\"\n";, +"rets = (\"iter\", \"num_iters_per_epoch\", \"beg\", \"end\", \"o\")\n", +"\n", +"prog = dml(autoencoderURL).input(**{\"$X\":inp+\"X.csv\"}) \\\n", +" .input(**{\"$H1\":500, \"$H2\":2, \"$BATCH\":36, \"$EPOCH\":5 \\\n", +", \"$W1_out\":outp+\"W1_out\", \"$b1_out\":outp+\"b1_out\" \\\n", +", \"$W2_out\":outp+\"W2_out\", \"$b2_out\":outp+\"b2_out\" \\\n", +", \"$W3_out\":outp+\"W3_out\", \"$b3_out\":outp+\"b3_out\" \\\n", +", \"$W4_out\":outp+\"W4_out\", \"$b4_out\":outp+\"b4_out\" \\\n", +" }).output(*rets)\n", +"iter, num_iters_per_epoch, beg, end, o = ml.execute(prog).get(*rets)\n", +"print (iter, num_iters_per_epoch, beg, end, o)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { +"collapsed": false + }, + "outputs": [], + "source": [ +"!ls -l /tmp/data/Output" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ +"## Alternatively to passing in/out file names, use Python variables." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { +"collapsed": false, +"scrolled": true + },
incubator-systemml-website git commit: [MINOR] Update get-started pip install
Repository: incubator-systemml-website Updated Branches: refs/heads/master 3b799fd0b -> 79ffbac91 [MINOR] Update get-started pip install Added instructions for Spark 1.6 users to include version specifier for pip install. Also incorporated site.data.project.release_version token so download link and artifact name match current release. Closes #45. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/79ffbac9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/79ffbac9 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/79ffbac9 Branch: refs/heads/master Commit: 79ffbac91dd259fb09c518b29c01bdba8708eba2 Parents: 3b799fd Author: Glenn Weidner Authored: Thu May 4 17:17:10 2017 -0700 Committer: Glenn Weidner Committed: Thu May 4 17:17:10 2017 -0700 -- _src/get-started.html | 20 ++-- 1 file changed, 14 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/79ffbac9/_src/get-started.html -- diff --git a/_src/get-started.html b/_src/get-started.html index d3208c5..5a75f66 100644 --- a/_src/get-started.html +++ b/_src/get-started.html @@ -95,26 +95,26 @@ pip install jupyter matplotlib numpy{% endhighlight %} Downloads - Download Spark and SystemML. + Download Apache Spark and Apache SystemML. - 3Download and Install Spark 1.6 + 3Download and Install Apache Spark {% highlight bash %} brew tap homebrew/versions -brew install apache-spark16{% endhighlight %} +brew install apache-spark{% endhighlight %} - Alternatively, you can http://spark.apache.org/downloads.html";>download Spark directly. + Alternatively, you can http://spark.apache.org/downloads.html";>download Apache Spark directly. - 4Download and Install SystemML + 4Download and Install Apache SystemML @@ -136,12 +136,20 @@ pip3 install systemml - Alternatively, if you intend to use SystemML via spark-shell (or spark-submit), you only need systemml-0.12.0-incubating.jar, which is packaged into our official binary release (http://www.apache.org/dyn/closer.lua/incubator/systemml/0.12.0-incubating/systemml-0.12.0-incubating-bin.zip"; target="_blank">systemml-0.12.0-incubating.zip). + Alternatively, if you intend to use SystemML via spark-shell (or spark-submit), you only need systemml-{{ site.data.project.release_version }}.jar, which is packaged into our official binary release (http://www.apache.org/dyn/closer.lua/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.zip" target="_blank">systemml-{{ site.data.project.release_version }}-bin.zip). Note: If you have installed SystemML via pip, you can get the location of this jar by executing following command: {% highlight bash %} python -c 'import imp; import os; print os.path.join(imp.find_module("systemml")[1], "systemml-java")'{% endhighlight %} + + Note - For Spark 1.6 users only, include a version specifier to download and install compatible Apache SystemML via pip: + + {% highlight bash %} +# For Spark 1.6 users with Python 2: +pip install "systemml<0.13.0" +{% endhighlight %} +
svn commit: r1794079 - /incubator/systemml/site/get-started.html
Author: gweidner Date: Fri May 5 17:28:11 2017 New Revision: 1794079 URL: http://svn.apache.org/viewvc?rev=1794079&view=rev Log: Update get-started pip install Modified: incubator/systemml/site/get-started.html Modified: incubator/systemml/site/get-started.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/get-started.html?rev=1794079&r1=1794078&r2=1794079&view=diff == --- incubator/systemml/site/get-started.html (original) +++ incubator/systemml/site/get-started.html Fri May 5 17:28:11 2017 @@ -209,25 +209,25 @@ pip install jupyter matplotlib numpy Downloads - Download Spark and SystemML. + Download Apache Spark and Apache SystemML. - 3Download and Install Spark 1.6 + 3Download and Install Apache Spark brew tap homebrew/versions -brew install apache-spark16 +brew install apache-spark - Alternatively, you can http://spark.apache.org/downloads.html";>download Spark directly. + Alternatively, you can http://spark.apache.org/downloads.html";>download Apache Spark directly. - 4Download and Install SystemML + 4Download and Install Apache SystemML @@ -247,11 +247,17 @@ pip3 install systemml - Alternatively, if you intend to use SystemML via spark-shell (or spark-submit), you only need systemml-0.12.0-incubating.jar, which is packaged into our official binary release (http://www.apache.org/dyn/closer.lua/incubator/systemml/0.12.0-incubating/systemml-0.12.0-incubating-bin.zip"; target="_blank">systemml-0.12.0-incubating.zip). + Alternatively, if you intend to use SystemML via spark-shell (or spark-submit), you only need systemml-0.13.0-incubating.jar, which is packaged into our official binary release (http://www.apache.org/dyn/closer.lua/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-bin.zip"; target="_blank">systemml-0.13.0-incubating-bin.zip). Note: If you have installed SystemML via pip, you can get the location of this jar by executing following command: python -c 'import imp; import os; print os.path.join(imp.find_module("systemml")[1], "systemml-java")' + + Note - For Spark 1.6 users only, include a version specifier to download and install compatible Apache SystemML via pip: + + # For Spark 1.6 users with Python 2: +pip install "systemml<0.13.0" +
incubator-systemml-website git commit: Update project, roadmap, release notes for 0.14
Repository: incubator-systemml-website Updated Branches: refs/heads/master 79ffbac91 -> 5cc53b958 Update project, roadmap, release notes for 0.14 Closes #46. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/5cc53b95 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/5cc53b95 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/5cc53b95 Branch: refs/heads/master Commit: 5cc53b95825073838f9cd46a4322a90a8e9a3c27 Parents: 79ffbac Author: Glenn Weidner Authored: Mon May 8 14:56:01 2017 -0700 Committer: Glenn Weidner Committed: Mon May 8 14:56:01 2017 -0700 -- 0.14.0-incubating/release_notes.md | 44 _src/_data/project.yml | 2 +- _src/roadmap.html | 45 +++-- 3 files changed, 82 insertions(+), 9 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/5cc53b95/0.14.0-incubating/release_notes.md -- diff --git a/0.14.0-incubating/release_notes.md b/0.14.0-incubating/release_notes.md new file mode 100644 index 000..68e810a --- /dev/null +++ b/0.14.0-incubating/release_notes.md @@ -0,0 +1,44 @@ +--- +layout: page +title: SystemML 0.14.0-incubating Release Notes +description: Project Release Notes +group: nav-right +--- + + + +The **Apache SystemML 0.14.0-incubating release** was approved on May 8, 2017. It is the sixth release of Apache SystemML since it +became an incubator project on November 2nd, 2015. Updates have been made to the project in several areas, as detailed below. + + +### New Features and Capabilities +- Runtime feature extensions (new libsvm-binary data converters, parfor spark buffer pool handling, parfor block partitioning of fixed size batches of rows or columns, native dataset support in parfor spark datapartition-execute) +- Compiler feature extensions (improved parfor execution type selection, improved literal replacement for nrow/ncol, simplified instruction generation across back-ends, consolidated static/dynamic rewrite utilities) + +### Experimental Features +- New Code Generation capabilities for automatic operator fusion (basic code generator, compiler integration, runtime integration, in-memory source code compilation, extended explain tool, support for right indexing and replace in cellwise and row aggregate templates, support for row, column, or no aggregation in rowwise template). Note code generation provides significant performance gains with fewer read/write intermediates, reduced scans of inputs and intermediates, and enhanced sparsity exploitation. To enable this feature, set `codegen.enabled` property to `true` in SystemML-config.xml file. +- New instructions and operators for GPU support (relu\_maxpooling, conv2d\_bias\_add, bias\_multiply) + +### Removals +- Removed support for Java 6 and Java 7 +- Removed parfor perftesttool and cost estimator + +### Various Fixes +* Bug fixes for diverse issues. See [JIRA release notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319522&version=12340322) for complete list. http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/5cc53b95/_src/_data/project.yml -- diff --git a/_src/_data/project.yml b/_src/_data/project.yml index 8012611..536c241 100644 --- a/_src/_data/project.yml +++ b/_src/_data/project.yml @@ -50,7 +50,7 @@ issues_list_archive_markmail: jira: SYSTEMML release_name: systemml -release_version: 0.13.0-incubating +release_version: 0.14.0-incubating source_repository: https://git-wip-us.apache.org/repos/asf/incubator-systemml.git source_repository_mirror: https://github.com/apache/incubator-systemml http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/5cc53b95/_src/roadmap.html -- diff --git a/_src/roadmap.html b/_src/roadmap.html index 0a133ea..a47b55f 100644 --- a/_src/roadmap.html +++ b/_src/roadmap.html @@ -39,8 +39,10 @@ limitations under the License. Planned for Future SystemML 1.0 Rigorous Performance and Scalability Testing (Bug Fixes) -Remove Deprecated APIs -Remove Deprecated Functions +Remove Deprecated APIs/Functions +Compression (additional operations) +Decomposition Algorithms +Experimental Features Improvements (Native BLAS, Single GPU, Deep Learning, Code Generation, Python DSL) @@ -48,14 +50,14 @@ limitations under the License. Planned for Future Releases Completion of Prior Experi
svn commit: r1794479 - in /incubator/systemml/site/docs/0.14.0: ./ api/ api/java/ api/java/org/ api/java/org/apache/ api/java/org/apache/sysml/ api/java/org/apache/sysml/api/ api/java/org/apache/sysml
Author: gweidner Date: Mon May 8 23:19:20 2017 New Revision: 1794479 URL: http://svn.apache.org/viewvc?rev=1794479&view=rev Log: Add 0.14.0 docs to website [This commit notification would consist of 245 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]
incubator-systemml-website git commit: Add 0.14.0 link to documentation page
Repository: incubator-systemml-website Updated Branches: refs/heads/master 5cc53b958 -> 2cafc92d9 Add 0.14.0 link to documentation page Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/2cafc92d Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/2cafc92d Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/2cafc92d Branch: refs/heads/master Commit: 2cafc92d9b0995090ee026a52819315dd585aaf3 Parents: 5cc53b9 Author: Glenn Weidner Authored: Mon May 8 16:54:37 2017 -0700 Committer: Glenn Weidner Committed: Mon May 8 16:54:38 2017 -0700 -- _src/documentation.html | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/2cafc92d/_src/documentation.html -- diff --git a/_src/documentation.html b/_src/documentation.html index fbbc74e..e8daeb7 100644 --- a/_src/documentation.html +++ b/_src/documentation.html @@ -39,6 +39,7 @@ limitations under the License. Documentation such as programming guides, reference guides, and API docs is available online for each version of SystemML below: http://apache.github.io/incubator-systemml/";>Latest +0.14.0 0.13.0 0.12.0
svn commit: r1794484 - in /incubator/systemml/site: documentation.html download.html get-started.html index.html roadmap.html
Author: gweidner Date: Tue May 9 00:17:56 2017 New Revision: 1794484 URL: http://svn.apache.org/viewvc?rev=1794484&view=rev Log: Update pages for 0.14.0 release Modified: incubator/systemml/site/documentation.html incubator/systemml/site/download.html incubator/systemml/site/get-started.html incubator/systemml/site/index.html incubator/systemml/site/roadmap.html Modified: incubator/systemml/site/documentation.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/documentation.html?rev=1794484&r1=1794483&r2=1794484&view=diff == --- incubator/systemml/site/documentation.html (original) +++ incubator/systemml/site/documentation.html Tue May 9 00:17:56 2017 @@ -155,6 +155,7 @@ Documentation such as programming guides, reference guides, and API docs is available online for each version of SystemML below: http://apache.github.io/incubator-systemml/";>Latest +0.14.0 0.13.0 0.12.0 Modified: incubator/systemml/site/download.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/download.html?rev=1794484&r1=1794483&r2=1794484&view=diff == --- incubator/systemml/site/download.html (original) +++ incubator/systemml/site/download.html Tue May 9 00:17:56 2017 @@ -155,7 +155,7 @@ Apache SystemML is released as a source artifact. Binary artifacts (including the ones available in Maven) are made available for your convenience. You will be prompted for a mirror - if the file is not found on yours, please be patient, as it may take 24 hours to reach all mirrors. Apache SystemML is distributed under the http://www.apache.org/licenses/LICENSE-2.0";>Apache License, version 2.0. - Download SystemML 0.13.0-incubating for Spark 2.0.2 and above + Download SystemML 0.14.0-incubating for Spark 2.0.2 and above @@ -167,33 +167,33 @@ Signature -http://www.apache.org/dyn/closer.lua/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-bin.tgz"; target="_blank"> systemml-0.13.0-incubating (Binary tgz) -http://www.apache.org/dist/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-bin.tgz.md5";>MD5 -http://www.apache.org/dist/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-bin.tgz.asc";>ASC +http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz"; target="_blank"> systemml-0.14.0-incubating (Binary tgz) +http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz.md5";>MD5 +http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz.asc";>ASC -http://www.apache.org/dyn/closer.lua/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-bin.zip"; target="_blank">systemml-0.13.0-incubating (Binary zip) -http://www.apache.org/dist/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-bin.zip.md5";>MD5 -http://www.apache.org/dist/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-bin.zip.asc";>ASC +http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip"; target="_blank">systemml-0.14.0-incubating (Binary zip) +http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip.md5";>MD5 +http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip.asc";>ASC -http://www.apache.org/dyn/closer.lua/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-src.tgz"; target="_blank">systemml-0.13.0-incubating (Source tgz) -http://www.apache.org/dist/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-src.tgz.md5";>MD5 -http://www.apache.org/dist/incubator/systemml/0.13.0-incubating/systemml-0.13.0-incubating-src.tgz.asc";>ASC +http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz"; target="_blank">systemml-0.14.0-incubating (Source tgz) +http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz.md5";>MD5 +http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz.asc";>ASC -http://www.apache.org/d
incubator-systemml git commit: [SYSTEMML-1601] Update sizzle license copyright
Repository: incubator-systemml Updated Branches: refs/heads/master 88e79bb34 -> 2fa6abdb4 [SYSTEMML-1601] Update sizzle license copyright Updated sizzle copyright to match contents at https://github.com/jquery/sizzle/blob/master/LICENSE.txt. Closes #486. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/2fa6abdb Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/2fa6abdb Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/2fa6abdb Branch: refs/heads/master Commit: 2fa6abdb4be95e2e3ae20030be6a28ca79f24b5e Parents: 88e79bb Author: Glenn Weidner Authored: Thu May 11 08:14:04 2017 -0700 Committer: Glenn Weidner Committed: Thu May 11 08:14:04 2017 -0700 -- src/assembly/source/LICENSE | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2fa6abdb/src/assembly/source/LICENSE -- diff --git a/src/assembly/source/LICENSE b/src/assembly/source/LICENSE index 204bbc1..973b8ba 100644 --- a/src/assembly/source/LICENSE +++ b/src/assembly/source/LICENSE @@ -312,8 +312,8 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. jQuery v1.12.0 (jquery-1.12.0.min.js) includes sizzle.js (sizzle.min.js) which is distributed under the following license (http://sizzlejs.com/): - -Copyright jQuery Foundation and other contributors, https://jquery.org/ + +Copyright JS Foundation and other contributors, https://js.foundation/ This software consists of voluntary contributions made by many individuals. For exact contribution history, see the revision history @@ -346,7 +346,7 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -Pygments (pygments-default.css) is distributed under the BSD license: +Pygments 2.2.0 (pygments-default.css) is distributed under the BSD license: Copyright (c) 2006-2017 by the respective authors (see AUTHORS file). All rights reserved. @@ -373,3 +373,5 @@ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +
incubator-systemml git commit: [SYSTEMML-1601] Restore jQuery copyright for sizzle license
Repository: incubator-systemml Updated Branches: refs/heads/master 5c9291b52 -> 2fbed1f1a [SYSTEMML-1601] Restore jQuery copyright for sizzle license Revert back to jQuery copyright matching https://github.com/jquery/sizzle/blob/2.2.1/LICENSE.txt. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/2fbed1f1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/2fbed1f1 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/2fbed1f1 Branch: refs/heads/master Commit: 2fbed1f1ab3b9d366fbad35204a0d33a20d33226 Parents: 5c9291b Author: Glenn Weidner Authored: Thu May 11 13:23:06 2017 -0700 Committer: Glenn Weidner Committed: Thu May 11 13:23:06 2017 -0700 -- src/assembly/source/LICENSE | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/2fbed1f1/src/assembly/source/LICENSE -- diff --git a/src/assembly/source/LICENSE b/src/assembly/source/LICENSE index 973b8ba..08f3df1 100644 --- a/src/assembly/source/LICENSE +++ b/src/assembly/source/LICENSE @@ -312,8 +312,8 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. jQuery v1.12.0 (jquery-1.12.0.min.js) includes sizzle.js (sizzle.min.js) which is distributed under the following license (http://sizzlejs.com/): - -Copyright JS Foundation and other contributors, https://js.foundation/ + +Copyright jQuery Foundation and other contributors, https://jquery.org/ This software consists of voluntary contributions made by many individuals. For exact contribution history, see the revision history
incubator-systemml git commit: [SYSTEMML-1605] Add zeppelin sample using new MLContext
Repository: incubator-systemml Updated Branches: refs/heads/master 6adcb369d -> 345e1ab43 [SYSTEMML-1605] Add zeppelin sample using new MLContext Added example for Linear Regression Conjugate Gradient using new MLContext and removed previous Linear Regression example based on deprecated MLContext. Closes #494. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/345e1ab4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/345e1ab4 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/345e1ab4 Branch: refs/heads/master Commit: 345e1ab432df4e7efec9d96f50f95c397f1eafb6 Parents: 6adcb36 Author: Glenn Weidner Authored: Fri May 12 10:19:38 2017 -0700 Committer: Glenn Weidner Committed: Fri May 12 10:19:38 2017 -0700 -- samples/zeppelin-notebooks/2AZ2AQ12B/note.json | 392 --- .../SystemML_LinearRegCG.json | 1 + 2 files changed, 1 insertion(+), 392 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/345e1ab4/samples/zeppelin-notebooks/2AZ2AQ12B/note.json -- diff --git a/samples/zeppelin-notebooks/2AZ2AQ12B/note.json b/samples/zeppelin-notebooks/2AZ2AQ12B/note.json deleted file mode 100644 index 174e9f7..000 --- a/samples/zeppelin-notebooks/2AZ2AQ12B/note.json +++ /dev/null @@ -1,392 +0,0 @@ -{ - "paragraphs": [ -{ - "text": "// Trigger Spark Startup\nsc", - "dateUpdated": "Oct 12, 2015 10:47:23 AM", - "config": { -"colWidth": 12.0, -"graph": { - "mode": "table", - "height": 300.0, - "optionOpen": false, - "keys": [], - "values": [], - "groups": [], - "scatter": {} -}, -"editorMode": "ace/mode/scala" - }, - "settings": { -"params": {}, -"forms": {} - }, - "jobName": "paragraph_1444073218763_655887574", - "id": "20151005-122658_592219673", - "result": { -"code": "SUCCESS", -"type": "TEXT", -"msg": "res8: org.apache.spark.SparkContext \u003d org.apache.spark.SparkContext@6ce70bf3\n" - }, - "dateCreated": "Oct 5, 2015 12:26:58 PM", - "dateStarted": "Oct 12, 2015 10:47:23 AM", - "dateFinished": "Oct 12, 2015 10:47:23 AM", - "status": "FINISHED", - "progressUpdateIntervalMs": 500 -}, -{ - "text": "// Generate data\nimport org.apache.spark.mllib.util.LinearDataGenerator\n\nval numRows \u003d 1\nval numCols \u003d 1000\nval rawData \u003d LinearDataGenerator.generateLinearRDD(sc, numRows, numCols, 1).toDF()\n\n// Repartition into a more parallelism-friendly number of partitions\nval data \u003d rawData.repartition(64).cache()", - "dateUpdated": "Oct 12, 2015 10:49:12 AM", - "config": { -"colWidth": 12.0, -"graph": { - "mode": "table", - "height": 300.0, - "optionOpen": false, - "keys": [], - "values": [], - "groups": [], - "scatter": {} -}, -"editorMode": "ace/mode/scala" - }, - "settings": { -"params": {}, -"forms": {} - }, - "jobName": "paragraph_1444067726904_-135213052", - "id": "20151005-105526_1974722763", - "result": { -"code": "SUCCESS", -"type": "TEXT", -"msg": "import org.apache.spark.mllib.util.LinearDataGenerator\nnumRows: Int \u003d 1\nnumCols: Int \u003d 1000\nrawData: org.apache.spark.sql.DataFrame \u003d [label: double, features: vector]\ndata: org.apache.spark.sql.DataFrame \u003d [label: double, features: vector]\n" - }, - "dateCreated": "Oct 5, 2015 10:55:26 AM", - "dateStarted": "Oct 12, 2015 10:49:12 AM", - "dateFinished": "Oct 12, 2015 10:49:13 AM", - "status": "FINISHED", - "progressUpdateIntervalMs": 500 -}, -{ - "text": "// Spark ML\nimport org.apache.spark.ml.regression.LinearRegression\n\n// Model Settings\nval maxIters \u003d 100\nval reg \u003d 0\nval elasticNetParam \u003d 0 // L2 reg\n\n// Fit the model\nval lr \u003d new LinearRegression()\n .setMaxIter(maxIters)\n .setRegParam(reg)\n .setElasticNetParam(elasticNetParam)\nval start \u003d System.currentTimeMillis()\nval model \u003d lr.fit(data)\nval trainingTime \u003d (System.currentTimeMillis() - start).toDouble / 1000.0\n\n// Summarize the model over the training set and gather some metrics\nval trainingSummary \u003d model.summary\nval r2 \u003d trainingSummary.r2\nval iters \u003d trainingSummary.totalIterations\nval trainingTimePerIter \u003d trainingTime / iters", - "dateUpdated": "Oct 12, 2015 10:47:23 AM", - "config": { -"colWidth": 12.
incubator-systemml git commit: [SYSTEMML-1605] Updated zeppelin tutorial notebook
Repository: incubator-systemml Updated Branches: refs/heads/master db92414b1 -> a3e7e5c49 [SYSTEMML-1605] Updated zeppelin tutorial notebook Closes #495. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/a3e7e5c4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/a3e7e5c4 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/a3e7e5c4 Branch: refs/heads/master Commit: a3e7e5c49678ca6fc9683a5f1c56ea39381765e1 Parents: db92414 Author: Glenn Weidner Authored: Fri May 12 12:20:42 2017 -0700 Committer: Glenn Weidner Committed: Fri May 12 12:20:43 2017 -0700 -- samples/zeppelin-notebooks/tutorial1_zeppelin.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/a3e7e5c4/samples/zeppelin-notebooks/tutorial1_zeppelin.json -- diff --git a/samples/zeppelin-notebooks/tutorial1_zeppelin.json b/samples/zeppelin-notebooks/tutorial1_zeppelin.json index a0385dd..e71b58c 100644 --- a/samples/zeppelin-notebooks/tutorial1_zeppelin.json +++ b/samples/zeppelin-notebooks/tutorial1_zeppelin.json @@ -1 +1 @@ -{"paragraphs":[{"config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/scala"},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1460062914890_499572682","id":"20160407-210154_742995576","dateCreated":"Apr 7, 2016 9:01:54 PM","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:354","text":"%dep\r\nz.load(\"org.apache.systemml:systemml:0.9.0-incubating\")","dateUpdated":"Apr 8, 2016 12:30:17 AM","dateFinished":"Apr 7, 2016 9:04:46 PM","dateStarted":"Apr 7, 2016 9:04:46 PM","result":{"code":"SUCCESS","type":"TEXT","msg":"res1: org.apache.zeppelin.spark.dep.Dependency = org.apache.zeppelin.spark.dep.Dependency@2652117f\n"}},{"config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/scala"},"settings":{"params":{},"forms":{}},"jobName":"paragraph _1460063047961_207364377","id":"20160407-210407_1127760007","dateCreated":"Apr 7, 2016 9:04:07 PM","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:375","dateUpdated":"Apr 7, 2016 9:04:46 PM","dateFinished":"Apr 7, 2016 9:05:10 PM","dateStarted":"Apr 7, 2016 9:04:46 PM","result":{"code":"SUCCESS","type":"TEXT","msg":"import org.apache.sysml.api.MLContext\n"},"text":"import org.apache.sysml.api.MLContext"},{"config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/scala"},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1460063086400_-566304364","id":"20160407-210446_523705226","dateCreated":"Apr 7, 2016 9:04:46 PM","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:395","dateUpdated":"Apr 7, 2016 9:44:43 PM","dateFinished":"Apr 7, 2016 9:44:44 PM","dateStarted":"Apr 7, 2016 9:44:43 PM","result":{"code":"SUCCESS","type":"TEXT"," msg":"import org.apache.spark.sql.SQLContext\nsqlCtx: org.apache.spark.sql.SQLContext = org.apache.spark.sql.SQLContext@44252a8a\nml: org.apache.sysml.api.MLContext = org.apache.sysml.api.MLContext@f5ff11c\ndml: String = \n\"\nX = rand(rows=100, cols=10)\nsumX = sum(X)\noutMatrix = matrix(sumX, rows=1, cols=1)\nwrite(outMatrix, \" \", format=\"csv\")\n\"\nout: org.apache.sysml.api.MLOutput = org.apache.sysml.api.MLOutput@7f2976c4\noutMatrix: org.apache.spark.sql.DataFrame = [ID: double, C1: double]\n"},"text":"import org.apache.spark.sql.SQLContext\nval sqlCtx = new SQLContext(sc)\nval ml = new MLContext(sc)\nval dml = \"\"\"\nX = rand(rows=100, cols=10)\nsumX = sum(X)\noutMatrix = matrix(sumX, rows=1, cols=1)\nwrite(outMatrix, \" \", format=\"csv\")\n\"\"\"\nml.reset()\nml.registerOutput(\"outMatrix\")\nval out = ml.executeScript(dml)\nval outMatrix = out.getDF(sqlCtx, \"outMatrix\")"},{"config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"value s":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/scala"},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1460063826194_1965196735","id":"20160407-211706_2075868632","dateCreated":"Apr 7, 2016 9:17:06 PM","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:413","dateUpdated":"Apr 7, 2016 9:45:23 PM","dateFinished":"Apr 7, 2016 9:45:23 PM","dateStarted":"Apr 7, 2016 9:45:23 PM","result":{"code":"SUCCESS","type":"TEXT","msg":"+---+--+\n| ID| C1|\n+---+--
[2/2] incubator-systemml git commit: [SYSTEMML-1605] Remove one sample zeppelin notebook
[SYSTEMML-1605] Remove one sample zeppelin notebook Closes #498. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/9ee569e0 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/9ee569e0 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/9ee569e0 Branch: refs/heads/master Commit: 9ee569e0f186fe78867795b047597dc83a1d82b5 Parents: f29ea30 Author: Glenn Weidner Authored: Fri May 12 19:07:48 2017 -0700 Committer: Glenn Weidner Committed: Fri May 12 19:07:49 2017 -0700 -- samples/zeppelin-notebooks/2BCHR4T1Q/note.json | 1666 --- 1 file changed, 1666 deletions(-) --
[1/2] incubator-systemml git commit: [SYSTEMML-1605] Remove one sample zeppelin notebook
Repository: incubator-systemml Updated Branches: refs/heads/master f29ea3090 -> 9ee569e0f http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/9ee569e0/samples/zeppelin-notebooks/2BCHR4T1Q/note.json -- diff --git a/samples/zeppelin-notebooks/2BCHR4T1Q/note.json b/samples/zeppelin-notebooks/2BCHR4T1Q/note.json deleted file mode 100755 index b340a2f..000 --- a/samples/zeppelin-notebooks/2BCHR4T1Q/note.json +++ /dev/null @@ -1,1666 +0,0 @@ -{ - "paragraphs": [ -{ - "title": "Let us first read the 311 complaint data for new york city from hDFS", - "text": "val nyc_sdff \u003d \nsqlContext.read.format(\"com.databricks.spark.csv\").option(\"header\",\"true\").load(\"hdfs://localhost:9000/datasets/nyc-311-service/311_Service_Requests_from_2010_to_Present.csv\").cache()", - "dateUpdated": "Feb 11, 2016 7:46:52 PM", - "config": { -"colWidth": 12.0, -"editorMode": "ace/mode/scala", -"graph": { - "mode": "table", - "height": 300.0, - "optionOpen": false, - "keys": [], - "values": [], - "groups": [], - "scatter": {} -}, -"title": true - }, - "settings": { -"params": {}, -"forms": {} - }, - "jobName": "paragraph_1454426714926_618059279", - "id": "20160202-152514_1906486717", - "result": { -"code": "SUCCESS", -"type": "TEXT", -"msg": "nyc_sdff: org.apache.spark.sql.DataFrame \u003d [Unique Key: string, Created Date: string, Closed Date: string, Agency: string, Agency Name: string, Complaint Type: string, Descriptor: string, Location Type: string, Incident Zip: string, Incident Address: string, Street Name: string, Cross Street 1: string, Cross Street 2: string, Intersection Street 1: string, Intersection Street 2: string, Address Type: string, City: string, Landmark: string, Facility Type: string, Status: string, Due Date: string, Resolution Description: string, Resolution Action Updated Date: string, Community Board: string, Borough: string, X Coordinate (State Plane): string, Y Coordinate (State Plane): string, Park Facility Name: string, Park Borough: string, School Name: string, School Number: string, School Region: s..." - }, - "dateCreated": "Feb 2, 2016 3:25:14 PM", - "dateStarted": "Feb 11, 2016 7:46:56 PM", - "dateFinished": "Feb 11, 2016 7:47:39 PM", - "status": "FINISHED", - "progressUpdateIntervalMs": 500 -}, -{ - "title": "Create a Temporary Spark Table out of the Spark DataFrame", - "text": "nyc_sdff.registerTempTable(\"nyc311f\")", - "dateUpdated": "Feb 11, 2016 7:51:07 PM", - "config": { -"colWidth": 12.0, -"editorMode": "ace/mode/scala", -"title": true, -"graph": { - "mode": "table", - "height": 300.0, - "optionOpen": false, - "keys": [], - "values": [], - "groups": [], - "scatter": {} -} - }, - "settings": { -"params": {}, -"forms": {} - }, - "jobName": "paragraph_1445353951173_-79877865", - "id": "20151020-151231_921365907", - "result": { -"code": "SUCCESS", -"type": "TEXT", -"msg": "" - }, - "dateCreated": "Oct 20, 2015 3:12:31 PM", - "dateStarted": "Feb 11, 2016 7:51:11 PM", - "dateFinished": "Feb 11, 2016 7:51:12 PM", - "status": "FINISHED", - "progressUpdateIntervalMs": 500 -}, -{ - "title": "Total 311 Complaints in New York City In last 1 year", - "text": "%sql select count(*) as TotalComplaintCountFrom2015 from nyc311f", - "dateUpdated": "Feb 11, 2016 7:51:19 PM", - "config": { -"colWidth": 12.0, -"graph": { - "mode": "table", - "height": 95.0, - "optionOpen": false, - "keys": [], - "values": [], - "groups": [], - "scatter": {} -}, -"editorMode": "ace/mode/sql", -"title": true - }, - "settings": { -"params": {}, -"forms": {} - }, - "jobName": "paragraph_1454428061459_102213864", - "id": "20160202-154741_356355521", - "result": { -"code": "SUCCESS", -"type": "TABLE", -"msg": "TotalComplaintCountFrom2015\n1652212\n" - }, - "dateCreated": "Feb 2, 2016 3:47:41 PM", - "dateStarted": "Feb 11, 2016 7:51:21 PM", - "dateFinished": "Feb 11, 2016 7:51:40 PM", - "status": "FINISHED", - "progressUpdateIntervalMs": 500 -}, -{ - "title": "Let us investigate some relevant columns in 311 data", - "text": "%sql select `Unique Key` as Unique_key, `Created Date` as Created_Date, `Complaint Type` as Complaint_Type, Borough, `Incident Address` as Incident_Address, `Incident Zip` as Incident_Zip from nyc311f", - "dateU
incubator-systemml-website git commit: [FIX] Removal of incubator from download and typos in install
Repository: incubator-systemml-website Updated Branches: refs/heads/master 7670c217b -> 494c4adb6 [FIX] Removal of incubator from download and typos in install Closes #57. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/494c4adb Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/494c4adb Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/494c4adb Branch: refs/heads/master Commit: 494c4adb6fcda64f92f697a52ea41ee0d99b2c16 Parents: 7670c21 Author: Arvind Surve Authored: Mon May 22 09:06:29 2017 -0700 Committer: Glenn Weidner Committed: Mon May 22 09:06:29 2017 -0700 -- _src/_includes/themes/apache/home.html | 2 +- _src/install-systemml.html | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/494c4adb/_src/_includes/themes/apache/home.html -- diff --git a/_src/_includes/themes/apache/home.html b/_src/_includes/themes/apache/home.html index d5dafef..5f16bdc 100644 --- a/_src/_includes/themes/apache/home.html +++ b/_src/_includes/themes/apache/home.html @@ -26,7 +26,7 @@ limitations under the License. A machine learning platform optimal for big data - Download SystemML {{ site.data.project.release_version }} + Download SystemML http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/494c4adb/_src/install-systemml.html -- diff --git a/_src/install-systemml.html b/_src/install-systemml.html index 27d2ef0..b79e2ff 100644 --- a/_src/install-systemml.html +++ b/_src/install-systemml.html @@ -99,14 +99,14 @@ pip install https://sparktc.ibmcloud.com/repo/latest/systemml-1.0.0-incubating-S - Start Jupyter Notebook Server -jupyter notebook + # Start Jupyter Notebook Server +PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark --master local[*] --conf "spark.driver.memory=12g" --conf spark.driver.maxResultSize=0 --conf spark.akka.frameSize=128 --conf spark.default.parallelism=100 Toree Kernel Setup (Required for Scala Kernel)Toree installation{% highlight bash %}# For detail instructions visit https://github.com/apache/incubator-toree pip install https://dist.apache.org/repos/dist/dev/incubator/toree/0.2.0/snapshots/dev1/toree-pip/toree-0.2.0.dev1.tar.gz{% endhighlight %}Installation of Toree component in Jupyter{% highlight bash %}# For detail instructions visit https://toree.apache.org/docs/current/user/installation/ -jupyter toree install â-replace â-interpreters=Scala,PySpark --spark_opts="--master=local --jars â --spark_home=${SPARK_HOME}{% endhighlight %}Start Jupyter Notebook Server{% highlight bash %}PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark --master local[*] --conf "spark.driver.memory=12g" --conf spark.driver.maxResultSize=0 --conf spark.akka.frameSize=128 --conf spark.default.parallelism=100{% endhighlight %}This will a default browser with contents from current directory where above command has run. You can create your own notebook example or download sample notebooks from SystemML resository https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks";>https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks +jupyter toree install â-replace â-interpreters=Scala,PySpark --spark_opts="--master=local --jars â --spark_home=${SPARK_HOME}{% endhighlight %}Start Jupyter Notebook Server{% highlight bash %}jupyter notebook{% endhighlight %}This will start a default browser with contents from current directory where above command has run. You can create your own notebook example or download sample notebooks from SystemML resository https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks";>https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks
svn commit: r1795829 - in /incubator/systemml/site: index.html install-systemml.html
Author: gweidner Date: Mon May 22 16:23:43 2017 New Revision: 1795829 URL: http://svn.apache.org/viewvc?rev=1795829&view=rev Log: Remove incubator from download button and fix jupyter install typos Modified: incubator/systemml/site/index.html incubator/systemml/site/install-systemml.html Modified: incubator/systemml/site/index.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/index.html?rev=1795829&r1=1795828&r2=1795829&view=diff == --- incubator/systemml/site/index.html (original) +++ incubator/systemml/site/index.html Mon May 22 16:23:43 2017 @@ -151,7 +151,7 @@ A machine learning platform optimal for big data - Download SystemML 0.14.0-incubating + Download SystemML Modified: incubator/systemml/site/install-systemml.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/install-systemml.html?rev=1795829&r1=1795828&r2=1795829&view=diff == --- incubator/systemml/site/install-systemml.html (original) +++ incubator/systemml/site/install-systemml.html Mon May 22 16:23:43 2017 @@ -221,14 +221,14 @@ pip install https://sparktc.ibmcloud.com - Start Jupyter Notebook Server -jupyter notebook + # Start Jupyter Notebook Server +PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark --master local[*] --conf "spark.driver.memory=12g" --conf spark.driver.maxResultSize=0 --conf spark.akka.frameSize=128 --conf spark.default.parallelism=100 Toree Kernel Setup (Required for Scala Kernel)Toree installation# For detail instructions visit https://github.com/apache/incubator-toree pip install https://dist.apache.org/repos/dist/dev/incubator/toree/0.2.0/snapshots/dev1/toree-pip/toree-0.2.0.dev1.tar.gzInstallation of Toree component in Jupyter# For detail instructions visit https://toree.apache.org/docs/current/user/installation/ -jupyter toree install â-replace â-interpreters=Scala,PySpark --spark_opts="--master=local --jars <SystemML JAR File>â --spark_home=${SPARK_HOME}Start Jupyter Notebook ServerPYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark --master local[*] --conf "spark.driver.memory=12g" --conf spark.driver.maxResultSize=0 --conf spark.akka.frameSize=128 --conf spark.default.parallelism=100This will a default browser with contents from current directory where >above command has run. You can create your own notebook example or download >sample notebooks from SystemML resository href="https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks";>https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks class="img-border">alt="Start Jupyter Notebook Server">src="/assets/img/systemml-juypter-install-2.jpeg" alt="Start Jupyter Notebook >Server"> +jupyter toree install â-replace â-interpreters=Scala,PySpark --spark_opts="--master=local --jars <SystemML JAR File>â --spark_home=${SPARK_HOME}Start Jupyter Notebook Serverjupyter notebookThis will start a default browser with contents from current directory where above command has run. You can create your own notebook example or download sample notebooks from SystemML resository https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks";>https://github.com/apache/incubator-systemml/tree/master/samples/jupyter-notebooks
incubator-systemml-website git commit: [MINOR] Fix punctuation direction for 'What is SystemML'
Repository: incubator-systemml-website Updated Branches: refs/heads/master 494c4adb6 -> 216764350 [MINOR] Fix punctuation direction for 'What is SystemML' Closes #58. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/21676435 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/21676435 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/21676435 Branch: refs/heads/master Commit: 2167643501d134ab76bb3c7d5271f7fc508f196d Parents: 494c4ad Author: Glenn Weidner Authored: Mon May 22 11:24:22 2017 -0700 Committer: Glenn Weidner Committed: Mon May 22 11:24:22 2017 -0700 -- _src/_includes/themes/apache/home.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/21676435/_src/_includes/themes/apache/home.html -- diff --git a/_src/_includes/themes/apache/home.html b/_src/_includes/themes/apache/home.html index 5f16bdc..c94623a 100644 --- a/_src/_includes/themes/apache/home.html +++ b/_src/_includes/themes/apache/home.html @@ -55,9 +55,9 @@ limitations under the License. - What is SystemML? + What is SystemML? - Apache SystemML provides an optimal workplace for machine learning using big data. It can be run on top of Apache Spark, where it automatically scales your data, line by line, determining whether your code should be run on the driver or an Apache Spark cluster. Future SystemML developments include additional deep learning with GPU capabilities such as importing and running neural network architectures and pre-trained models for training. + Apache SystemML provides an optimal workplace for machine learning using big data. It can be run on top of Apache Spark, where it automatically scales your data, line by line, determining whether your code should be run on the driver or an Apache Spark cluster. Future SystemML developments include additional deep learning with GPU capabilities such as importing and running neural network architectures and pre-trained models for training. Learn more about SystemML
svn commit: r1795836 - /incubator/systemml/site/index.html
Author: gweidner Date: Mon May 22 18:37:33 2017 New Revision: 1795836 URL: http://svn.apache.org/viewvc?rev=1795836&view=rev Log: [MINOR] Fix punctuation direction for 'What is SystemML' Modified: incubator/systemml/site/index.html Modified: incubator/systemml/site/index.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/index.html?rev=1795836&r1=1795835&r2=1795836&view=diff == --- incubator/systemml/site/index.html (original) +++ incubator/systemml/site/index.html Mon May 22 18:37:33 2017 @@ -180,9 +180,9 @@ - What is SystemML? + What is SystemML? - Apache SystemML provides an optimal workplace for machine learning using big data. It can be run on top of Apache Spark, where it automatically scales your data, line by line, determining whether your code should be run on the driver or an Apache Spark cluster. Future SystemML developments include additional deep learning with GPU capabilities such as importing and running neural network architectures and pre-trained models for training. + Apache SystemML provides an optimal workplace for machine learning using big data. It can be run on top of Apache Spark, where it automatically scales your data, line by line, determining whether your code should be run on the driver or an Apache Spark cluster. Future SystemML developments include additional deep learning with GPU capabilities such as importing and running neural network architectures and pre-trained models for training. Learn more about SystemML
incubator-systemml-website git commit: [MINOR] Linear Regression Algorithms Demo notebook first in list
Repository: incubator-systemml-website Updated Branches: refs/heads/master 216764350 -> dc4e296be [MINOR] Linear Regression Algorithms Demo notebook first in list Moved Linear Regression Algorithms Demo notebook to first in list and removed one Zeppelin notebook sample. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/dc4e296b Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/dc4e296b Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/dc4e296b Branch: refs/heads/master Commit: dc4e296be2fa5267b91a7f40eb6aa9e713394ecc Parents: 2167643 Author: Glenn Weidner Authored: Mon May 22 12:24:39 2017 -0700 Committer: Glenn Weidner Committed: Mon May 22 12:24:39 2017 -0700 -- _src/get-started.html | 17 ++--- 1 file changed, 6 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/dc4e296b/_src/get-started.html -- diff --git a/_src/get-started.html b/_src/get-started.html index 60adbdf..3025f01 100644 --- a/_src/get-started.html +++ b/_src/get-started.html @@ -50,9 +50,9 @@ limitations under the License. Sample Notebooks -SystemML LinearRegCG -SystemML Linear Regression in Zeppelin Notebook. -https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/SystemML_LinearRegCG.json"; target="_blank">View on Github +Linear Regression Algorithms Demo +This notebook demonstrates the development of various linear regression algorithms in SystemML. +https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb"; target="_blank">View on Github Deep Learning Image Classification @@ -60,11 +60,6 @@ limitations under the License. https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb"; target="_blank">View on Github -Linear Regression Algorithms Demo -This notebook demonstrates the development of various linear regression algorithms in SystemML. -https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb"; target="_blank">View on Github - - SystemML PySpark Recommendation Demo This demonstrates using SystemML for product recommendation using Poisson NonNegative Matrix Factorization (PNMF) with PNMF algorithm written using R like syntax. https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/SystemML-PySpark-Recommendation-Demo.ipynb"; target="_blank">View on Github @@ -80,9 +75,9 @@ limitations under the License. https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Autoencoder.ipynb"; target="_blank">View on Github -SystemML Zeppelin Tutorial -SystemML Zeppelin tutorial. -https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/tutorial1_zeppelin.json"; target="_blank">View on Github +SystemML Linear Regression Conjugate Gradient +This example shows SystemML Linear Regression Conjugate Gradient algorithm in Zeppelin Notebook. +https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/SystemML_LinearRegCG.json"; target="_blank">View on Github
svn commit: r1795842 - /incubator/systemml/site/get-started.html
Author: gweidner Date: Mon May 22 19:36:19 2017 New Revision: 1795842 URL: http://svn.apache.org/viewvc?rev=1795842&view=rev Log: [MINOR] Linear Regression Algorithms Demo notebook first in list Modified: incubator/systemml/site/get-started.html Modified: incubator/systemml/site/get-started.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/get-started.html?rev=1795842&r1=1795841&r2=1795842&view=diff == --- incubator/systemml/site/get-started.html (original) +++ incubator/systemml/site/get-started.html Mon May 22 19:36:19 2017 @@ -173,9 +173,9 @@ Sample Notebooks -SystemML LinearRegCG -SystemML Linear Regression in Zeppelin Notebook. -https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/SystemML_LinearRegCG.json"; target="_blank">View on Github +Linear Regression Algorithms Demo +This notebook demonstrates the development of various linear regression algorithms in SystemML. +https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb"; target="_blank">View on Github Deep Learning Image Classification @@ -183,11 +183,6 @@ https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb"; target="_blank">View on Github -Linear Regression Algorithms Demo -This notebook demonstrates the development of various linear regression algorithms in SystemML. -https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb"; target="_blank">View on Github - - SystemML PySpark Recommendation Demo This demonstrates using SystemML for product recommendation using Poisson NonNegative Matrix Factorization (PNMF) with PNMF algorithm written using R like syntax. https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/SystemML-PySpark-Recommendation-Demo.ipynb"; target="_blank">View on Github @@ -203,9 +198,9 @@ https://github.com/apache/incubator-systemml/blob/master/samples/jupyter-notebooks/Autoencoder.ipynb"; target="_blank">View on Github -SystemML Zeppelin Tutorial -SystemML Zeppelin tutorial. -https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/tutorial1_zeppelin.json"; target="_blank">View on Github +SystemML Linear Regression Conjugate Gradient +This example shows SystemML Linear Regression Conjugate Gradient algorithm in Zeppelin Notebook. +https://github.com/apache/incubator-systemml/blob/master/samples/zeppelin-notebooks/SystemML_LinearRegCG.json"; target="_blank">View on Github
incubator-systemml git commit: [SYSTEMML-1614] Updated python pre_setup and classloader for TLP jar names
Repository: incubator-systemml Updated Branches: refs/heads/master 9802c4b0b -> 5dd4d44ec [SYSTEMML-1614] Updated python pre_setup and classloader for TLP jar names Removed '-incubating' from file pattern matching expression for python pre_setup and classloader so the renamed jar is found. Closes #510. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/5dd4d44e Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/5dd4d44e Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/5dd4d44e Branch: refs/heads/master Commit: 5dd4d44ecb44ab4fff4db3954fa09ff137e4b8ba Parents: 9802c4b Author: Glenn Weidner Authored: Tue May 23 12:22:52 2017 -0700 Committer: Glenn Weidner Committed: Tue May 23 12:22:53 2017 -0700 -- src/main/python/pre_setup.py| 6 -- src/main/python/systemml/classloader.py | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/5dd4d44e/src/main/python/pre_setup.py -- diff --git a/src/main/python/pre_setup.py b/src/main/python/pre_setup.py index cb421a9..0b7d7b5 100644 --- a/src/main/python/pre_setup.py +++ b/src/main/python/pre_setup.py @@ -30,9 +30,11 @@ if os.path.exists(java_dir_full_path): os.mkdir(java_dir_full_path) root_dir = os.path.dirname(os.path.dirname(os.path.dirname(os.getcwd( for file in os.listdir(os.path.join(root_dir, 'target')): -if fnmatch.fnmatch(file, 'systemml-*-incubating-SNAPSHOT.jar') or fnmatch.fnmatch(file, 'systemml-*-incubating.jar'): +if (fnmatch.fnmatch(file, 'systemml-*-SNAPSHOT.jar') or fnmatch.fnmatch(file, 'systemml-*.jar') +and not (fnmatch.fnmatch(file, 'systemml-*javadoc.jar') + or fnmatch.fnmatch(file, 'systemml-*sources.jar'))): shutil.copyfile(os.path.join(root_dir, 'target', file), os.path.join(java_dir_full_path, file)) -if fnmatch.fnmatch(file, 'systemml-*-incubating-SNAPSHOT-extra.jar') or fnmatch.fnmatch(file, 'systemml-*-incubating-extra.jar'): +if fnmatch.fnmatch(file, 'systemml-*-SNAPSHOT-extra.jar') or fnmatch.fnmatch(file, 'systemml-*-extra.jar'): shutil.copyfile(os.path.join(root_dir, 'target', file), os.path.join(java_dir_full_path, file)) http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/5dd4d44e/src/main/python/systemml/classloader.py -- diff --git a/src/main/python/systemml/classloader.py b/src/main/python/systemml/classloader.py index 45fe4de..8738dc5 100644 --- a/src/main/python/systemml/classloader.py +++ b/src/main/python/systemml/classloader.py @@ -43,7 +43,7 @@ def _getJarFileName(sc, suffix): jar_file_name = '_ignore.jar' java_dir = os.path.join(imp.find_module("systemml")[1], "systemml-java") for file in os.listdir(java_dir): -if fnmatch.fnmatch(file, 'systemml-*-incubating-SNAPSHOT' + suffix + '.jar') or fnmatch.fnmatch(file, 'systemml-*-incubating' + suffix + '.jar'): +if fnmatch.fnmatch(file, 'systemml-*-SNAPSHOT' + suffix + '.jar') or fnmatch.fnmatch(file, 'systemml-*' + suffix + '.jar'): jar_file_name = os.path.join(java_dir, file) return jar_file_name
incubator-systemml-website git commit: [SYSTEMML-1619] Remove nightly builds link from download page
Repository: incubator-systemml-website Updated Branches: refs/heads/master 45a45d15b -> 372e84f72 [SYSTEMML-1619] Remove nightly builds link from download page Removed nightly builds link as per Apache release policy at http://www.apache.org/legal/release-policy.html#what. Closes #59. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/372e84f7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/372e84f7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/372e84f7 Branch: refs/heads/master Commit: 372e84f7235118bea8319b0cd19ce1d471ca19a1 Parents: 45a45d1 Author: Glenn Weidner Authored: Tue May 23 13:56:09 2017 -0700 Committer: Glenn Weidner Committed: Tue May 23 13:56:09 2017 -0700 -- _src/download.html | 3 --- 1 file changed, 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/372e84f7/_src/download.html -- diff --git a/_src/download.html b/_src/download.html index f6e10f5..929b0b9 100644 --- a/_src/download.html +++ b/_src/download.html @@ -86,9 +86,6 @@ limitations under the License. Instructions for checking hashes and signatures is described on the http://www.apache.org/info/verification.html"; target="_blank">Verifying Apache Software Foundation Releases page. - Nightly Experimental Builds - Nightly experimental builds for Spark 2.x can be obtained from https://sparktc.ibmcloud.com/repo/latest/"; target="_blank">our nightly build page. - Bleeding-Edge You can also retrieve the source files from our https://github.com/apache/incubator-systemml"; target="_blank">Git repository and create a bleeding-edge build by typing:
svn commit: r1795964 - /incubator/systemml/site/download.html
Author: gweidner Date: Tue May 23 21:14:24 2017 New Revision: 1795964 URL: http://svn.apache.org/viewvc?rev=1795964&view=rev Log: [SYSTEMML-1619] Remove nightly builds link from download page Modified: incubator/systemml/site/download.html Modified: incubator/systemml/site/download.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/download.html?rev=1795964&r1=1795963&r2=1795964&view=diff == --- incubator/systemml/site/download.html (original) +++ incubator/systemml/site/download.html Tue May 23 21:14:24 2017 @@ -209,9 +209,6 @@ Instructions for checking hashes and signatures is described on the http://www.apache.org/info/verification.html"; target="_blank">Verifying Apache Software Foundation Releases page. - Nightly Experimental Builds - Nightly experimental builds for Spark 2.x can be obtained from https://sparktc.ibmcloud.com/repo/latest/"; target="_blank">our nightly build page. - Bleeding-Edge You can also retrieve the source files from our https://github.com/apache/incubator-systemml"; target="_blank">Git repository and create a bleeding-edge build by typing:
incubator-systemml git commit: [SYSTEMML-1620] Remove incubation from python package info
Repository: incubator-systemml Updated Branches: refs/heads/master 5dd4d44ec -> c4228349d [SYSTEMML-1620] Remove incubation from python package info As part of top level project updates, removed incubation disclaimer from description text in setup.py used for generating pkg-info. Closes #512. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/c4228349 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/c4228349 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/c4228349 Branch: refs/heads/master Commit: c4228349d2765c3b88e63e272cc0753988db8675 Parents: 5dd4d44 Author: Glenn Weidner Authored: Tue May 23 16:28:06 2017 -0700 Committer: Glenn Weidner Committed: Tue May 23 16:28:06 2017 -0700 -- src/main/python/setup.py | 8 +--- 1 file changed, 1 insertion(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/c4228349/src/main/python/setup.py -- diff --git a/src/main/python/setup.py b/src/main/python/setup.py index 3ba9ac6..d55c99f 100644 --- a/src/main/python/setup.py +++ b/src/main/python/setup.py @@ -63,12 +63,6 @@ setup( description='Apache SystemML is a distributed and declarative machine learning platform.', long_description=''' -Apache SystemML is an effort undergoing incubation at the Apache Software Foundation (ASF), -sponsored by the Apache Incubator PMC. -While incubation status is not necessarily a reflection of the completeness -or stability of the code, it does indicate that the project has yet to be -fully endorsed by the ASF. - Apache SystemML provides declarative large-scale machine learning (ML) that aims at flexible specification of ML algorithms and automatic generation of hybrid runtime plans ranging from single-node, in-memory computations, to distributed computations on Apache @@ -76,7 +70,7 @@ setup( ''', url='http://systemml.apache.org/', author='Apache SystemML', -author_email='d...@systemml.incubator.apache.org', +author_email='d...@systemml.apache.org', packages=find_packages(), install_requires=REQUIRED_PACKAGES, include_package_data=True,
incubator-systemml-website git commit: [SYSTEMML-1618] Use https for KEYS, md5, asc on download page
Repository: incubator-systemml-website Updated Branches: refs/heads/master 372e84f72 -> 5f6d450fa [SYSTEMML-1618] Use https for KEYS, md5, asc on download page Switched to https for .md5 and .asc download links. Also added link to KEYS file stored on main distribution site. Closes #60. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/5f6d450f Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/5f6d450f Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/5f6d450f Branch: refs/heads/master Commit: 5f6d450fa68b82f4e7ff90d9ae10c16859ec0222 Parents: 372e84f Author: Glenn Weidner Authored: Wed May 24 11:51:51 2017 -0700 Committer: Glenn Weidner Committed: Wed May 24 11:51:51 2017 -0700 -- _src/download.html | 24 +--- 1 file changed, 13 insertions(+), 11 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/5f6d450f/_src/download.html -- diff --git a/_src/download.html b/_src/download.html index 929b0b9..52734cc 100644 --- a/_src/download.html +++ b/_src/download.html @@ -52,28 +52,28 @@ limitations under the License. http://www.apache.org/dyn/closer.lua/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.tgz" target="_blank"> {{ site.data.project.release_name }}-{{ site.data.project.release_version }} (Binary tgz) -http://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.tgz.md5">MD5 -http://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.tgz.asc">ASC +https://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.tgz.md5">MD5 +https://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.tgz.asc">ASC http://www.apache.org/dyn/closer.lua/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.zip" target="_blank">{{ site.data.project.release_name }}-{{ site.data.project.release_version }} (Binary zip) -http://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.zip.md5">MD5 -http://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.zip.asc">ASC +https://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.zip.md5">MD5 +https://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.zip.asc">ASC http://www.apache.org/dyn/closer.lua/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-src.tgz" target="_blank">{{ site.data.project.release_name }}-{{ site.data.project.release_version }} (Source tgz) -http://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-src.tgz.md5">MD5 -http://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-src.tgz.asc">ASC +https://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-src.tgz.md5">MD5 +https://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-src.tgz.asc">ASC http://www.apache.org/dyn/closer.lua/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-src.zip" target="_blank">{{ site.data.project.release_name }}-{{ site.data.project.release_version }} (Source zip) -http://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-src.zip.md5">MD5 -http://www.apache.org/dist/incubator/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}
svn commit: r1796088 - /incubator/systemml/site/download.html
Author: gweidner Date: Wed May 24 19:04:45 2017 New Revision: 1796088 URL: http://svn.apache.org/viewvc?rev=1796088&view=rev Log: [SYSTEMML-1618] Use https for KEYS, md5, asc on download page Modified: incubator/systemml/site/download.html Modified: incubator/systemml/site/download.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/download.html?rev=1796088&r1=1796087&r2=1796088&view=diff == --- incubator/systemml/site/download.html (original) +++ incubator/systemml/site/download.html Wed May 24 19:04:45 2017 @@ -175,28 +175,28 @@ http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz"; target="_blank"> systemml-0.14.0-incubating (Binary tgz) -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz.md5";>MD5 -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz.asc";>ASC +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz.md5";>MD5 +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz.asc";>ASC http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip"; target="_blank">systemml-0.14.0-incubating (Binary zip) -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip.md5";>MD5 -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip.asc";>ASC +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip.md5";>MD5 +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip.asc";>ASC http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz"; target="_blank">systemml-0.14.0-incubating (Source tgz) -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz.md5";>MD5 -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz.asc";>ASC +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz.md5";>MD5 +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz.asc";>ASC http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.zip"; target="_blank">systemml-0.14.0-incubating (Source zip) -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.zip.md5";>MD5 -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.zip.asc";>ASC +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.zip.md5";>MD5 +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.zip.asc";>ASC http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-python.tgz"; target="_blank">systemml-0.14.0-incubating (Python package .tgz) -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-python.tgz.md5";>MD5 -http://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-python.tgz.asc";>ASC +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-python.tgz.md5";>MD5 +https://www.apache.org/dist/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-python.tgz.asc";>ASC @@ -206,7 +206,9 @@ - Instructions for checking hashes and signatures is described on the http://www.apache.org/info/verification.html"; target="_blank">Verifying Apache Software Foundation Releases page. + Instructions for checking hashes and signatures is described on the http://www.apache.org/info/verification.html"; target="_blank">Verifying Apache Software Foundation Releases page. + These https://www.apache.org/dist/incubator/systemml/KEYS"; target="_blank">KEYS can be used to http://www.apache.org/dyn/closer.cgi#verify"; target="_blank">verify the integrity of the downloaded artifact using hash (.md5) or PGP signature (.asc). + Bleeding-Edge
incubator-systemml-website git commit: Minor correction for BLAS in roadmap
Repository: incubator-systemml-website Updated Branches: refs/heads/master 83b7ec8b5 -> 65eb7dc27 Minor correction for BLAS in roadmap Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/65eb7dc2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/65eb7dc2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/65eb7dc2 Branch: refs/heads/master Commit: 65eb7dc278019e571d6cef3ee94067849c587d23 Parents: 83b7ec8 Author: Glenn Weidner Authored: Thu May 25 13:25:04 2017 -0700 Committer: Glenn Weidner Committed: Thu May 25 13:25:05 2017 -0700 -- _src/roadmap.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/65eb7dc2/_src/roadmap.html -- diff --git a/_src/roadmap.html b/_src/roadmap.html index 8c6fd1b..127ea16 100644 --- a/_src/roadmap.html +++ b/_src/roadmap.html @@ -43,7 +43,7 @@ limitations under the License. Deep Learning Single GPU support -Native Blass support +Native BLAS support Code Generation
svn commit: r1796193 - /incubator/systemml/site/roadmap.html
Author: gweidner Date: Thu May 25 20:30:06 2017 New Revision: 1796193 URL: http://svn.apache.org/viewvc?rev=1796193&view=rev Log: Minor correction for BLAS in roadmap Modified: incubator/systemml/site/roadmap.html Modified: incubator/systemml/site/roadmap.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/roadmap.html?rev=1796193&r1=1796192&r2=1796193&view=diff == --- incubator/systemml/site/roadmap.html (original) +++ incubator/systemml/site/roadmap.html Thu May 25 20:30:06 2017 @@ -167,7 +167,7 @@ Deep Learning Single GPU support -Native Blass support +Native BLAS support Code Generation
incubator-systemml git commit: [SYSTEMML-1622] Remove incubator from email references
Repository: incubator-systemml Updated Branches: refs/heads/master eea72ce57 -> 841a4d030 [SYSTEMML-1622] Remove incubator from email references Updated email addresses to systemml.apache.org (removed incubator) in pom, md, readme files. Closes #514. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/841a4d03 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/841a4d03 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/841a4d03 Branch: refs/heads/master Commit: 841a4d0301e666a5719559552294417e19e21221 Parents: eea72ce Author: Glenn Weidner Authored: Thu May 25 13:45:31 2017 -0700 Committer: Glenn Weidner Committed: Thu May 25 13:45:31 2017 -0700 -- README.md| 2 +- docs/contributing-to-systemml.md | 20 ++-- pom.xml | 12 ++-- src/main/standalone/README.txt | 2 +- 4 files changed, 18 insertions(+), 18 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/841a4d03/README.md -- diff --git a/README.md b/README.md index 592e422..40337cb 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ limitations under the License. # SystemML **Documentation:** [SystemML Documentation](http://apache.github.io/incubator-systemml/) -**Mailing List:** [Dev Mailing List](mailto:d...@systemml.incubator.apache.org) +**Mailing List:** [Dev Mailing List](mailto:d...@systemml.apache.org) **Build Status:** [![Build Status](https://sparktc.ibmcloud.com/jenkins/job/SystemML-DailyTest/badge/icon)](https://sparktc.ibmcloud.com/jenkins/job/SystemML-DailyTest) **Issue Tracker:** [JIRA](https://issues.apache.org/jira/browse/SYSTEMML) **Download:** [Download SystemML](http://systemml.apache.org/download.html) http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/841a4d03/docs/contributing-to-systemml.md -- diff --git a/docs/contributing-to-systemml.md b/docs/contributing-to-systemml.md index 2b98dd0..5e36267 100644 --- a/docs/contributing-to-systemml.md +++ b/docs/contributing-to-systemml.md @@ -34,28 +34,28 @@ There are many ways to become involved with SystemML: ### Development Mailing List Perhaps the easiest way to obtain help and contribute to SystemML is to join the SystemML Development -mailing list (d...@systemml.incubator.apache.org). You can subscribe to this list by sending an email to -[dev-subscr...@systemml.incubator.apache.org](mailto:dev-subscr...@systemml.incubator.apache.org). -You can unsubscribe from this list by sending an email to [dev-unsubscr...@systemml.incubator.apache.org](mailto:dev-unsubscr...@systemml.incubator.apache.org). The dev mailing list archive can be found -[here](http://mail-archives.apache.org/mod_mbox/incubator-systemml-dev/). +mailing list (d...@systemml.apache.org). You can subscribe to this list by sending an email to +[dev-subscr...@systemml.apache.org](mailto:dev-subscr...@systemml.apache.org). +You can unsubscribe from this list by sending an email to [dev-unsubscr...@systemml.apache.org](mailto:dev-unsubscr...@systemml.apache.org). The dev mailing list archive can be found +[here](http://mail-archives.apache.org/mod_mbox/systemml-dev/). ### Issues Mailing List The issues mailing list archive can be found -[here](http://mail-archives.apache.org/mod_mbox/incubator-systemml-issues/). +[here](http://mail-archives.apache.org/mod_mbox/systemml-issues/). To subscribe to the issues list, send an email to -[issues-subscr...@systemml.incubator.apache.org](mailto:issues-subscr...@systemml.incubator.apache.org). +[issues-subscr...@systemml.apache.org](mailto:issues-subscr...@systemml.apache.org). To unsubscribe from the issues list, send an email to -[issues-unsubscr...@systemml.incubator.apache.org](mailto:issues-unsubscr...@systemml.incubator.apache.org). +[issues-unsubscr...@systemml.apache.org](mailto:issues-unsubscr...@systemml.apache.org). ### Commits Mailing List The commits mailing list archive can be found -[here](http://mail-archives.apache.org/mod_mbox/incubator-systemml-commits/). +[here](http://mail-archives.apache.org/mod_mbox/systemml-commits/). To subscribe to the issues list, send an email to -[commits-subscr...@systemml.incubator.apache.org](mailto:commits-subscr...@systemml.incubator.apache.org). +[commits-subscr...@systemml.apache.org](mailto:commits-subscr...@systemml.apache.org). To unsubscribe from the issues list, send an email to -[commits-unsubscr...@systemml.incubator.apache.org](mailto:commits-unsubscr...@systemml.incubator.apache.org). +[commits-unsubscr...@systemml.apache.org](mailto:commits-unsubscr...@systemml.a
incubator-systemml git commit: [SYSTEMML-1622] Remove incubator from email references
Repository: incubator-systemml Updated Branches: refs/heads/gh-pages de81e9bbf -> 8b4c2b7a8 [SYSTEMML-1622] Remove incubator from email references Updated email addresses to systemml.apache.org (removed incubator) in pom, md, readme files. Closes #514. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/8b4c2b7a Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/8b4c2b7a Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/8b4c2b7a Branch: refs/heads/gh-pages Commit: 8b4c2b7a843d33e509bdb16ccc3113be63ad2693 Parents: de81e9b Author: Glenn Weidner Authored: Thu May 25 13:45:31 2017 -0700 Committer: Glenn Weidner Committed: Thu May 25 13:45:31 2017 -0700 -- contributing-to-systemml.md | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/8b4c2b7a/contributing-to-systemml.md -- diff --git a/contributing-to-systemml.md b/contributing-to-systemml.md index 2b98dd0..5e36267 100644 --- a/contributing-to-systemml.md +++ b/contributing-to-systemml.md @@ -34,28 +34,28 @@ There are many ways to become involved with SystemML: ### Development Mailing List Perhaps the easiest way to obtain help and contribute to SystemML is to join the SystemML Development -mailing list (d...@systemml.incubator.apache.org). You can subscribe to this list by sending an email to -[dev-subscr...@systemml.incubator.apache.org](mailto:dev-subscr...@systemml.incubator.apache.org). -You can unsubscribe from this list by sending an email to [dev-unsubscr...@systemml.incubator.apache.org](mailto:dev-unsubscr...@systemml.incubator.apache.org). The dev mailing list archive can be found -[here](http://mail-archives.apache.org/mod_mbox/incubator-systemml-dev/). +mailing list (d...@systemml.apache.org). You can subscribe to this list by sending an email to +[dev-subscr...@systemml.apache.org](mailto:dev-subscr...@systemml.apache.org). +You can unsubscribe from this list by sending an email to [dev-unsubscr...@systemml.apache.org](mailto:dev-unsubscr...@systemml.apache.org). The dev mailing list archive can be found +[here](http://mail-archives.apache.org/mod_mbox/systemml-dev/). ### Issues Mailing List The issues mailing list archive can be found -[here](http://mail-archives.apache.org/mod_mbox/incubator-systemml-issues/). +[here](http://mail-archives.apache.org/mod_mbox/systemml-issues/). To subscribe to the issues list, send an email to -[issues-subscr...@systemml.incubator.apache.org](mailto:issues-subscr...@systemml.incubator.apache.org). +[issues-subscr...@systemml.apache.org](mailto:issues-subscr...@systemml.apache.org). To unsubscribe from the issues list, send an email to -[issues-unsubscr...@systemml.incubator.apache.org](mailto:issues-unsubscr...@systemml.incubator.apache.org). +[issues-unsubscr...@systemml.apache.org](mailto:issues-unsubscr...@systemml.apache.org). ### Commits Mailing List The commits mailing list archive can be found -[here](http://mail-archives.apache.org/mod_mbox/incubator-systemml-commits/). +[here](http://mail-archives.apache.org/mod_mbox/systemml-commits/). To subscribe to the issues list, send an email to -[commits-subscr...@systemml.incubator.apache.org](mailto:commits-subscr...@systemml.incubator.apache.org). +[commits-subscr...@systemml.apache.org](mailto:commits-subscr...@systemml.apache.org). To unsubscribe from the issues list, send an email to -[commits-unsubscr...@systemml.incubator.apache.org](mailto:commits-unsubscr...@systemml.incubator.apache.org). +[commits-unsubscr...@systemml.apache.org](mailto:commits-unsubscr...@systemml.apache.org). ## Issue Tracker
incubator-systemml-website git commit: [SYSTEMML-1622] Remove incubator from email references
Repository: incubator-systemml-website Updated Branches: refs/heads/master 65eb7dc27 -> 44355b0cc [SYSTEMML-1622] Remove incubator from email references Switched email addresses to systemml.apache.org (removed incubator) in community.html, index.html via changes to home.html, project.yml and _config.yml. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/44355b0c Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/44355b0c Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/44355b0c Branch: refs/heads/master Commit: 44355b0cc908381eead316f05c0023faf9100f16 Parents: 65eb7dc Author: Glenn Weidner Authored: Thu May 25 16:30:25 2017 -0700 Committer: Glenn Weidner Committed: Thu May 25 16:30:25 2017 -0700 -- _config.yml| 2 +- _src/_data/project.yml | 22 +++--- _src/_includes/themes/apache/home.html | 2 +- 3 files changed, 13 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/44355b0c/_config.yml -- diff --git a/_config.yml b/_config.yml index 647fdf5..40c8d51 100644 --- a/_config.yml +++ b/_config.yml @@ -8,7 +8,7 @@ # Site settings author: Apache SystemML title: Apache SystemML - Declarative Large-Scale Machine Learning -email: d...@systemml.incubator.apache.org +email: d...@systemml.apache.org description: Apache SystemML provides an optimal workplace for Machine Learning using big data baseurl: "" # the subpath of your site, e.g. /blog url: "http://systemml.apache.org"; # the base hostname & protocol for your site http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/44355b0c/_src/_data/project.yml -- diff --git a/_src/_data/project.yml b/_src/_data/project.yml index 14f5f62..b3d2b31 100644 --- a/_src/_data/project.yml +++ b/_src/_data/project.yml @@ -19,11 +19,11 @@ name: Apache SystemML™ short_name: SystemML™ description: Apache SystemML™ is a distributed and declarative machine learning platform. -dev_list: d...@systemml.incubator.apache.org -dev_list_subscribe: dev-subscr...@systemml.incubator.apache.org -dev_list_unsubscribe: dev-unsubscr...@systemml.incubator.apache.org +dev_list: d...@systemml.apache.org +dev_list_subscribe: dev-subscr...@systemml.apache.org +dev_list_unsubscribe: dev-unsubscr...@systemml.apache.org dev_list_archive: http://mail-archives.apache.org/mod_mbox/systemml-dev/ -dev_list_archive_mailarchive: https://www.mail-archive.com/dev@systemml.incubator.apache.org/ +dev_list_archive_mailarchive: https://www.mail-archive.com/dev@systemml.apache.org/ dev_list_archive_markmail: user_list: @@ -33,18 +33,18 @@ user_list_archive: user_list_archive_mailarchive: user_list_archive_markmail: -commits_list: comm...@systemml.incubator.apache.org -commits_list_subscribe: commits-subscr...@systemml.incubator.apache.org -commits_list_unsubscribe: commits-unsubscr...@systemml.incubator.apache.org +commits_list: commits@systemml.apache.org +commits_list_subscribe: commits-subscr...@systemml.apache.org +commits_list_unsubscribe: commits-unsubscr...@systemml.apache.org commits_list_archive: http://mail-archives.apache.org/mod_mbox/systemml-commits/ commits_list_archive_mailarchive: https://www.mail-archive.com/commits@systemml.apache.org/ commits_list_archive_markmail: -issues_list: iss...@systemml.incubator.apache.org -issues_list_subscribe: issues-subscr...@systemml.incubator.apache.org -issues_list_unsubscribe: issues-unsubscr...@systemml.incubator.apache.org +issues_list: iss...@systemml.apache.org +issues_list_subscribe: issues-subscr...@systemml.apache.org +issues_list_unsubscribe: issues-unsubscr...@systemml.apache.org issues_list_archive: http://mail-archives.apache.org/mod_mbox/systemml-issues/ -issues_list_archive_mailarchive: https://www.mail-archive.com/issues@systemml.incubator.apache.org/ +issues_list_archive_mailarchive: https://www.mail-archive.com/issues@systemml.apache.org/ issues_list_archive_markmail: jira: SYSTEMML http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/44355b0c/_src/_includes/themes/apache/home.html -- diff --git a/_src/_includes/themes/apache/home.html b/_src/_includes/themes/apache/home.html index c94623a..ba8a9e1 100644 --- a/_src/_includes/themes/apache/home.html +++ b/_src/_includes/themes/apache/home.html @@ -90,7 +90,7 @@ limitations under the License. mailto:{{ site.data.project.dev_list_subscribe }}?subject=send this email to subscribe" class=
svn commit: r1796228 - in /incubator/systemml/site: community.html index.html
Author: gweidner Date: Thu May 25 23:42:15 2017 New Revision: 1796228 URL: http://svn.apache.org/viewvc?rev=1796228&view=rev Log: [SYSTEMML-1622] Remove incubator from email references Modified: incubator/systemml/site/community.html incubator/systemml/site/index.html Modified: incubator/systemml/site/community.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/community.html?rev=1796228&r1=1796227&r2=1796228&view=diff == --- incubator/systemml/site/community.html (original) +++ incubator/systemml/site/community.html Thu May 25 23:42:15 2017 @@ -169,22 +169,22 @@ Get help using SystemMLmailto:d...@systemml.incubator.apache.org) is for people who want to contribute code to SystemML™. +[d...@systemml.apache.org](mailto:d...@systemml.apache.org) is for people who want to contribute code to SystemML™. -[subscribe](mailto:dev-subscr...@systemml.incubator.apache.org?subject=send this email to subscribe), -[unsubscribe](mailto:dev-unsubscr...@systemml.incubator.apache.org?subject=send this email to unsubscribe), -[archives](https://www.mail-archive.com/dev@systemml.incubator.apache.org/) +[subscribe](mailto:dev-subscr...@systemml.apache.org?subject=send this email to subscribe), +[unsubscribe](mailto:dev-unsubscr...@systemml.apache.org?subject=send this email to unsubscribe), +[archives](https://www.mail-archive.com/dev@systemml.apache.org/) -[comm...@systemml.incubator.apache.org](mailto:comm...@systemml.incubator.apache.org) is for commit messages and patches to SystemML™. +[commits@systemml.apache.org](mailto:commits@systemml.apache.org) is for commit messages and patches to SystemML™. -[subscribe](mailto:commits-subscr...@systemml.incubator.apache.org?subject=send this email to subscribe), -[unsubscribe](mailto:commits-unsubscr...@systemml.incubator.apache.org?subject=send this email to unsubscribe), +[subscribe](mailto:commits-subscr...@systemml.apache.org?subject=send this email to subscribe), +[unsubscribe](mailto:commits-unsubscr...@systemml.apache.org?subject=send this email to unsubscribe), [archives](https://www.mail-archive.com/commits@systemml.apache.org/) - [iss...@systemml.incubator.apache.org](mailto:iss...@systemml.incubator.apache.org) is for issues workflow notifications. [subscribe](mailto:issues-subscr...@systemml.incubator.apache.org?subject=send this email to subscribe), - [unsubscribe](mailto:issues-unsubscr...@systemml.incubator.apache.org?subject=send this email to unsubscribe), - [archives](https://www.mail-archive.com/issues@systemml.incubator.apache.org/) + [iss...@systemml.apache.org](mailto:iss...@systemml.apache.org) is for issues workflow notifications. [subscribe](mailto:issues-subscr...@systemml.apache.org?subject=send this email to subscribe), + [unsubscribe](mailto:issues-unsubscr...@systemml.apache.org?subject=send this email to unsubscribe), + [archives](https://www.mail-archive.com/issues@systemml.apache.org/) ### Issue tracker @@ -566,8 +566,8 @@ Before submitting an issue, please: Questions, feedback, and other SystemML related items are discussed on the dev mailing list. - mailto:dev-subscr...@systemml.incubator.apache.org?subject=send this email to subscribe" class="button button-primary">Subscribe - https://www.mail-archive.com/dev@systemml.incubator.apache.org/"; target="_blank" class="button button-secondary">View Archives + mailto:dev-subscr...@systemml.apache.org?subject=send this email to subscribe" class="button button-primary">Subscribe + https://www.mail-archive.com/dev@systemml.apache.org/"; target="_blank" class="button button-secondary">View Archives @@ -578,8 +578,8 @@ Before submitting an issue, please: JIRA notifications are sent to this list when issues are created, modified or pull requests are submitted. - mailto:issues-subscr...@systemml.incubator.apache.org?subject=send this email to subscribe" class="button button-primary">Subscribe - https://www.mail-archive.com/issues@systemml.incubator.apache.org/"; target="_blank" class="button button-secondary">View Archives + mailto:issues-subscr...@systemml.apache.org?subject=send this email to subscribe" class="button button-primary">Subscribe + https://www.mail-archive.com/issues@systemml.apache.org/"; target="_blank" class="button button-secondary">View Archives @@ -590,7 +590,7 @@ Before submitting an issue, please: Notifications with code diffs are sent to this list when changes are committed to the SystemML source tree. - mailto:commits-subscr...@systemml.incubator.apache.org?subject=send this email to subscribe" class=&qu
incubator-systemml-website git commit: [SYSTEMML-1618] Reword signature (.asc) and KEYS usage
Repository: incubator-systemml-website Updated Branches: refs/heads/master 44355b0cc -> d3831e667 [SYSTEMML-1618] Reword signature (.asc) and KEYS usage Corrected verification description on dowload page since hash file (.md5) independent of KEYS as mentioned in JIRA. Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/d3831e66 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/d3831e66 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/d3831e66 Branch: refs/heads/master Commit: d3831e667c3f98bc65ee2df7a0684569994a7ce4 Parents: 44355b0 Author: Glenn Weidner Authored: Fri May 26 16:47:31 2017 -0700 Committer: Glenn Weidner Committed: Fri May 26 16:47:31 2017 -0700 -- _src/download.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/d3831e66/_src/download.html -- diff --git a/_src/download.html b/_src/download.html index 52734cc..3cb71b0 100644 --- a/_src/download.html +++ b/_src/download.html @@ -84,8 +84,8 @@ limitations under the License. Instructions for checking hashes and signatures is described on the http://www.apache.org/info/verification.html"; target="_blank">Verifying Apache Software Foundation Releases page. - These https://www.apache.org/dist/incubator/systemml/KEYS"; target="_blank">KEYS can be used to http://www.apache.org/dyn/closer.cgi#verify"; target="_blank">verify the integrity of the downloaded artifact using hash (.md5) or PGP signature (.asc). - + These https://www.apache.org/dist/incubator/systemml/KEYS"; target="_blank">KEYS can be used to http://www.apache.org/dyn/closer.cgi#verify"; target="_blank">verify the integrity of the downloaded artifact using PGP signature (.asc). + Bleeding-Edge
svn commit: r1796361 - /incubator/systemml/site/download.html
Author: gweidner Date: Fri May 26 23:58:29 2017 New Revision: 1796361 URL: http://svn.apache.org/viewvc?rev=1796361&view=rev Log: [SYSTEMML-1618] Reword signature (.asc) and KEYS usage Modified: incubator/systemml/site/download.html Modified: incubator/systemml/site/download.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/download.html?rev=1796361&r1=1796360&r2=1796361&view=diff == --- incubator/systemml/site/download.html (original) +++ incubator/systemml/site/download.html Fri May 26 23:58:29 2017 @@ -208,8 +208,8 @@ Instructions for checking hashes and signatures is described on the http://www.apache.org/info/verification.html"; target="_blank">Verifying Apache Software Foundation Releases page. - These https://www.apache.org/dist/incubator/systemml/KEYS"; target="_blank">KEYS can be used to http://www.apache.org/dyn/closer.cgi#verify"; target="_blank">verify the integrity of the downloaded artifact using hash (.md5) or PGP signature (.asc). - + These https://www.apache.org/dist/incubator/systemml/KEYS"; target="_blank">KEYS can be used to http://www.apache.org/dyn/closer.cgi#verify"; target="_blank">verify the integrity of the downloaded artifact using PGP signature (.asc). + Bleeding-Edge
incubator-systemml-website git commit: [MINOR] Remove best paper award from home page
Repository: incubator-systemml-website Updated Branches: refs/heads/master d3831e667 -> b5388549a [MINOR] Remove best paper award from home page Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/b5388549 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/b5388549 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/b5388549 Branch: refs/heads/master Commit: b5388549a3b96b4c44092260dcfef2e41b9a6ae5 Parents: d3831e6 Author: Glenn Weidner Authored: Sun May 28 22:15:06 2017 -0700 Committer: Glenn Weidner Committed: Sun May 28 22:15:06 2017 -0700 -- _src/_includes/themes/apache/home.html | 13 - 1 file changed, 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/b5388549/_src/_includes/themes/apache/home.html -- diff --git a/_src/_includes/themes/apache/home.html b/_src/_includes/themes/apache/home.html index ba8a9e1..4298006 100644 --- a/_src/_includes/themes/apache/home.html +++ b/_src/_includes/themes/apache/home.html @@ -34,19 +34,6 @@ limitations under the License. - - - - - - - - VLDB 2016 Best Paper - Read Compressed Linear Algebra for Large-Scale Machine Learning.http://www.vldb.org/pvldb/vol9/p960-elgohary.pdf"; target="_blank"> Download Paper - - - -
svn commit: r1796574 - /incubator/systemml/site/index.html
Author: gweidner Date: Mon May 29 05:20:03 2017 New Revision: 1796574 URL: http://svn.apache.org/viewvc?rev=1796574&view=rev Log: [MINOR] Remove best paper award from home index Modified: incubator/systemml/site/index.html Modified: incubator/systemml/site/index.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/index.html?rev=1796574&r1=1796573&r2=1796574&view=diff == --- incubator/systemml/site/index.html (original) +++ incubator/systemml/site/index.html Mon May 29 05:20:03 2017 @@ -160,19 +160,6 @@ - - - - - - - - VLDB 2016 Best Paper - Read Compressed Linear Algebra for Large-Scale Machine Learning.http://www.vldb.org/pvldb/vol9/p960-elgohary.pdf"; target="_blank"> Download Paper - - - -
incubator-systemml-website git commit: [MINOR] Restore best paper section
Repository: incubator-systemml-website Updated Branches: refs/heads/master b5388549a -> 0e699197b [MINOR] Restore best paper section Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/commit/0e699197 Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/tree/0e699197 Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/diff/0e699197 Branch: refs/heads/master Commit: 0e699197b0b666302c322ebf81b06bb4a7faa0dc Parents: b538854 Author: Glenn Weidner Authored: Tue May 30 11:09:37 2017 -0700 Committer: Glenn Weidner Committed: Tue May 30 11:09:37 2017 -0700 -- _src/_includes/themes/apache/home.html | 13 + 1 file changed, 13 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-systemml-website/blob/0e699197/_src/_includes/themes/apache/home.html -- diff --git a/_src/_includes/themes/apache/home.html b/_src/_includes/themes/apache/home.html index 4298006..ba8a9e1 100644 --- a/_src/_includes/themes/apache/home.html +++ b/_src/_includes/themes/apache/home.html @@ -34,6 +34,19 @@ limitations under the License. + + + + + + + + VLDB 2016 Best Paper + Read Compressed Linear Algebra for Large-Scale Machine Learning.http://www.vldb.org/pvldb/vol9/p960-elgohary.pdf"; target="_blank"> Download Paper + + + +
svn commit: r1796909 - /incubator/systemml/site/index.html
Author: gweidner Date: Tue May 30 18:15:58 2017 New Revision: 1796909 URL: http://svn.apache.org/viewvc?rev=1796909&view=rev Log: [MINOR] Restore best paper section Modified: incubator/systemml/site/index.html Modified: incubator/systemml/site/index.html URL: http://svn.apache.org/viewvc/incubator/systemml/site/index.html?rev=1796909&r1=1796908&r2=1796909&view=diff == --- incubator/systemml/site/index.html (original) +++ incubator/systemml/site/index.html Tue May 30 18:15:58 2017 @@ -160,6 +160,19 @@ + + + + + + + + VLDB 2016 Best Paper + Read Compressed Linear Algebra for Large-Scale Machine Learning.http://www.vldb.org/pvldb/vol9/p960-elgohary.pdf"; target="_blank"> Download Paper + + + +
systemml git commit: [SYSTEMML-1643] Add assembly for building extra jar under distribution profile
Repository: systemml Updated Branches: refs/heads/master 762709a87 -> 46fa64280 [SYSTEMML-1643] Add assembly for building extra jar under distribution profile Added assembly for building extra jar under distribution profile only and created separate license file containing Caffe and TensorFlow information. Closes #538. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/46fa6428 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/46fa6428 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/46fa6428 Branch: refs/heads/master Commit: 46fa642802d4071a30c32b5b5a1b3557ed870790 Parents: 762709a Author: Glenn Weidner Authored: Tue Jun 13 09:06:11 2017 -0700 Committer: Glenn Weidner Committed: Tue Jun 13 09:06:12 2017 -0700 -- pom.xml| 42 ++-- src/assembly/extra.xml | 53 + src/assembly/extra/LICENSE | 462 src/assembly/extra/NOTICE | 5 + 4 files changed, 540 insertions(+), 22 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/46fa6428/pom.xml -- diff --git a/pom.xml b/pom.xml index 5b914d5..bde1215 100644 --- a/pom.xml +++ b/pom.xml @@ -277,28 +277,6 @@ - - extra - jar - package - - - true - - ${maven.build.timestamp} - org.apache.sysml.api.DMLScript - - - extra - - **/META-INF/* - **/caffe/* - **/org/tensorflow/framework/* - **/org/tensorflow/util/* - **/org/apache/sysml/api/dl/* - - - @@ -952,6 +930,26 @@ + + create-extra-jar + package + + single + + + + src/assembly/extra.xml + + + + ${maven.build.timestamp} + ${project.artifactId}-extra + ${project.version} + + + extra + + http://git-wip-us.apache.org/repos/asf/systemml/blob/46fa6428/src/assembly/extra.xml -- diff --git a/src/assembly/extra.xml b/src/assembly/extra.xml new file mode 100644 index 000..301aaaf --- /dev/null +++ b/src/assembly/extra.xml @@ -0,0 +1,53 @@ + +http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/plugins/m
systemml git commit: [SYSTEMML-1728] Reshape util to convert tensors in NCHW to CNHW format
Repository: systemml Updated Branches: refs/heads/master f516e4bdc -> 345682404 [SYSTEMML-1728] Reshape util to convert tensors in NCHW to CNHW format Closes #552. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/34568240 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/34568240 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/34568240 Branch: refs/heads/master Commit: 345682404c3fb1348484c375e811ee3f5805a691 Parents: f516e4b Author: prithvirajsen Authored: Thu Jun 22 11:05:24 2017 -0700 Committer: Glenn Weidner Committed: Thu Jun 22 11:05:25 2017 -0700 -- scripts/nn/test/run_tests.dml | 1 + scripts/nn/test/test.dml | 33 scripts/nn/util.dml | 52 ++ 3 files changed, 86 insertions(+) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/34568240/scripts/nn/test/run_tests.dml -- diff --git a/scripts/nn/test/run_tests.dml b/scripts/nn/test/run_tests.dml index 5f3ca6e..cca0d0d 100644 --- a/scripts/nn/test/run_tests.dml +++ b/scripts/nn/test/run_tests.dml @@ -97,6 +97,7 @@ test::max_pool2d() test::padding() test::tanh() test::threshold() +test::transpose_NCHW_to_CNHW() print("---") print("Other tests complete -- look for any ERRORs or WARNINGs.") http://git-wip-us.apache.org/repos/asf/systemml/blob/34568240/scripts/nn/test/test.dml -- diff --git a/scripts/nn/test/test.dml b/scripts/nn/test/test.dml index 37f9f73..b0899e3 100644 --- a/scripts/nn/test/test.dml +++ b/scripts/nn/test/test.dml @@ -488,6 +488,39 @@ padding = function() { } } +transpose_NCHW_to_CNHW = function() { + /* + * Test for `transpose_NCHW_to_CNHW` function. + */ + print("Testing transpose_NCHW_to_CNHW function.") + + # Generate data + N = 2 + C = 3 + H = 4 + W = 5 + X = matrix(seq(1, N*C*H*W), rows=N, cols=C*H*W) + + out = util::transpose_NCHW_to_CNHW(X, C) + + target = +matrix("1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 +61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 +21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 +81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 +41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 +101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120", + rows=C, cols=N*H*W) + + # Equivalency check + for (i in 1:nrow(out)) { +for(j in 1:ncol(out)) { + rel_error = test_util::check_rel_error(as.scalar(out[i,j]), + as.scalar(target[i,j]), 1e-10, 1e-12) +} + } +} + max_pool2d = function() { /* * Test for the 2D max pooling functions. http://git-wip-us.apache.org/repos/asf/systemml/blob/34568240/scripts/nn/util.dml -- diff --git a/scripts/nn/util.dml b/scripts/nn/util.dml index c4da16a..329f22f 100644 --- a/scripts/nn/util.dml +++ b/scripts/nn/util.dml @@ -216,3 +216,55 @@ threshold = function(matrix[double] X, double thresh) out = X > thresh } +/* + * Reshape util for tensors in NCHW format. + * Transposes the 1st and 2nd dimensions. + */ +transpose_NCHW_to_CNHW = function(matrix[double] X, int C) return (matrix[double] out){ + /* + * Inputs: + * - X: Input with N rows and channels flattened within each row in + * channel-major format (NCHW). + * - C: Number of channels (dimensionality of depth). + * + * Outputs: + * - out: Transposed output with C rows. + */ + N = nrow(X) + D = ncol(X) / C + + /* + * This is an easy reshape because the channels remain intact. By + * reshaping X to a matrix with N*C rows, we can reduce our task to + * re-ordering rows (followed by the obvious reshape to achieve the + * required output shape with C rows). + * + * The difficult part is to obtain the permutation matrix required + * for re-ordering the rows. In this case, since we want to bring the + * ith channels from all rows together, we will need a column vector + * of the following form: + * [1, 1+C, 1+2C, ..., 1+(N-1)C, + * 2, 2+C, ..., 2+(N-1)C, + * 3, 3+C, ..., 3+(N-1)C, + * . + * . + * . + * C, 2C, ..., NC]' + * This vector can be produced via an outer call. + */ + col_idx = outer(seq(1,C), C*t(seq(0,N-1)), "+") + + /* + * Generate the permutation matrix by: + * - reshaping the result of outer into a col + * - invoking table + */ + permut = table(seq(1, N*C), matrix(co
systemml-website git commit: [MINOR] Add tutorial page for KDD 2017 conference
Repository: systemml-website Updated Branches: refs/heads/master 084af4df6 -> 8509414c1 [MINOR] Add tutorial page for KDD 2017 conference Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/8509414c Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/8509414c Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/8509414c Branch: refs/heads/master Commit: 8509414c1be4bbecc65041769b30f69a134380d0 Parents: 084af4d Author: Glenn Weidner Authored: Thu Jun 29 21:29:00 2017 -0700 Committer: Glenn Weidner Committed: Thu Jun 29 21:29:01 2017 -0700 -- _src/get-started.html | 2 +- _src/tutorial-kdd2017.html | 50 + _src/tutorials.html| 15 - 3 files changed, 60 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/8509414c/_src/get-started.html -- diff --git a/_src/get-started.html b/_src/get-started.html index 00f047d..17a64e3 100644 --- a/_src/get-started.html +++ b/_src/get-started.html @@ -42,7 +42,7 @@ limitations under the License. Install SystemML Level: Beginner | Time: 20 minutes New to Apache SystemML? Try our quick install guide that will walk you through setting up your environment and getting you up and going with SystemML. - https://apache.github.io/systemml"; target="_blank">Docs + Tutorials Install SystemML http://git-wip-us.apache.org/repos/asf/systemml-website/blob/8509414c/_src/tutorial-kdd2017.html -- diff --git a/_src/tutorial-kdd2017.html b/_src/tutorial-kdd2017.html new file mode 100644 index 000..56c4494 --- /dev/null +++ b/_src/tutorial-kdd2017.html @@ -0,0 +1,50 @@ +--- +layout: page +title: KDD 2017 Tutorial +description: KDD 2017 Tutorial Page +group: nav-right +--- + + + + + + + {{ site.data.project.name }} KDD 2017 Tutorial + + + + + + + + + + + + + + +Tutorial Coming Soon. + + + + + http://git-wip-us.apache.org/repos/asf/systemml-website/blob/8509414c/_src/tutorials.html -- diff --git a/_src/tutorials.html b/_src/tutorials.html index 866a69f..998abdc 100644 --- a/_src/tutorials.html +++ b/_src/tutorials.html @@ -28,7 +28,7 @@ limitations under the License. {{ page.title }} - Learn SystemML in simple and easy steps. + Learn Apache SystemML. @@ -39,18 +39,18 @@ limitations under the License. - + -Get Started -New to Apache SystemML? Try our Get Started tutorial that will walk you through setting up your environment and getting you up and going with SystemML. -Begin Tutorial -https://apache.github.io/systemml/";>Docs +KDD 2017 +Coming Soon. +KDD 2017 Tutorial + +
svn commit: r1800356 - in /systemml/site: get-started.html tutorial-kdd2017.html tutorials.html
Author: gweidner Date: Fri Jun 30 04:48:46 2017 New Revision: 1800356 URL: http://svn.apache.org/viewvc?rev=1800356&view=rev Log: Add tutorial page for KDD 2017 conference Added: systemml/site/tutorial-kdd2017.html Modified: systemml/site/get-started.html systemml/site/tutorials.html Modified: systemml/site/get-started.html URL: http://svn.apache.org/viewvc/systemml/site/get-started.html?rev=1800356&r1=1800355&r2=1800356&view=diff == --- systemml/site/get-started.html (original) +++ systemml/site/get-started.html Fri Jun 30 04:48:46 2017 @@ -166,7 +166,7 @@ Install SystemML Level: Beginner | Time: 20 minutes New to Apache SystemML? Try our quick install guide that will walk you through setting up your environment and getting you up and going with SystemML. - https://apache.github.io/systemml"; target="_blank">Docs + Tutorials Install SystemML Added: systemml/site/tutorial-kdd2017.html URL: http://svn.apache.org/viewvc/systemml/site/tutorial-kdd2017.html?rev=1800356&view=auto == --- systemml/site/tutorial-kdd2017.html (added) +++ systemml/site/tutorial-kdd2017.html Fri Jun 30 04:48:46 2017 @@ -0,0 +1,211 @@ + + + + + + + + + + KDD 2017 Tutorial + + + + + + + + + + + https://systemml.apache.org/"; /> + + + + + + + + + + + + + + + + + + + + Apache SystemML™ + + + + + + +Download + + + + + + +Get Started + + + + + + +Docs + + + + + + +Roadmap + + + + + +Community + + + + Get Involved + + + https://issues.apache.org/jira/browse/SYSTEMML"; target="_blank">Issue Tracker + + + https://github.com/apache/systemml"; target="_blank">Source Code + + + https://github.com/apache/systemml-website"; target="_blank">Website Source Code + + + Roadmap + + + + + + + +Apache + + + + http://www.apache.org/foundation/how-it-works.html"; target="_blank">Apache Software Foundation + + + http://www.apache.org/licenses/"; target="_blank">Apache License + + + http://www.apache.org/foundation/sponsorship"; target="_blank">Sponsorship + + + http://www.apache.org/foundation/thanks.html"; target="_blank">Thanks + + + Privacy Policy + + + Security + + + + + + + + + + + + + + + + + + + Apache SystemML™ KDD 2017 Tutorial + + + + + + + + + + + + + + +Tutorial Coming Soon. + + + + + + + + + + + + + + +http://www.apache.org";> + + + Copyright © 2015-2017 http://www.apache.org";>The Apache Software Foundation. + Licensed under the http://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0. + + Apache and the Apache Feather logo are trademarks of The Apache Software Foundation. + + + + + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); + + ga('create', 'UA-71553733-1', 'auto'); + ga('send', 'pageview'); + + + + + + + + Modified: systemml/site/tutorials.html URL: http://svn.apache.org/viewvc/systemml/site/tutorials.html?rev=1800356&r1=1800355&r2=1800356&view=diff == --- systemml/site/tutorials.html (original) +++ systemml/site/tutorials.html Fri Jun 30 04:48:46 2017 @@ -152,7 +152,7 @@ Tutorials - Learn SystemML in simple and easy steps. + Learn Apache SystemML. @@ -163,18 +163,18 @@ - + -Get Started -New to Apache SystemML? Try our Get Started tutorial that will walk you through setting up your environment and getting you up and going with SystemML. -Begin Tutorial -https://apache.github.io/systemml/";>Docs +KDD 2017 +Coming Soon. +KDD 2017 Tutorial + +
svn commit: r1800441 - /systemml/site/assets/css/main.css
Author: gweidner Date: Fri Jun 30 16:31:17 2017 New Revision: 1800441 URL: http://svn.apache.org/viewvc?rev=1800441&view=rev Log: Include main.css updated from gulp Modified: systemml/site/assets/css/main.css Modified: systemml/site/assets/css/main.css URL: http://svn.apache.org/viewvc/systemml/site/assets/css/main.css?rev=1800441&r1=1800440&r2=1800441&view=diff == --- systemml/site/assets/css/main.css (original) +++ systemml/site/assets/css/main.css Fri Jun 30 16:31:17 2017 @@ -5807,4 +5807,4 @@ ul.ml-tabs { top: 50px; right: 50px; } [... 3 lines stripped ...]
systemml-website git commit: Match case for image file on KDD 2017 tutorial page
Repository: systemml-website Updated Branches: refs/heads/master 8509414c1 -> e9e7be648 Match case for image file on KDD 2017 tutorial page Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/e9e7be64 Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/e9e7be64 Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/e9e7be64 Branch: refs/heads/master Commit: e9e7be6485a0db8775ccfcfa0efb7b95dbc0107b Parents: 8509414 Author: Glenn Weidner Authored: Fri Jun 30 09:53:09 2017 -0700 Committer: Glenn Weidner Committed: Fri Jun 30 09:53:09 2017 -0700 -- _src/tutorial-kdd2017.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/e9e7be64/_src/tutorial-kdd2017.html -- diff --git a/_src/tutorial-kdd2017.html b/_src/tutorial-kdd2017.html index 56c4494..5f77492 100644 --- a/_src/tutorial-kdd2017.html +++ b/_src/tutorial-kdd2017.html @@ -38,7 +38,7 @@ limitations under the License. - +
svn commit: r1800442 - /systemml/site/tutorial-kdd2017.html
Author: gweidner Date: Fri Jun 30 16:58:56 2017 New Revision: 1800442 URL: http://svn.apache.org/viewvc?rev=1800442&view=rev Log: Match case for image file on KDD 2017 tutorial page Modified: systemml/site/tutorial-kdd2017.html Modified: systemml/site/tutorial-kdd2017.html URL: http://svn.apache.org/viewvc/systemml/site/tutorial-kdd2017.html?rev=1800442&r1=1800441&r2=1800442&view=diff == --- systemml/site/tutorial-kdd2017.html (original) +++ systemml/site/tutorial-kdd2017.html Fri Jun 30 16:58:56 2017 @@ -162,7 +162,7 @@ - +
systemml git commit: [SYSTEMML-1712] Minor corrections to algorithms demo notebook
Repository: systemml Updated Branches: refs/heads/master 66b28c6e3 -> c5a330d7d [SYSTEMML-1712] Minor corrections to algorithms demo notebook Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/c5a330d7 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/c5a330d7 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/c5a330d7 Branch: refs/heads/master Commit: c5a330d7d6ec296bd7dfb8f385ebf3614abcecf1 Parents: 66b28c6 Author: Glenn Weidner Authored: Fri Jul 7 09:05:26 2017 -0700 Committer: Glenn Weidner Committed: Fri Jul 7 09:05:26 2017 -0700 -- .../jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/c5a330d7/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb -- diff --git a/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb b/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb index 93558d2..681b277 100644 --- a/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb +++ b/samples/jupyter-notebooks/Linear_Regression_Algorithms_Demo.ipynb @@ -12,7 +12,7 @@ " - SystemML 'Hello World'\n", "- Example 1: Matrix Multiplication\n", " - SystemML script to generate a random matrix, perform matrix multiplication, and compute the sum of the output\n", -" - Examine execution plans, and increase data size to obverve changed execution plans\n", +" - Examine execution plans, and increase data size to observe changed execution plans\n", "- Load diabetes dataset from scikit-learn\n", "- Example 2: Implement three different algorithms to train linear regression model\n", " - Algorithm 1: Linear Regression - Direct Solve (no regularization)\n", @@ -145,7 +145,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ -"### Examine execution plans, and increase data size to obverve changed execution plans" +"### Examine execution plans, and increase data size to observe changed execution plans" ] }, { @@ -406,7 +406,7 @@ " Step 2: Compute gradient dw.\n", " Step 3: Compute stepsize alpha.\n", " Step 4: Compute next direction p by enforcing conjugacy with previous direction.\n", -" Step 4: Update: w_new = w_old + alpha*p\n", +" Step 5: Update: w_new = w_old + alpha*p\n", "}`\n", "\n", "![Gradient Descent vs Conjugate Gradient](http://i.stack.imgur.com/zh1HH.png)\n"
systemml git commit: [SYSTEMML-1820] Add isStatistics, isExplain, close methods to Python MLContext
Repository: systemml Updated Branches: refs/heads/master e94374afb -> 2942f89b3 [SYSTEMML-1820] Add isStatistics, isExplain, close methods to Python MLContext Closes #600. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/2942f89b Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/2942f89b Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/2942f89b Branch: refs/heads/master Commit: 2942f89b3573443e588e9fbfb1e33e4cd4658835 Parents: e94374a Author: Glenn Weidner Authored: Tue Aug 1 14:23:41 2017 -0700 Committer: Glenn Weidner Committed: Tue Aug 1 14:23:41 2017 -0700 -- src/main/python/systemml/mlcontext.py | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/2942f89b/src/main/python/systemml/mlcontext.py -- diff --git a/src/main/python/systemml/mlcontext.py b/src/main/python/systemml/mlcontext.py index 304ea26..1e79648 100644 --- a/src/main/python/systemml/mlcontext.py +++ b/src/main/python/systemml/mlcontext.py @@ -592,8 +592,8 @@ class MLContext(object): Parameters -- -sc: SparkContext -SparkContext +sc: SparkContext or SparkSession +An instance of pyspark.SparkContext or pyspark.sql.SparkSession. """ def __init__(self, sc): if isinstance(sc, pyspark.sql.session.SparkSession): @@ -718,3 +718,19 @@ class MLContext(object): def info(self): """Display the project information.""" return self._ml.info().toString() + +def isExplain(self): +"""Returns True if program instruction details should be output, False otherwise.""" +return self._ml.isExplain() + +def isStatistics(self): +"""Returns True if program execution statistics should be output, False otherwise.""" +return self._ml.isStatistics() + +def close(self): +""" +Closes this MLContext instance to cleanup buffer pool, static/local state and scratch space. +Note the SparkContext is not explicitly closed to allow external reuse. +""" +self._ml.close() +return self
systemml git commit: [MINOR] Ensure jar loaded for jvm_stdout
Repository: systemml Updated Branches: refs/heads/master 70e5f29e8 -> 98a9d653d [MINOR] Ensure jar loaded for jvm_stdout Use _get_spark_context() instead of SparkContext._active_spark_context. Closes #606. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/98a9d653 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/98a9d653 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/98a9d653 Branch: refs/heads/master Commit: 98a9d653de5aa9670b084d6480921fe98ce01861 Parents: 70e5f29 Author: Glenn Weidner Authored: Mon Aug 7 14:11:01 2017 -0700 Committer: Glenn Weidner Committed: Mon Aug 7 14:11:01 2017 -0700 -- src/main/python/systemml/mlcontext.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/98a9d653/src/main/python/systemml/mlcontext.py -- diff --git a/src/main/python/systemml/mlcontext.py b/src/main/python/systemml/mlcontext.py index 4838f98..8ab683a 100644 --- a/src/main/python/systemml/mlcontext.py +++ b/src/main/python/systemml/mlcontext.py @@ -43,7 +43,8 @@ import threading, time _loadedSystemML = False def _get_spark_context(): """ -Internal method to get already initialized SparkContext. +Internal method to get already initialized SparkContext. Developers should always use +_get_spark_context() instead of SparkContext._active_spark_context to ensure SystemML loaded. Returns --- @@ -74,7 +75,7 @@ class jvm_stdout(object): Should flush the stdout in parallel """ def __init__(self, parallel_flush=False): -self.util = SparkContext._active_spark_context._jvm.org.apache.sysml.api.ml.Utils() +self.util = _get_spark_context()._jvm.org.apache.sysml.api.ml.Utils() self.parallel_flush = parallel_flush self.t = threading.Thread(target=self.flush_stdout) self.stop = False
systemml git commit: [SYSTEMML-1829] Add setConfig, resetConfig to Python MLContext
Repository: systemml Updated Branches: refs/heads/master 815ca4f2a -> e1a762f65 [SYSTEMML-1829] Add setConfig, resetConfig to Python MLContext Added methods setConfig, resetConfig, isGPU, isForceGPU to Python MLContext for consistency with Java MLContext. Closes #608. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/e1a762f6 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/e1a762f6 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/e1a762f6 Branch: refs/heads/master Commit: e1a762f654a945a8cc36ffeb6ac665ac308a8204 Parents: 815ca4f Author: Glenn Weidner Authored: Tue Aug 8 13:44:58 2017 -0700 Committer: Glenn Weidner Committed: Tue Aug 8 13:44:58 2017 -0700 -- src/main/python/systemml/mlcontext.py | 28 +++- 1 file changed, 27 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/e1a762f6/src/main/python/systemml/mlcontext.py -- diff --git a/src/main/python/systemml/mlcontext.py b/src/main/python/systemml/mlcontext.py index 8ab683a..5841ab5 100644 --- a/src/main/python/systemml/mlcontext.py +++ b/src/main/python/systemml/mlcontext.py @@ -798,7 +798,25 @@ class MLContext(object): """ self._ml.setConfigProperty(propertyName, propertyValue) return self - + +def setConfig(self, configFilePath): +""" +Set SystemML configuration based on a configuration file. + +Parameters +-- +configFilePath: String +""" +self._ml.setConfig(configFilePath) +return self + +def resetConfig(self): +""" +Reset configuration settings to default values. +""" +self._ml.resetConfig() +return self + def version(self): """Display the project version.""" return self._ml.version() @@ -819,6 +837,14 @@ class MLContext(object): """Returns True if program execution statistics should be output, False otherwise.""" return self._ml.isStatistics() +def isGPU(self): +"""Returns True if GPU mode is enabled, False otherwise.""" +return self._ml.isGPU() + +def isForceGPU(self): +"""Returns True if "force" GPU mode is enabled, False otherwise.""" +return self._ml.isForceGPU() + def close(self): """ Closes this MLContext instance to cleanup buffer pool, static/local state and scratch space.
systemml-website git commit: Add download button for KDD tutorial
Repository: systemml-website Updated Branches: refs/heads/master e9e7be648 -> ea8e21aaf Add download button for KDD tutorial Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/ea8e21aa Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/ea8e21aa Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/ea8e21aa Branch: refs/heads/master Commit: ea8e21aaf79c257134f4ca54e1b0baac34b29cfe Parents: e9e7be6 Author: Glenn Weidner Authored: Thu Aug 10 12:11:32 2017 -0700 Committer: Glenn Weidner Committed: Thu Aug 10 12:11:32 2017 -0700 -- _src/get-started.html | 2 +- _src/tutorial-kdd2017.html | 6 +++--- _src/tutorials.html | 7 +++ talks/systemml-kdd-tutorial.zip | Bin 0 -> 237 bytes 4 files changed, 7 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/ea8e21aa/_src/get-started.html -- diff --git a/_src/get-started.html b/_src/get-started.html index 17a64e3..25ff087 100644 --- a/_src/get-started.html +++ b/_src/get-started.html @@ -42,7 +42,7 @@ limitations under the License. Install SystemML Level: Beginner | Time: 20 minutes New to Apache SystemML? Try our quick install guide that will walk you through setting up your environment and getting you up and going with SystemML. - Tutorials + Tutorials Install SystemML http://git-wip-us.apache.org/repos/asf/systemml-website/blob/ea8e21aa/_src/tutorial-kdd2017.html -- diff --git a/_src/tutorial-kdd2017.html b/_src/tutorial-kdd2017.html index 5f77492..6a25d6b 100644 --- a/_src/tutorial-kdd2017.html +++ b/_src/tutorial-kdd2017.html @@ -38,11 +38,11 @@ limitations under the License. - + - -Tutorial Coming Soon. +Declarative, Large-Scale Machine Learning with Apache SystemML +https://github.com/apache/systemml-website/raw/master/talks/systemml-kdd-tutorial.zip";>Download Hands-On Tutorial http://git-wip-us.apache.org/repos/asf/systemml-website/blob/ea8e21aa/_src/tutorials.html -- diff --git a/_src/tutorials.html b/_src/tutorials.html index 998abdc..6e19da5 100644 --- a/_src/tutorials.html +++ b/_src/tutorials.html @@ -39,12 +39,11 @@ limitations under the License. - + -KDD 2017 -Coming Soon. -KDD 2017 Tutorial +Declarative, Large-Scale Machine Learning with Apache SystemML +KDD 2017 Tutorial http://git-wip-us.apache.org/repos/asf/systemml-website/blob/ea8e21aa/talks/systemml-kdd-tutorial.zip -- diff --git a/talks/systemml-kdd-tutorial.zip b/talks/systemml-kdd-tutorial.zip new file mode 100644 index 000..59906b5 Binary files /dev/null and b/talks/systemml-kdd-tutorial.zip differ
svn commit: r1804713 - in /systemml/site: get-started.html tutorial-kdd2017.html tutorials.html
Author: gweidner Date: Thu Aug 10 19:42:18 2017 New Revision: 1804713 URL: http://svn.apache.org/viewvc?rev=1804713&view=rev Log: Add download button for KDD tutorial Modified: systemml/site/get-started.html systemml/site/tutorial-kdd2017.html systemml/site/tutorials.html Modified: systemml/site/get-started.html URL: http://svn.apache.org/viewvc/systemml/site/get-started.html?rev=1804713&r1=1804712&r2=1804713&view=diff == --- systemml/site/get-started.html (original) +++ systemml/site/get-started.html Thu Aug 10 19:42:18 2017 @@ -166,7 +166,7 @@ Install SystemML Level: Beginner | Time: 20 minutes New to Apache SystemML? Try our quick install guide that will walk you through setting up your environment and getting you up and going with SystemML. - Tutorials + Tutorials Install SystemML Modified: systemml/site/tutorial-kdd2017.html URL: http://svn.apache.org/viewvc/systemml/site/tutorial-kdd2017.html?rev=1804713&r1=1804712&r2=1804713&view=diff == --- systemml/site/tutorial-kdd2017.html (original) +++ systemml/site/tutorial-kdd2017.html Thu Aug 10 19:42:18 2017 @@ -162,11 +162,11 @@ - + - -Tutorial Coming Soon. +Declarative, Large-Scale Machine Learning with Apache SystemML +https://github.com/apache/systemml-website/raw/master/talks/systemml-kdd-tutorial.zip";>Download Hands-On Tutorial Modified: systemml/site/tutorials.html URL: http://svn.apache.org/viewvc/systemml/site/tutorials.html?rev=1804713&r1=1804712&r2=1804713&view=diff == --- systemml/site/tutorials.html (original) +++ systemml/site/tutorials.html Thu Aug 10 19:42:18 2017 @@ -163,12 +163,11 @@ - + -KDD 2017 -Coming Soon. -KDD 2017 Tutorial +Declarative, Large-Scale Machine Learning with Apache SystemML +KDD 2017 Tutorial
systemml-website git commit: Switch KDD Tutorial download link
Repository: systemml-website Updated Branches: refs/heads/master ea8e21aaf -> 4620131c3 Switch KDD Tutorial download link Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/4620131c Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/4620131c Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/4620131c Branch: refs/heads/master Commit: 4620131c3b69c2ebd1f2f79d02485e54e8869864 Parents: ea8e21a Author: Glenn Weidner Authored: Mon Aug 14 17:02:39 2017 -0700 Committer: Glenn Weidner Committed: Mon Aug 14 17:02:39 2017 -0700 -- _src/tutorial-kdd2017.html | 2 +- talks/systemml-kdd-tutorial.zip | Bin 237 -> 0 bytes 2 files changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/4620131c/_src/tutorial-kdd2017.html -- diff --git a/_src/tutorial-kdd2017.html b/_src/tutorial-kdd2017.html index 6a25d6b..ae84f46 100644 --- a/_src/tutorial-kdd2017.html +++ b/_src/tutorial-kdd2017.html @@ -42,7 +42,7 @@ limitations under the License. Declarative, Large-Scale Machine Learning with Apache SystemML -https://github.com/apache/systemml-website/raw/master/talks/systemml-kdd-tutorial.zip";>Download Hands-On Tutorial +https://sparktc.ibmcloud.com/repo/KDD2017/systemml-kdd-tutorial.zip";>Download Hands-On Tutorial http://git-wip-us.apache.org/repos/asf/systemml-website/blob/4620131c/talks/systemml-kdd-tutorial.zip -- diff --git a/talks/systemml-kdd-tutorial.zip b/talks/systemml-kdd-tutorial.zip deleted file mode 100644 index 59906b5..000 Binary files a/talks/systemml-kdd-tutorial.zip and /dev/null differ
svn commit: r1805045 - /systemml/site/tutorial-kdd2017.html
Author: gweidner Date: Tue Aug 15 00:14:07 2017 New Revision: 1805045 URL: http://svn.apache.org/viewvc?rev=1805045&view=rev Log: Switch KDD Tutorial download link Modified: systemml/site/tutorial-kdd2017.html Modified: systemml/site/tutorial-kdd2017.html URL: http://svn.apache.org/viewvc/systemml/site/tutorial-kdd2017.html?rev=1805045&r1=1805044&r2=1805045&view=diff == --- systemml/site/tutorial-kdd2017.html (original) +++ systemml/site/tutorial-kdd2017.html Tue Aug 15 00:14:07 2017 @@ -166,7 +166,7 @@ Declarative, Large-Scale Machine Learning with Apache SystemML -https://github.com/apache/systemml-website/raw/master/talks/systemml-kdd-tutorial.zip";>Download Hands-On Tutorial +https://sparktc.ibmcloud.com/repo/KDD2017/systemml-kdd-tutorial.zip";>Download Hands-On Tutorial
systemml git commit: [MINOR] Fix javadoc warning
Repository: systemml Updated Branches: refs/heads/master 830e6589f -> c18352f29 [MINOR] Fix javadoc warning Closes #625. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/c18352f2 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/c18352f2 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/c18352f2 Branch: refs/heads/master Commit: c18352f29368569e6f1ffc475ba2f9b0fab6c403 Parents: 830e658 Author: Glenn Weidner Authored: Thu Aug 17 12:13:23 2017 -0700 Committer: Glenn Weidner Committed: Thu Aug 17 12:13:23 2017 -0700 -- .../instructions/gpu/context/GPUContext.java| 96 +++- 1 file changed, 72 insertions(+), 24 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/c18352f2/src/main/java/org/apache/sysml/runtime/instructions/gpu/context/GPUContext.java -- diff --git a/src/main/java/org/apache/sysml/runtime/instructions/gpu/context/GPUContext.java b/src/main/java/org/apache/sysml/runtime/instructions/gpu/context/GPUContext.java index 4c0562d..f107f47 100644 --- a/src/main/java/org/apache/sysml/runtime/instructions/gpu/context/GPUContext.java +++ b/src/main/java/org/apache/sysml/runtime/instructions/gpu/context/GPUContext.java @@ -65,8 +65,8 @@ import jcuda.runtime.JCuda; import jcuda.runtime.cudaDeviceProp; /** - * Represents a context per GPU accessible through the same JVM - * Each context holds cublas, cusparse, cudnn... handles which are separate for each GPU + * Represents a context per GPU accessible through the same JVM. + * Each context holds cublas, cusparse, cudnn... handles which are separate for each GPU. */ public class GPUContext { @@ -187,12 +187,22 @@ public class GPUContext { } } + /** +* Returns which device is currently being used. +* +* @return the current device for the calling host thread +*/ public static int cudaGetDevice() { int[] device = new int[1]; JCuda.cudaGetDevice(device); return device[0]; } + /** +* Returns which device is assigned to this GPUContext instance. +* +* @return active device assigned to this GPUContext instance +*/ public int getDeviceNum() { return deviceNum; } @@ -201,8 +211,10 @@ public class GPUContext { * Sets the device for the calling thread. * This method must be called after * {@link org.apache.sysml.runtime.controlprogram.context.ExecutionContext#getGPUContext(int)} -* If in a multi-threaded env like parfor, this method must be called when in the -* appropriate thread +* If in a multi-threaded environment like parfor, this method must be called when in the +* appropriate thread. +* +* @throws DMLRuntimeException if DMLRuntimeException occurs */ public void initializeThread() throws DMLRuntimeException { cudaSetDevice(deviceNum); @@ -295,7 +307,7 @@ public class GPUContext { } /** -* Does lazy cudaFree calls +* Does lazy cudaFree calls. * * @param toFree {@link Pointer} instance to be freed */ @@ -304,7 +316,7 @@ public class GPUContext { } /** -* does lazy/eager cudaFree calls +* Does lazy/eager cudaFree calls. * * @param toFree {@link Pointer} instance to be freed * @param eager true if to be done eagerly @@ -314,7 +326,7 @@ public class GPUContext { } /** -* Does lazy cudaFree calls +* Does lazy cudaFree calls. * * @param instructionName name of the instruction for which to record per instruction free time, null if do not want to record * @param toFree {@link Pointer} instance to be freed @@ -324,7 +336,7 @@ public class GPUContext { } /** -* Does cudaFree calls, lazily +* Does cudaFree calls, lazily. * * @param instructionName name of the instruction for which to record per instruction free time, null if do not want to record * @param toFree {@link Pointer} instance to be freed @@ -366,7 +378,7 @@ public class GPUContext { } /** -* Thin wrapper over {@link GPUContext#evict(long)} +* Thin wrapper over {@link GPUContext#evict(long)}. * * @param size size to check * @throws DMLRuntimeException if DMLRuntimeException occurs @@ -376,7 +388,7 @@ public class GPUContext { } /** -* Thin wrapper over {@link GPUContext#evict(long)} +* Thin wrapper
systemml-website git commit: Add 'DML tips and tricks' sample Jupyter notebook
Repository: systemml-website Updated Branches: refs/heads/master 4620131c3 -> 75c86d9be Add 'DML tips and tricks' sample Jupyter notebook Added link to 'DML tips and tricks' Jupyter notebook in place of Linear Regression Conjugate Gradient Zeppelin notebook example. Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/75c86d9b Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/75c86d9b Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/75c86d9b Branch: refs/heads/master Commit: 75c86d9be582c149314e5878ff5826b1b6c33405 Parents: 4620131 Author: Glenn Weidner Authored: Sun Aug 20 14:03:47 2017 -0700 Committer: Glenn Weidner Committed: Sun Aug 20 14:03:47 2017 -0700 -- _src/get-started.html | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/75c86d9b/_src/get-started.html -- diff --git a/_src/get-started.html b/_src/get-started.html index 25ff087..81344d4 100644 --- a/_src/get-started.html +++ b/_src/get-started.html @@ -75,9 +75,9 @@ limitations under the License. https://github.com/apache/systemml/blob/master/samples/jupyter-notebooks/Autoencoder.ipynb"; target="_blank">View on Github -SystemML Linear Regression Conjugate Gradient -This example shows SystemML Linear Regression Conjugate Gradient algorithm in Zeppelin Notebook. -https://github.com/apache/systemml/blob/master/samples/zeppelin-notebooks/SystemML_LinearRegCG.json"; target="_blank">View on Github +DML Tips and Tricks +This example shows helpful matrix processing techniques and other linear algebra operations using SystemML's Declarative Machine Learning (DML) scripting language. +https://github.com/apache/systemml/blob/master/samples/jupyter-notebooks/DML%20Tips%20and%20Tricks%20(aka%20Fun%20With%20DML).ipynb" target="_blank">View on Github
svn commit: r1805583 - /systemml/site/get-started.html
Author: gweidner Date: Sun Aug 20 21:12:48 2017 New Revision: 1805583 URL: http://svn.apache.org/viewvc?rev=1805583&view=rev Log: Add 'DML tips and tricks' sample Jupyter notebook Modified: systemml/site/get-started.html Modified: systemml/site/get-started.html URL: http://svn.apache.org/viewvc/systemml/site/get-started.html?rev=1805583&r1=1805582&r2=1805583&view=diff == --- systemml/site/get-started.html (original) +++ systemml/site/get-started.html Sun Aug 20 21:12:48 2017 @@ -199,9 +199,9 @@ https://github.com/apache/systemml/blob/master/samples/jupyter-notebooks/Autoencoder.ipynb"; target="_blank">View on Github -SystemML Linear Regression Conjugate Gradient -This example shows SystemML Linear Regression Conjugate Gradient algorithm in Zeppelin Notebook. -https://github.com/apache/systemml/blob/master/samples/zeppelin-notebooks/SystemML_LinearRegCG.json"; target="_blank">View on Github +DML Tips and Tricks +This example shows helpful matrix processing techniques and other linear algebra operations using SystemML's Declarative Machine Learning (DML) scripting language. +https://github.com/apache/systemml/blob/master/samples/jupyter-notebooks/DML%20Tips%20and%20Tricks%20(aka%20Fun%20With%20DML).ipynb" target="_blank">View on Github
systemml git commit: [MINOR] Fix sphinx build warnings for python documentation
Repository: systemml Updated Branches: refs/heads/master 628ffad1b -> 6363295cd [MINOR] Fix sphinx build warnings for python documentation Closes #638. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/6363295c Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/6363295c Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/6363295c Branch: refs/heads/master Commit: 6363295cd44c09365cd93ebfee04384495a67000 Parents: 628ffad Author: Glenn Weidner Authored: Thu Aug 24 15:24:18 2017 -0700 Committer: Glenn Weidner Committed: Thu Aug 24 15:24:18 2017 -0700 -- src/main/python/systemml/converters.py | 4 ++-- src/main/python/systemml/defmatrix.py | 8 2 files changed, 6 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/6363295c/src/main/python/systemml/converters.py -- diff --git a/src/main/python/systemml/converters.py b/src/main/python/systemml/converters.py index 1309cb9..aaa9f88 100644 --- a/src/main/python/systemml/converters.py +++ b/src/main/python/systemml/converters.py @@ -218,8 +218,8 @@ def convertToNumPyArr(sc, mb): # Returns the mean of a model if defined otherwise None def getDatasetMean(dataset_name): """ -Input Parameters - +Parameters +-- dataset_name: Name of the dataset used to train model. This name is artificial name based on dataset used to train the model. Returns http://git-wip-us.apache.org/repos/asf/systemml/blob/6363295c/src/main/python/systemml/defmatrix.py -- diff --git a/src/main/python/systemml/defmatrix.py b/src/main/python/systemml/defmatrix.py index 3e13bf2..e9bd4b1 100644 --- a/src/main/python/systemml/defmatrix.py +++ b/src/main/python/systemml/defmatrix.py @@ -406,17 +406,17 @@ class matrix(object): 3. A matrix object can either be evaluated or not. If evaluated, the attribute 'data' is set to one of the supported types (for example: NumPy array or DataFrame). In this case, the attribute 'op' is set to None. If not evaluated, the attribute 'op' which refers to one of the intermediate node of AST and if of type DMLOp. In this case, the attribute 'data' is set to None. -5. DMLOp has an attribute 'inputs' which contains list of matrix objects or DMLOp. -6. To simplify the traversal, every matrix object is considered immutable and an matrix operations creates a new matrix object. +4. DMLOp has an attribute 'inputs' which contains list of matrix objects or DMLOp. +5. To simplify the traversal, every matrix object is considered immutable and an matrix operations creates a new matrix object. As an example: `m1 = sml.matrix(np.ones((3,3)))` creates a matrix object backed by 'data=(np.ones((3,3))'. `m1 = m1 * 2` will create a new matrix object which is now backed by 'op=DMLOp( ... )' whose input is earlier created matrix object. -7. Left indexing (implemented in __setitem__ method) is a special case, where Python expects the existing object to be mutated. +6. Left indexing (implemented in __setitem__ method) is a special case, where Python expects the existing object to be mutated. To ensure the above property, we make deep copy of existing object and point any references to the left-indexed matrix to the newly created object. Then the left-indexed matrix is set to be backed by DMLOp consisting of following pydml: left-indexed-matrix = new-deep-copied-matrix left-indexed-matrix[index] = value -8. Please use m.print_ast() and/or type `m` for debugging. Here is a sample session: +7. Please use m.print_ast() and/or type `m` for debugging. Here is a sample session: >>> npm = np.ones((3,3)) >>> m1 = sml.matrix(npm + 3)
systemml git commit: [SYSTEMML-1825] Add sphinx build files for Python API doc generation
Repository: systemml Updated Branches: refs/heads/master 4744da167 -> c249c6fdf [SYSTEMML-1825] Add sphinx build files for Python API doc generation Added files for generating Python API documentation with Sphinx. Closes #646. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/c249c6fd Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/c249c6fd Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/c249c6fd Branch: refs/heads/master Commit: c249c6fdfeff23833faca0ecfe8399e4d54bee71 Parents: 4744da1 Author: Glenn Weidner Authored: Wed Aug 30 14:52:36 2017 -0700 Committer: Glenn Weidner Committed: Wed Aug 30 14:52:37 2017 -0700 -- pom.xml | 2 + src/main/pythondoc/Makefile | 37 + src/main/pythondoc/conf.py | 211 +++ src/main/pythondoc/index.rst| 21 +++ src/main/pythondoc/make.bat | 57 src/main/pythondoc/systemml.mllearn.rst | 22 +++ src/main/pythondoc/systemml.random.rst | 22 +++ src/main/pythondoc/systemml.rst | 54 +++ 8 files changed, 426 insertions(+) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/c249c6fd/pom.xml -- diff --git a/pom.xml b/pom.xml index 0ee382c..4bbe714 100644 --- a/pom.xml +++ b/pom.xml @@ -878,6 +878,8 @@ src/main/java/*.tokens src/main/python/systemml.egg-info/** + + src/main/pythondoc/*.rst src/main/cpp/kernels/SystemML.ptx http://git-wip-us.apache.org/repos/asf/systemml/blob/c249c6fd/src/main/pythondoc/Makefile -- diff --git a/src/main/pythondoc/Makefile b/src/main/pythondoc/Makefile new file mode 100644 index 000..e68aeaf --- /dev/null +++ b/src/main/pythondoc/Makefile @@ -0,0 +1,37 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS= +SPHINXBUILD = python -msphinx +SPHINXPROJ= SystemML +SOURCEDIR = . +BUILDDIR = ../../../target/pydocs + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) http://git-wip-us.apache.org/repos/asf/systemml/blob/c249c6fd/src/main/pythondoc/conf.py -- diff --git a/src/main/pythondoc/conf.py b/src/main/pythondoc/conf.py new file mode 100644 index 000..11ca735 --- /dev/null +++ b/src/main/pythondoc/conf.py @@ -0,0 +1,211 @@ +# -*- coding: utf-8 -*- +# +#- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed
systemml git commit: [SYSTEMML-1860] Update release-process for python doc and add api menu item
Repository: systemml Updated Branches: refs/heads/master c249c6fdf -> 62a02464a [SYSTEMML-1860] Update release-process for python doc and add api menu item Closes #648. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/62a02464 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/62a02464 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/62a02464 Branch: refs/heads/master Commit: 62a02464ad533e57d84191d4bcaa60e056e2f00c Parents: c249c6f Author: Glenn Weidner Authored: Thu Aug 31 16:16:39 2017 -0700 Committer: Glenn Weidner Committed: Thu Aug 31 16:16:39 2017 -0700 -- docs/_layouts/global.html | 3 ++- docs/release-process.md | 18 -- 2 files changed, 18 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/62a02464/docs/_layouts/global.html -- diff --git a/docs/_layouts/global.html b/docs/_layouts/global.html index 5e84276..4286c9c 100644 --- a/docs/_layouts/global.html +++ b/docs/_layouts/global.html @@ -77,7 +77,8 @@ API Docs -Javadoc +Java +Python {% endif %} http://git-wip-us.apache.org/repos/asf/systemml/blob/62a02464/docs/release-process.md -- diff --git a/docs/release-process.md b/docs/release-process.md index 4a31f8b..d8d74f4 100644 --- a/docs/release-process.md +++ b/docs/release-process.md @@ -204,7 +204,7 @@ this OS X example. ./runStandaloneSystemML.sh hello.dml cd .. - # verify standalon zip works + # verify standalone zip works rm -rf systemml-1.0.0-bin unzip systemml-1.0.0-bin.zip cd systemml-1.0.0-bin @@ -428,6 +428,7 @@ Copy contents of `systemml/docs/_site` to `systemml-website-site/docs/1.0.0`. Delete any unnecessary files (`Gemfile`, `Gemfile.lock`). Create `systemml-website-site/docs/1.0.0/api/java` folder for javadocs. +Create `systemml-website-site/docs/1.0.0/api/python` folder for pythondocs. Update `systemml/pom.xml` project version to what should be displayed in javadocs (such as `1.0.0`). @@ -437,7 +438,20 @@ Build project (which generates javadocs). Copy contents of `systemml/target/apidocs` to `systemml-website-site/docs/1.0.0/api/java`. -Open up `file:///.../systemml-website-site/docs/1.0.0/index.html` and verify `API Docs` → `Javadoc` link works and that the correct Javadoc version is displayed. Verify feedback links under `Issues` menu are not present. +Define environment variables to match version and release number used in updated `systemml/pom.xml`. Both environment variables are referenced when building pythondocs with Sphinx. + + $ export SYSTEMML_VERSION=1.0 + $ export SYSTEMML_RELEASE=1.0.0 + +Generate pythondocs with Sphinx. + + $ cd systemml/src/main/pythondoc + $ make html + +Copy contents of `systemml/target/pydocs/html` to `systemml-website-site/docs/1.0.0/api/python`. + +Open up `file:///.../systemml-website-site/docs/1.0.0/index.html` and verify `API Docs` → `Java` link works and that the correct Javadoc version is displayed. +Verify `API Docs` → `Python` link works and that the same Pythondoc version is displayed. Verify feedback links under `Issues` menu are not present. Clean up any unnecessary files (such as deleting `.DS_Store` files on OS X).
systemml git commit: [MINOR] Update version in README files
Repository: systemml Updated Branches: refs/heads/master 4d376637a -> 824e4c05c [MINOR] Update version in README files Updated version and removed incubating. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/824e4c05 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/824e4c05 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/824e4c05 Branch: refs/heads/master Commit: 824e4c05c53102943c045a765fa522ee82a6dab5 Parents: 4d37663 Author: Glenn Weidner Authored: Thu Sep 7 16:42:32 2017 -0700 Committer: Glenn Weidner Committed: Thu Sep 7 16:42:32 2017 -0700 -- README.md | 4 ++-- docs/spark-mlcontext-programming-guide.md | 8 src/main/standalone/README.txt| 4 ++-- 3 files changed, 8 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/824e4c05/README.md -- diff --git a/README.md b/README.md index dcc90db..565854a 100644 --- a/README.md +++ b/README.md @@ -107,14 +107,14 @@ Before you get started on SystemML, make sure that your environment is set up an 4. **Download SystemML.** - Go to the [SystemML Downloads page](http://systemml.apache.org/download.html), download `systemml-0.14.0-incubating.zip` (should be 2nd), and unzip it to a location of your choice. + Go to the [SystemML Downloads page](http://systemml.apache.org/download.html), download `systemml-0.15.0.zip` (should be 2nd), and unzip it to a location of your choice. *The next step is optional, but it will make your life a lot easier.* 5. **[OPTIONAL] Set `SYSTEMML_HOME` in your bash profile.** Add the following to `~/.bash_profile`, replacing `path/to/` with the location of the download in step 5. ``` - export SYSTEMML_HOME=path/to/systemml-0.14.0-incubating + export SYSTEMML_HOME=path/to/systemml-0.15.0 ``` *Make sure to open a new tab in terminal so that you make sure the changes have been made.* http://git-wip-us.apache.org/repos/asf/systemml/blob/824e4c05/docs/spark-mlcontext-programming-guide.md -- diff --git a/docs/spark-mlcontext-programming-guide.md b/docs/spark-mlcontext-programming-guide.md index 22f5a1b..e935c65 100644 --- a/docs/spark-mlcontext-programming-guide.md +++ b/docs/spark-mlcontext-programming-guide.md @@ -174,7 +174,7 @@ MLResults ## LeNet on MNIST Example -SystemML features the DML-based [`nn` library for deep learning](https://github.com/apache/incubator-systemml/tree/master/scripts/nn). +SystemML features the DML-based [`nn` library for deep learning](https://github.com/apache/systemml/tree/master/scripts/nn). At project build time, SystemML automatically generates wrapper classes for DML scripts to enable convenient access to scripts and execution of functions. @@ -1220,7 +1220,7 @@ via `ml.scripts.algorithms.Univar_Stats`, as shown below. This feature is not av {% highlight scala %} val habermanUrl = "http://archive.ics.uci.edu/ml/machine-learning-databases/haberman/haberman.data"; val typesRDD = sc.parallelize(Array("1.0,1.0,1.0,2.0")) -val scriptUrl = "https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/algorithms/Univar-Stats.dml"; +val scriptUrl = "https://raw.githubusercontent.com/apache/systemml/master/scripts/algorithms/Univar-Stats.dml"; ml.scripts.algorithms.Univar_Stats.in("A", new java.net.URL(habermanUrl)).in("K", typesRDD).in("$CONSOLE_OUTPUT", true).execute {% endhighlight %} @@ -1233,8 +1233,8 @@ habermanUrl: String = http://archive.ics.uci.edu/ml/machine-learning-databases/h scala> val typesRDD = sc.parallelize(Array("1.0,1.0,1.0,2.0")) typesRDD: org.apache.spark.rdd.RDD[String] = ParallelCollectionRDD[21] at parallelize at :30 -scala> val scriptUrl = "https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/algorithms/Univar-Stats.dml"; -scriptUrl: String = https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/algorithms/Univar-Stats.dml +scala> val scriptUrl = "https://raw.githubusercontent.com/apache/systemml/master/scripts/algorithms/Univar-Stats.dml"; +scriptUrl: String = https://raw.githubusercontent.com/apache/systemml/master/scripts/algorithms/Univar-Stats.dml scala> ml.scripts.algorithms.Univar_Stats.in("A", new java.net.URL(habermanUrl)).in("K", typesRDD).in("$CONSOLE_OUTPUT", true).execute 17/06/05 17:23:37 WARN RewriteRemovePersistentReadWrite: Non-registered persistent write of variable 'baseStats' (line 186). http://git-wip-us.apache.org/repos/asf/systemml/blob/824e4c05/src/main/standalone/README.txt -- diff --git a/src/main/sta
systemml git commit: [MINOR] Update version in README files
Repository: systemml Updated Branches: refs/heads/branch-0.15 ea8e951bc -> 2811479c9 [MINOR] Update version in README files Updated version and removed incubating. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/2811479c Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/2811479c Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/2811479c Branch: refs/heads/branch-0.15 Commit: 2811479c969d4769472c7b1ca7eab74c98339fd7 Parents: ea8e951 Author: Glenn Weidner Authored: Thu Sep 7 16:42:32 2017 -0700 Committer: Glenn Weidner Committed: Thu Sep 7 16:42:32 2017 -0700 -- README.md | 4 ++-- docs/spark-mlcontext-programming-guide.md | 8 src/main/standalone/README.txt| 4 ++-- 3 files changed, 8 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/2811479c/README.md -- diff --git a/README.md b/README.md index dcc90db..565854a 100644 --- a/README.md +++ b/README.md @@ -107,14 +107,14 @@ Before you get started on SystemML, make sure that your environment is set up an 4. **Download SystemML.** - Go to the [SystemML Downloads page](http://systemml.apache.org/download.html), download `systemml-0.14.0-incubating.zip` (should be 2nd), and unzip it to a location of your choice. + Go to the [SystemML Downloads page](http://systemml.apache.org/download.html), download `systemml-0.15.0.zip` (should be 2nd), and unzip it to a location of your choice. *The next step is optional, but it will make your life a lot easier.* 5. **[OPTIONAL] Set `SYSTEMML_HOME` in your bash profile.** Add the following to `~/.bash_profile`, replacing `path/to/` with the location of the download in step 5. ``` - export SYSTEMML_HOME=path/to/systemml-0.14.0-incubating + export SYSTEMML_HOME=path/to/systemml-0.15.0 ``` *Make sure to open a new tab in terminal so that you make sure the changes have been made.* http://git-wip-us.apache.org/repos/asf/systemml/blob/2811479c/docs/spark-mlcontext-programming-guide.md -- diff --git a/docs/spark-mlcontext-programming-guide.md b/docs/spark-mlcontext-programming-guide.md index 22f5a1b..e935c65 100644 --- a/docs/spark-mlcontext-programming-guide.md +++ b/docs/spark-mlcontext-programming-guide.md @@ -174,7 +174,7 @@ MLResults ## LeNet on MNIST Example -SystemML features the DML-based [`nn` library for deep learning](https://github.com/apache/incubator-systemml/tree/master/scripts/nn). +SystemML features the DML-based [`nn` library for deep learning](https://github.com/apache/systemml/tree/master/scripts/nn). At project build time, SystemML automatically generates wrapper classes for DML scripts to enable convenient access to scripts and execution of functions. @@ -1220,7 +1220,7 @@ via `ml.scripts.algorithms.Univar_Stats`, as shown below. This feature is not av {% highlight scala %} val habermanUrl = "http://archive.ics.uci.edu/ml/machine-learning-databases/haberman/haberman.data"; val typesRDD = sc.parallelize(Array("1.0,1.0,1.0,2.0")) -val scriptUrl = "https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/algorithms/Univar-Stats.dml"; +val scriptUrl = "https://raw.githubusercontent.com/apache/systemml/master/scripts/algorithms/Univar-Stats.dml"; ml.scripts.algorithms.Univar_Stats.in("A", new java.net.URL(habermanUrl)).in("K", typesRDD).in("$CONSOLE_OUTPUT", true).execute {% endhighlight %} @@ -1233,8 +1233,8 @@ habermanUrl: String = http://archive.ics.uci.edu/ml/machine-learning-databases/h scala> val typesRDD = sc.parallelize(Array("1.0,1.0,1.0,2.0")) typesRDD: org.apache.spark.rdd.RDD[String] = ParallelCollectionRDD[21] at parallelize at :30 -scala> val scriptUrl = "https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/algorithms/Univar-Stats.dml"; -scriptUrl: String = https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/algorithms/Univar-Stats.dml +scala> val scriptUrl = "https://raw.githubusercontent.com/apache/systemml/master/scripts/algorithms/Univar-Stats.dml"; +scriptUrl: String = https://raw.githubusercontent.com/apache/systemml/master/scripts/algorithms/Univar-Stats.dml scala> ml.scripts.algorithms.Univar_Stats.in("A", new java.net.URL(habermanUrl)).in("K", typesRDD).in("$CONSOLE_OUTPUT", true).execute 17/06/05 17:23:37 WARN RewriteRemovePersistentReadWrite: Non-registered persistent write of variable 'baseStats' (line 186). http://git-wip-us.apache.org/repos/asf/systemml/blob/2811479c/src/main/standalone/README.txt -- diff --git a/sr
[2/2] systemml git commit: [MINOR] Update version in README files
[MINOR] Update version in README files Updated version and removed incubating. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/288bc35c Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/288bc35c Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/288bc35c Branch: refs/heads/gh-pages Commit: 288bc35c3f8bde4c3478ad093ade1e03275488e8 Parents: 24245fd Author: Glenn Weidner Authored: Thu Sep 7 16:42:32 2017 -0700 Committer: Glenn Weidner Committed: Thu Sep 7 16:42:32 2017 -0700 -- spark-mlcontext-programming-guide.md | 8 1 file changed, 4 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/288bc35c/spark-mlcontext-programming-guide.md -- diff --git a/spark-mlcontext-programming-guide.md b/spark-mlcontext-programming-guide.md index 22f5a1b..e935c65 100644 --- a/spark-mlcontext-programming-guide.md +++ b/spark-mlcontext-programming-guide.md @@ -174,7 +174,7 @@ MLResults ## LeNet on MNIST Example -SystemML features the DML-based [`nn` library for deep learning](https://github.com/apache/incubator-systemml/tree/master/scripts/nn). +SystemML features the DML-based [`nn` library for deep learning](https://github.com/apache/systemml/tree/master/scripts/nn). At project build time, SystemML automatically generates wrapper classes for DML scripts to enable convenient access to scripts and execution of functions. @@ -1220,7 +1220,7 @@ via `ml.scripts.algorithms.Univar_Stats`, as shown below. This feature is not av {% highlight scala %} val habermanUrl = "http://archive.ics.uci.edu/ml/machine-learning-databases/haberman/haberman.data"; val typesRDD = sc.parallelize(Array("1.0,1.0,1.0,2.0")) -val scriptUrl = "https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/algorithms/Univar-Stats.dml"; +val scriptUrl = "https://raw.githubusercontent.com/apache/systemml/master/scripts/algorithms/Univar-Stats.dml"; ml.scripts.algorithms.Univar_Stats.in("A", new java.net.URL(habermanUrl)).in("K", typesRDD).in("$CONSOLE_OUTPUT", true).execute {% endhighlight %} @@ -1233,8 +1233,8 @@ habermanUrl: String = http://archive.ics.uci.edu/ml/machine-learning-databases/h scala> val typesRDD = sc.parallelize(Array("1.0,1.0,1.0,2.0")) typesRDD: org.apache.spark.rdd.RDD[String] = ParallelCollectionRDD[21] at parallelize at :30 -scala> val scriptUrl = "https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/algorithms/Univar-Stats.dml"; -scriptUrl: String = https://raw.githubusercontent.com/apache/incubator-systemml/master/scripts/algorithms/Univar-Stats.dml +scala> val scriptUrl = "https://raw.githubusercontent.com/apache/systemml/master/scripts/algorithms/Univar-Stats.dml"; +scriptUrl: String = https://raw.githubusercontent.com/apache/systemml/master/scripts/algorithms/Univar-Stats.dml scala> ml.scripts.algorithms.Univar_Stats.in("A", new java.net.URL(habermanUrl)).in("K", typesRDD).in("$CONSOLE_OUTPUT", true).execute 17/06/05 17:23:37 WARN RewriteRemovePersistentReadWrite: Non-registered persistent write of variable 'baseStats' (line 186).
[1/2] systemml git commit: [SYSTEMML-1860] Update release-process for python doc and add api menu item
Repository: systemml Updated Branches: refs/heads/gh-pages 4e22b91ea -> 288bc35c3 [SYSTEMML-1860] Update release-process for python doc and add api menu item Closes #648. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/24245fd5 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/24245fd5 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/24245fd5 Branch: refs/heads/gh-pages Commit: 24245fd5ec5ac288bc3b2599ef3892a269e33ccc Parents: 4e22b91 Author: Glenn Weidner Authored: Thu Aug 31 16:16:39 2017 -0700 Committer: Glenn Weidner Committed: Thu Aug 31 16:16:39 2017 -0700 -- _layouts/global.html | 3 ++- release-process.md | 18 -- 2 files changed, 18 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/24245fd5/_layouts/global.html -- diff --git a/_layouts/global.html b/_layouts/global.html index 5e84276..4286c9c 100644 --- a/_layouts/global.html +++ b/_layouts/global.html @@ -77,7 +77,8 @@ API Docs -Javadoc +Java +Python {% endif %} http://git-wip-us.apache.org/repos/asf/systemml/blob/24245fd5/release-process.md -- diff --git a/release-process.md b/release-process.md index 4a31f8b..d8d74f4 100644 --- a/release-process.md +++ b/release-process.md @@ -204,7 +204,7 @@ this OS X example. ./runStandaloneSystemML.sh hello.dml cd .. - # verify standalon zip works + # verify standalone zip works rm -rf systemml-1.0.0-bin unzip systemml-1.0.0-bin.zip cd systemml-1.0.0-bin @@ -428,6 +428,7 @@ Copy contents of `systemml/docs/_site` to `systemml-website-site/docs/1.0.0`. Delete any unnecessary files (`Gemfile`, `Gemfile.lock`). Create `systemml-website-site/docs/1.0.0/api/java` folder for javadocs. +Create `systemml-website-site/docs/1.0.0/api/python` folder for pythondocs. Update `systemml/pom.xml` project version to what should be displayed in javadocs (such as `1.0.0`). @@ -437,7 +438,20 @@ Build project (which generates javadocs). Copy contents of `systemml/target/apidocs` to `systemml-website-site/docs/1.0.0/api/java`. -Open up `file:///.../systemml-website-site/docs/1.0.0/index.html` and verify `API Docs` → `Javadoc` link works and that the correct Javadoc version is displayed. Verify feedback links under `Issues` menu are not present. +Define environment variables to match version and release number used in updated `systemml/pom.xml`. Both environment variables are referenced when building pythondocs with Sphinx. + + $ export SYSTEMML_VERSION=1.0 + $ export SYSTEMML_RELEASE=1.0.0 + +Generate pythondocs with Sphinx. + + $ cd systemml/src/main/pythondoc + $ make html + +Copy contents of `systemml/target/pydocs/html` to `systemml-website-site/docs/1.0.0/api/python`. + +Open up `file:///.../systemml-website-site/docs/1.0.0/index.html` and verify `API Docs` → `Java` link works and that the correct Javadoc version is displayed. +Verify `API Docs` → `Python` link works and that the same Pythondoc version is displayed. Verify feedback links under `Issues` menu are not present. Clean up any unnecessary files (such as deleting `.DS_Store` files on OS X).
systemml git commit: [SYSTEMML-1605] Refresh zeppelin samples
Repository: systemml Updated Branches: refs/heads/master 5a2b49f20 -> 8395ffb29 [SYSTEMML-1605] Refresh zeppelin samples Updated dependency to use release and refreshed cells. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/8395ffb2 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/8395ffb2 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/8395ffb2 Branch: refs/heads/master Commit: 8395ffb29d8b502619fa667bc873182977b418dd Parents: 5a2b49f Author: Glenn Weidner Authored: Thu Sep 7 20:16:53 2017 -0700 Committer: Glenn Weidner Committed: Thu Sep 7 20:16:53 2017 -0700 -- samples/zeppelin-notebooks/SystemML_LinearRegCG.json | 2 +- samples/zeppelin-notebooks/tutorial1_zeppelin.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/8395ffb2/samples/zeppelin-notebooks/SystemML_LinearRegCG.json -- diff --git a/samples/zeppelin-notebooks/SystemML_LinearRegCG.json b/samples/zeppelin-notebooks/SystemML_LinearRegCG.json index 1dc91cb..49976cd 100644 --- a/samples/zeppelin-notebooks/SystemML_LinearRegCG.json +++ b/samples/zeppelin-notebooks/SystemML_LinearRegCG.json @@ -1 +1 @@ -{"paragraphs":[{"user":"anonymous","config":{"colWidth":12,"enabled":true,"results":{},"editorSetting":{"language":"scala"},"editorMode":"ace/mode/scala"},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1494545879068_-1034995073","id":"20170511-163759_12734001","dateCreated":"2017-05-11T16:37:59-0700","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:8101","text":"%dep\r\nz.load(\"org.apache.systemml:systemml:0.14.0-incubating\")","dateUpdated":"2017-05-11T16:38:53-0700","dateFinished":"2017-05-11T16:38:54-0700","dateStarted":"2017-05-11T16:38:53-0700","results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"res1: org.apache.zeppelin.dep.Dependency = org.apache.zeppelin.dep.Dependency@7e785cff\n"}]}},{"text":"sc.version","user":"anonymous","dateUpdated":"2017-05-11T16:38:53-0700","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"form s":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"\nres0: String = 2.1.0\n"}]},"apps":[],"jobName":"paragraph_1494543525010_-885655540","id":"20170511-144323_52625604","dateCreated":"2017-05-11T15:58:45-0700","dateStarted":"2017-05-11T16:38:54-0700","dateFinished":"2017-05-11T16:39:07-0700","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:7688"},{"text":"import org.apache.sysml.api.mlcontext._","user":"anonymous","dateUpdated":"2017-05-11T16:38:53-0700","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"\nimport org.apache.sysml.api.mlcontext._\n"}]},"apps":[],"jobName":"paragraph_1494543525011_-886040289","id":"20170511-144349_335242548","dateCreated":"2017-05-11T15:58:45-0700","dateStarted":"2017-05-11T16:38:54-0700","dateFinished":"2017-05-11T16:39:08-0700","status":"FIN ISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:7689"},{"text":"val ml = new MLContext(sc)","user":"anonymous","dateUpdated":"2017-05-11T16:38:53-0700","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"\nml: org.apache.sysml.api.mlcontext.MLContext = org.apache.sysml.api.mlcontext.MLContext@403d1309\n"}]},"apps":[],"jobName":"paragraph_1494543525011_-886040289","id":"20170511-144600_1148672764","dateCreated":"2017-05-11T15:58:45-0700","dateStarted":"2017-05-11T16:39:08-0700","dateFinished":"2017-05-11T16:39:08-0700","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:7690"},{"text":"ml.info","user":"anonymous","dateUpdated":"2017-05-11T16:38:53-0700","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{}," forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"\n\n\n\n\n\n\n\n\n\n\n\nres1: org.apache.sysml.api.mlcontext.ProjectInfo =\nArchiver-Version: Plexus Archiver\nArtifact-Id: systemml\nBuild-Jdk: 1.8.0_121\nBuild-Time: 2017-04-19 21:45:10 UTC\nBuilt-By: asurve\nCreated-By: Apache Maven 3.3.9\nGroup-Id: org.apache.systemml\nMain-Class: org.apache.sysml.api.DMLScript\nManifest-Version: 1.0\nMinimum-Recommended-Spark-Version: 2.1.0\nVersion: 0.14.0-incubating\n"}]},"apps":[],"jobName":"paragraph_1494
systemml git commit: [SYSTEMML-1605] Refresh zeppelin samples
Repository: systemml Updated Branches: refs/heads/branch-0.15 26166d219 -> e912550f2 [SYSTEMML-1605] Refresh zeppelin samples Updated dependency to use release and refreshed cells. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/e912550f Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/e912550f Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/e912550f Branch: refs/heads/branch-0.15 Commit: e912550f29fa93d314cc2c5afe5185cebef9cc9a Parents: 26166d2 Author: Glenn Weidner Authored: Thu Sep 7 20:16:53 2017 -0700 Committer: Glenn Weidner Committed: Thu Sep 7 20:16:53 2017 -0700 -- samples/zeppelin-notebooks/SystemML_LinearRegCG.json | 2 +- samples/zeppelin-notebooks/tutorial1_zeppelin.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/e912550f/samples/zeppelin-notebooks/SystemML_LinearRegCG.json -- diff --git a/samples/zeppelin-notebooks/SystemML_LinearRegCG.json b/samples/zeppelin-notebooks/SystemML_LinearRegCG.json index 1dc91cb..49976cd 100644 --- a/samples/zeppelin-notebooks/SystemML_LinearRegCG.json +++ b/samples/zeppelin-notebooks/SystemML_LinearRegCG.json @@ -1 +1 @@ -{"paragraphs":[{"user":"anonymous","config":{"colWidth":12,"enabled":true,"results":{},"editorSetting":{"language":"scala"},"editorMode":"ace/mode/scala"},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1494545879068_-1034995073","id":"20170511-163759_12734001","dateCreated":"2017-05-11T16:37:59-0700","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:8101","text":"%dep\r\nz.load(\"org.apache.systemml:systemml:0.14.0-incubating\")","dateUpdated":"2017-05-11T16:38:53-0700","dateFinished":"2017-05-11T16:38:54-0700","dateStarted":"2017-05-11T16:38:53-0700","results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"res1: org.apache.zeppelin.dep.Dependency = org.apache.zeppelin.dep.Dependency@7e785cff\n"}]}},{"text":"sc.version","user":"anonymous","dateUpdated":"2017-05-11T16:38:53-0700","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"form s":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"\nres0: String = 2.1.0\n"}]},"apps":[],"jobName":"paragraph_1494543525010_-885655540","id":"20170511-144323_52625604","dateCreated":"2017-05-11T15:58:45-0700","dateStarted":"2017-05-11T16:38:54-0700","dateFinished":"2017-05-11T16:39:07-0700","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:7688"},{"text":"import org.apache.sysml.api.mlcontext._","user":"anonymous","dateUpdated":"2017-05-11T16:38:53-0700","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"\nimport org.apache.sysml.api.mlcontext._\n"}]},"apps":[],"jobName":"paragraph_1494543525011_-886040289","id":"20170511-144349_335242548","dateCreated":"2017-05-11T15:58:45-0700","dateStarted":"2017-05-11T16:38:54-0700","dateFinished":"2017-05-11T16:39:08-0700","status":"FIN ISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:7689"},{"text":"val ml = new MLContext(sc)","user":"anonymous","dateUpdated":"2017-05-11T16:38:53-0700","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"\nml: org.apache.sysml.api.mlcontext.MLContext = org.apache.sysml.api.mlcontext.MLContext@403d1309\n"}]},"apps":[],"jobName":"paragraph_1494543525011_-886040289","id":"20170511-144600_1148672764","dateCreated":"2017-05-11T15:58:45-0700","dateStarted":"2017-05-11T16:39:08-0700","dateFinished":"2017-05-11T16:39:08-0700","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:7690"},{"text":"ml.info","user":"anonymous","dateUpdated":"2017-05-11T16:38:53-0700","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{}," forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"\n\n\n\n\n\n\n\n\n\n\n\nres1: org.apache.sysml.api.mlcontext.ProjectInfo =\nArchiver-Version: Plexus Archiver\nArtifact-Id: systemml\nBuild-Jdk: 1.8.0_121\nBuild-Time: 2017-04-19 21:45:10 UTC\nBuilt-By: asurve\nCreated-By: Apache Maven 3.3.9\nGroup-Id: org.apache.systemml\nMain-Class: org.apache.sysml.api.DMLScript\nManifest-Version: 1.0\nMinimum-Recommended-Spark-Version: 2.1.0\nVersion: 0.14.0-incubating\n"}]},"apps":[],"jobName":"para
systemml-website git commit: Update documentation link, project, roadmap, release notes for 0.15 release
Repository: systemml-website Updated Branches: refs/heads/master 89ce85a63 -> 49fa497e8 Update documentation link, project, roadmap, release notes for 0.15 release Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/49fa497e Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/49fa497e Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/49fa497e Branch: refs/heads/master Commit: 49fa497e87ed63168378e0cc9e004f439dd4acb8 Parents: 89ce85a Author: Glenn Weidner Authored: Wed Sep 13 13:44:34 2017 -0700 Committer: Glenn Weidner Committed: Wed Sep 13 13:44:34 2017 -0700 -- _src/_data/project.yml | 2 +- _src/documentation.html | 1 + .../systemml-release-notes-0.15.0.md| 57 _src/roadmap.html | 33 ++-- 4 files changed, 88 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/49fa497e/_src/_data/project.yml -- diff --git a/_src/_data/project.yml b/_src/_data/project.yml index afad611..65b84b9 100644 --- a/_src/_data/project.yml +++ b/_src/_data/project.yml @@ -50,7 +50,7 @@ issues_list_archive_markmail: jira: SYSTEMML release_name: systemml -release_version: 0.14.0-incubating +release_version: 0.15.0 source_repository: https://git-wip-us.apache.org/repos/asf/systemml.git source_repository_mirror: https://github.com/apache/systemml http://git-wip-us.apache.org/repos/asf/systemml-website/blob/49fa497e/_src/documentation.html -- diff --git a/_src/documentation.html b/_src/documentation.html index 7a2ade5..ea4e948 100644 --- a/_src/documentation.html +++ b/_src/documentation.html @@ -39,6 +39,7 @@ limitations under the License. Documentation such as programming guides, reference guides, and API docs is available online for each version of SystemML below: http://apache.github.io/systemml/";>Latest +0.15.0 0.14.0 0.13.0 0.12.0 http://git-wip-us.apache.org/repos/asf/systemml-website/blob/49fa497e/_src/release-notes/systemml-release-notes-0.15.0.md -- diff --git a/_src/release-notes/systemml-release-notes-0.15.0.md b/_src/release-notes/systemml-release-notes-0.15.0.md new file mode 100644 index 000..10201c3 --- /dev/null +++ b/_src/release-notes/systemml-release-notes-0.15.0.md @@ -0,0 +1,57 @@ +--- +layout: page +title: SystemML 0.15.0 Release Notes +description: Project Release Notes +group: nav-right +--- + + + + + + {{ site.data.project.name }} 0.15.0 Release Notes + + + + + + + + +The **Apache SystemML 0.15.0 release** was approved on September 13, 2017. It is the first release of Apache SystemML since it +[became a Top-Level Project (TLP)](https://blogs.apache.org/foundation/entry/the-apache-software-foundation-announces13) on May 31, 2017. Updates have been made to the project in several areas, as detailed below. + + +### New Features and Capabilities +- Added several new 2D convolution layers. +- Graduated `nn` library from staging to `scripts/nn`. + +### Experimental Features +- Expanded Code Generation for broader performance improvements. +- Enhanced GPU support and scalability. + +### Removals +- Removed file-based transform. +- Removed original MLContext API. + +### Various Fixes +* Bug fixes for diverse issues. See [JIRA release notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319522&version=12341587) for complete list. + + http://git-wip-us.apache.org/repos/asf/systemml-website/blob/49fa497e/_src/roadmap.html -- diff --git a/_src/roadmap.html b/_src/roadmap.html index 127ea16..b0a2681 100644 --- a/_src/roadmap.html +++ b/_src/roadmap.html @@ -72,6 +72,35 @@ limitations under the License. Current Release + SystemML 0.15.0 (released in September, 2017) + https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319522&version=12341587";>details + +Added several new 2D convolution layers +Graduated `nn` library from staging to `scripts/nn` + + + + Experimental Features + +Expanded Code Generation for broader performance improvements +Enhanced GPU support and scalability + + + + Removals + +Removed file-based transform +Removed original MLContext API + +
svn commit: r1808290 - in /systemml/site/docs/0.15.0: ./ api/ api/java/ api/java/org/ api/java/org/apache/ api/java/org/apache/sysml/ api/java/org/apache/sysml/api/ api/java/org/apache/sysml/api/class
Author: gweidner Date: Wed Sep 13 21:05:25 2017 New Revision: 1808290 URL: http://svn.apache.org/viewvc?rev=1808290&view=rev Log: Add 0.15.0 docs to website [This commit notification would consist of 264 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]
svn commit: r1808306 - in /systemml/site: documentation.html download.html release-notes/systemml-release-notes-0.15.0.html roadmap.html
Author: gweidner Date: Thu Sep 14 04:00:10 2017 New Revision: 1808306 URL: http://svn.apache.org/viewvc?rev=1808306&view=rev Log: Switch download to 0.15.0, add docs link and release notes Added: systemml/site/release-notes/systemml-release-notes-0.15.0.html Modified: systemml/site/documentation.html systemml/site/download.html systemml/site/roadmap.html Modified: systemml/site/documentation.html URL: http://svn.apache.org/viewvc/systemml/site/documentation.html?rev=1808306&r1=1808305&r2=1808306&view=diff == --- systemml/site/documentation.html (original) +++ systemml/site/documentation.html Thu Sep 14 04:00:10 2017 @@ -163,6 +163,7 @@ Documentation such as programming guides, reference guides, and API docs is available online for each version of SystemML below: http://apache.github.io/systemml/";>Latest +0.15.0 0.14.0 0.13.0 0.12.0 Modified: systemml/site/download.html URL: http://svn.apache.org/viewvc/systemml/site/download.html?rev=1808306&r1=1808305&r2=1808306&view=diff == --- systemml/site/download.html (original) +++ systemml/site/download.html Thu Sep 14 04:00:10 2017 @@ -163,7 +163,7 @@ Apache SystemML is released as a source artifact. Binary artifacts (including the ones available in Maven) are made available for your convenience. You will be prompted for a mirror - if the file is not found on yours, please be patient, as it may take 24 hours to reach all mirrors. Apache SystemML is distributed under the http://www.apache.org/licenses/LICENSE-2.0";>Apache License, version 2.0. - Download SystemML 0.14.0-incubating for Spark 2.0.2 and above + Download SystemML 0.15.0 for Spark 2.0.2 and above @@ -175,33 +175,33 @@ Signature -http://www.apache.org/dyn/closer.lua/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz"; target="_blank"> systemml-0.14.0-incubating (Binary tgz) -https://www.apache.org/dist/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz.md5";>MD5 -https://www.apache.org/dist/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz.asc";>ASC +http://www.apache.org/dyn/closer.lua/systemml/0.15.0/systemml-0.15.0-bin.tgz"; target="_blank"> systemml-0.15.0 (Binary tgz) +https://www.apache.org/dist/systemml/0.15.0/systemml-0.15.0-bin.tgz.md5";>MD5 +https://www.apache.org/dist/systemml/0.15.0/systemml-0.15.0-bin.tgz.asc";>ASC -http://www.apache.org/dyn/closer.lua/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip"; target="_blank">systemml-0.14.0-incubating (Binary zip) -https://www.apache.org/dist/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip.md5";>MD5 -https://www.apache.org/dist/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.zip.asc";>ASC +http://www.apache.org/dyn/closer.lua/systemml/0.15.0/systemml-0.15.0-bin.zip"; target="_blank">systemml-0.15.0 (Binary zip) +https://www.apache.org/dist/systemml/0.15.0/systemml-0.15.0-bin.zip.md5";>MD5 +https://www.apache.org/dist/systemml/0.15.0/systemml-0.15.0-bin.zip.asc";>ASC -http://www.apache.org/dyn/closer.lua/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz"; target="_blank">systemml-0.14.0-incubating (Source tgz) -https://www.apache.org/dist/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz.md5";>MD5 -https://www.apache.org/dist/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.tgz.asc";>ASC +http://www.apache.org/dyn/closer.lua/systemml/0.15.0/systemml-0.15.0-src.tgz"; target="_blank">systemml-0.15.0 (Source tgz) +https://www.apache.org/dist/systemml/0.15.0/systemml-0.15.0-src.tgz.md5";>MD5 +https://www.apache.org/dist/systemml/0.15.0/systemml-0.15.0-src.tgz.asc";>ASC -http://www.apache.org/dyn/closer.lua/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.zip"; target="_blank">systemml-0.14.0-incubating (Source zip) -https://www.apache.org/dist/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.zip.md5";>MD5 -https://www.apache.org/dist/systemml/0.14.0-incubating/systemml-0.14.0-incubating-src.zip.asc";>ASC +http://www.apache.org/dyn/closer.lua/systemml/0.15.0/systemml-0.15.0-src.zip"; ta
systemml-website git commit: [HOTFIX] Update install page to use project.release_version token
Repository: systemml-website Updated Branches: refs/heads/master 49fa497e8 -> 9c86a9602 [HOTFIX] Update install page to use project.release_version token Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/9c86a960 Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/9c86a960 Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/9c86a960 Branch: refs/heads/master Commit: 9c86a960214f744844f78cf6d323a77271cf63ed Parents: 49fa497 Author: Glenn Weidner Authored: Fri Sep 15 16:55:02 2017 -0700 Committer: Glenn Weidner Committed: Fri Sep 15 16:55:02 2017 -0700 -- _src/install-systemml.html | 16 1 file changed, 8 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/9c86a960/_src/install-systemml.html -- diff --git a/_src/install-systemml.html b/_src/install-systemml.html index c6972fa..06eae0a 100644 --- a/_src/install-systemml.html +++ b/_src/install-systemml.html @@ -74,23 +74,23 @@ limitations under the License. pip install systemml{% endhighlight %} 2) For more information, please see the SystemML project documentation: -http://systemml.apache.org/docs/0.14.0/index.html";>http://systemml.apache.org/docs/0.14.0/index.html -http://systemml.apache.org/docs/0.14.0/beginners-guide-python";>http://systemml.apache.org/docs/0.14.0/beginners-guide-python +http://systemml.apache.org/docs/{{ site.data.project.release_version }}/index.html">http://systemml.apache.org/docs/{{ site.data.project.release_version }}/index.html +http://systemml.apache.org/docs/{{ site.data.project.release_version }}/beginners-guide-python">http://systemml.apache.org/docs/{{ site.data.project.release_version }}/beginners-guide-python 1) Download Apache SystemML binary release (tgz or zip): -http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz";>http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz +http://www.apache.org/dyn/closer.lua/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.tgz">http://www.apache.org/dyn/closer.lua/systemml/{{ site.data.project.release_version }}/systemml-{{ site.data.project.release_version }}-bin.tgz 2) Extract binary release contents: -tar -xvzf systemml-0.14.0-incubating-bin.tgz +tar -xvzf systemml-{{ site.data.project.release_version }}-bin.tgz 3) Go to project root directory: -cd systemml-0.14.0-incubating-bin +cd systemml-{{ site.data.project.release_version }}-bin 4) Start Spark Shell with SystemML jar file: -spark-shell --executor-memory 4G --driver-memory 4G --jars lib/systemml-0.14.0-incubating.jar +spark-shell --executor-memory 4G --driver-memory 4G --jars lib/systemml-{{ site.data.project.release_version }}.jar 5) You're all set to run SystemML on Spark: @@ -104,8 +104,8 @@ ml.execute(helloScript) 6) For more information, please see the SystemML project documentation: -http://systemml.apache.org/docs/0.14.0/index.html";>http://systemml.apache.org/docs/0.14.0/index.html -http://systemml.apache.org/docs/0.14.0/spark-mlcontext-programming-guide";>http://systemml.apache.org/docs/0.14.0/spark-mlcontext-programming-guide +http://systemml.apache.org/docs/{{ site.data.project.release_version }}/index.html">http://systemml.apache.org/docs/{{ site.data.project.release_version }}/index.html +http://systemml.apache.org/docs/{{ site.data.project.release_version }}/spark-mlcontext-programming-guide">http://systemml.apache.org/docs/{{ site.data.project.release_version }}/spark-mlcontext-programming-guide
svn commit: r1808490 - /systemml/site/install-systemml.html
Author: gweidner Date: Sat Sep 16 00:04:51 2017 New Revision: 1808490 URL: http://svn.apache.org/viewvc?rev=1808490&view=rev Log: [HOTFIX] Update install page to use project.release_version token Modified: systemml/site/install-systemml.html Modified: systemml/site/install-systemml.html URL: http://svn.apache.org/viewvc/systemml/site/install-systemml.html?rev=1808490&r1=1808489&r2=1808490&view=diff == --- systemml/site/install-systemml.html (original) +++ systemml/site/install-systemml.html Sat Sep 16 00:04:51 2017 @@ -197,23 +197,23 @@ pip install systemml 2) For more information, please see the SystemML project documentation: -http://systemml.apache.org/docs/0.14.0/index.html";>http://systemml.apache.org/docs/0.14.0/index.html -http://systemml.apache.org/docs/0.14.0/beginners-guide-python";>http://systemml.apache.org/docs/0.14.0/beginners-guide-python +http://systemml.apache.org/docs/0.15.0/index.html";>http://systemml.apache.org/docs/0.15.0/index.html +http://systemml.apache.org/docs/0.15.0/beginners-guide-python";>http://systemml.apache.org/docs/0.15.0/beginners-guide-python 1) Download Apache SystemML binary release (tgz or zip): -http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz";>http://www.apache.org/dyn/closer.lua/incubator/systemml/0.14.0-incubating/systemml-0.14.0-incubating-bin.tgz +http://www.apache.org/dyn/closer.lua/systemml/0.15.0/systemml-0.15.0-bin.tgz";>http://www.apache.org/dyn/closer.lua/systemml/0.15.0/systemml-0.15.0-bin.tgz 2) Extract binary release contents: -tar -xvzf systemml-0.14.0-incubating-bin.tgz +tar -xvzf systemml-0.15.0-bin.tgz 3) Go to project root directory: -cd systemml-0.14.0-incubating-bin +cd systemml-0.15.0-bin 4) Start Spark Shell with SystemML jar file: -spark-shell --executor-memory 4G --driver-memory 4G --jars lib/systemml-0.14.0-incubating.jar +spark-shell --executor-memory 4G --driver-memory 4G --jars lib/systemml-0.15.0.jar 5) You're all set to run SystemML on Spark: @@ -227,8 +227,8 @@ ml.execute(helloScript) 6) For more information, please see the SystemML project documentation: -http://systemml.apache.org/docs/0.14.0/index.html";>http://systemml.apache.org/docs/0.14.0/index.html -http://systemml.apache.org/docs/0.14.0/spark-mlcontext-programming-guide";>http://systemml.apache.org/docs/0.14.0/spark-mlcontext-programming-guide +http://systemml.apache.org/docs/0.15.0/index.html";>http://systemml.apache.org/docs/0.15.0/index.html +http://systemml.apache.org/docs/0.15.0/spark-mlcontext-programming-guide";>http://systemml.apache.org/docs/0.15.0/spark-mlcontext-programming-guide
systemml git commit: [SYSTEMML-1907] Rename python package to systemml-*-python.tar.gz
Repository: systemml Updated Branches: refs/heads/master 6ea9b33b4 -> 2e6b577c5 [SYSTEMML-1907] Rename python package to systemml-*-python.tar.gz Switched Python artifact to tar.gz for PyPi upload compatibility with twine. Closes #666. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/2e6b577c Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/2e6b577c Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/2e6b577c Branch: refs/heads/master Commit: 2e6b577c513393022f87e4770d7761a3726a07aa Parents: 6ea9b33 Author: Glenn Weidner Authored: Sat Sep 16 14:53:55 2017 -0700 Committer: Glenn Weidner Committed: Sat Sep 16 14:53:55 2017 -0700 -- dev/release/release-build.sh| 6 +- dev/release/release-verify.sh | 12 +-- dev/release/src/test/bin/verifyBuild.sh | 6 +- .../org/apache/sysml/validation/Constants.java | 1 + .../sysml/validation/ValidateLicAndNotice.java | 82 ++-- docs/beginners-guide-python.md | 4 +- pom.xml | 4 +- src/main/python/post_setup.py | 2 +- 8 files changed, 60 insertions(+), 57 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/2e6b577c/dev/release/release-build.sh -- diff --git a/dev/release/release-build.sh b/dev/release/release-build.sh index 120664b..6142e72 100755 --- a/dev/release/release-build.sh +++ b/dev/release/release-build.sh @@ -289,11 +289,11 @@ if [[ "$RELEASE_PREPARE" == "true" ]]; then cd svn-release-staging/$RELEASE_VERSION-$RELEASE_RC/ rm -f *.asc -for i in *.zip *.tgz; do gpg --output $i.asc --detach-sig --armor $i; done +for i in *.zip *.tgz *.tar.gz; do gpg --output $i.asc --detach-sig --armor $i; done rm -f *.md5 -for i in *.zip *.tgz; do openssl md5 -hex $i | sed 's/MD5(\([^)]*\))= \([0-9a-f]*\)/\2 *\1/' > $i.md5; done +for i in *.zip *.tgz *.tar.gz; do openssl md5 -hex $i | sed 's/MD5(\([^)]*\))= \([0-9a-f]*\)/\2 *\1/' > $i.md5; done rm -f *.sha512 -for i in *.zip *.tgz; do shasum -a 512 $i > $i.sha512; done +for i in *.zip *.tgz *.tar.gz; do shasum -a 512 $i > $i.sha512; done cd .. #exit $RELEASE_VERSION-$RELEASE_RC/ http://git-wip-us.apache.org/repos/asf/systemml/blob/2e6b577c/dev/release/release-verify.sh -- diff --git a/dev/release/release-verify.sh b/dev/release/release-verify.sh index 6071ae4..f53604c 100755 --- a/dev/release/release-verify.sh +++ b/dev/release/release-verify.sh @@ -36,15 +36,15 @@ This will compile the utility source code which is not on regular source code pa --verifyAll <--tag="Code based on Git tag will be validated."> [--workDir="Directory where output files will be created] This will verify license, notice and binary files. ---verifyLic [--distDir="Directory Containing zip/tgz files] -This will verify license, notice in zip/tgz files. +--verifyLic [--distDir="Directory Containing zip/tgz/tar.gz files] +This will verify license, notice in zip/tgz/tar.gz files. --verifyBin <--tag="Code based on Git tag will be validated."> [--workDir="Directory where output files will be created] This will verify binary distribution files for runtime correctness. OPTIONS ---distDir- Directory containing release artifacts (zip/tzg) files. +--distDir- Directory containing release artifacts (zip/tgz/tar.gz) files. --workDir - Directory where output files will be created. @@ -73,7 +73,7 @@ EOF set -e if [ $# -eq 0 ]; then - echo "`date +%Y-%m-%dT%H:%M:%S`: ERROR: Insuffient parameters passed."; + echo "`date +%Y-%m-%dT%H:%M:%S`: ERROR: Insufficient parameters passed."; exit_with_usage fi @@ -160,7 +160,7 @@ if [[ "$COMPILE_CODE" == "true" ]]; then javac -Xlint:unchecked -classpath ../../../../..//target/lib/commons-compress-1.4.1.jar:../../../../..//target/lib/commons-io-2.4.jar:. org/apache/sysml/validation/*.java -cd "$ORIG_DIR" # Return to directoryt from it was called. +cd "$ORIG_DIR" # Return to directory from it was called. exit 0 fi @@ -180,7 +180,7 @@ if [[ "$BIN_VERIFY" == "true" ]]; then fi if [[ "$LIC_NOTICE_VERIFY" == "true" ]]; then -echo "`date +%Y-%m-%dT%H:%M:%S`: INFO: Verifying license and notices from zip/tgz files..." +echo "`date +%Y-%m-%dT%H:%M:%S`: INFO: Verifying license and notices from zip/tgz/tar.gz files..." java -classpath ../../../../..//target/lib/commons-compress-1.4.1.jar:../../../../..//target/lib/commons-io-2.4.jar:. org/apache/sysml/validation/ValidateLicAndNotice $DIST_DIR
systemml git commit: [SYSTEMML-1896] Update python tests to use getOrCreate
Repository: systemml Updated Branches: refs/heads/master 2e6b577c5 -> 60d20c2b7 [SYSTEMML-1896] Update python tests to use getOrCreate Closes #657. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/60d20c2b Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/60d20c2b Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/60d20c2b Branch: refs/heads/master Commit: 60d20c2b7720feba05009f1f2ca5169e50059b98 Parents: 2e6b577 Author: Glenn Weidner Authored: Sat Sep 16 15:05:24 2017 -0700 Committer: Glenn Weidner Committed: Sat Sep 16 15:05:24 2017 -0700 -- src/main/python/tests/test_matrix_agg_fn.py| 2 +- src/main/python/tests/test_matrix_binary_op.py | 2 +- src/main/python/tests/test_mlcontext.py| 2 +- src/main/python/tests/test_mllearn_df.py | 2 -- src/main/python/tests/test_mllearn_numpy.py| 3 --- 5 files changed, 3 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/60d20c2b/src/main/python/tests/test_matrix_agg_fn.py -- diff --git a/src/main/python/tests/test_matrix_agg_fn.py b/src/main/python/tests/test_matrix_agg_fn.py index be3df14..8ff026d 100644 --- a/src/main/python/tests/test_matrix_agg_fn.py +++ b/src/main/python/tests/test_matrix_agg_fn.py @@ -34,7 +34,7 @@ import systemml as sml import numpy as np from scipy.stats import kurtosis, skew, moment from pyspark.context import SparkContext -sc = SparkContext() +sc = SparkContext.getOrCreate() dim = 5 m1 = np.array(np.random.randint(100, size=dim*dim) + 1.01, dtype=np.double) http://git-wip-us.apache.org/repos/asf/systemml/blob/60d20c2b/src/main/python/tests/test_matrix_binary_op.py -- diff --git a/src/main/python/tests/test_matrix_binary_op.py b/src/main/python/tests/test_matrix_binary_op.py index 6bba3e9..db5f736 100644 --- a/src/main/python/tests/test_matrix_binary_op.py +++ b/src/main/python/tests/test_matrix_binary_op.py @@ -33,7 +33,7 @@ import unittest import systemml as sml import numpy as np from pyspark.context import SparkContext -sc = SparkContext() +sc = SparkContext.getOrCreate() dim = 5 m1 = np.array(np.random.randint(100, size=dim*dim) + 1.01, dtype=np.double) http://git-wip-us.apache.org/repos/asf/systemml/blob/60d20c2b/src/main/python/tests/test_mlcontext.py -- diff --git a/src/main/python/tests/test_mlcontext.py b/src/main/python/tests/test_mlcontext.py index 9ba3790..e0db346 100644 --- a/src/main/python/tests/test_mlcontext.py +++ b/src/main/python/tests/test_mlcontext.py @@ -36,7 +36,7 @@ from pyspark.context import SparkContext from systemml import MLContext, dml, pydml -sc = SparkContext() +sc = SparkContext.getOrCreate() ml = MLContext(sc) class TestAPI(unittest.TestCase): http://git-wip-us.apache.org/repos/asf/systemml/blob/60d20c2b/src/main/python/tests/test_mllearn_df.py -- diff --git a/src/main/python/tests/test_mllearn_df.py b/src/main/python/tests/test_mllearn_df.py index d949f4e..c2f8a3e 100644 --- a/src/main/python/tests/test_mllearn_df.py +++ b/src/main/python/tests/test_mllearn_df.py @@ -33,7 +33,6 @@ sys.path.insert(0, path) import unittest import numpy as np -from pyspark.context import SparkContext from pyspark.ml import Pipeline from pyspark.ml.feature import HashingTF, Tokenizer from pyspark.sql import SparkSession @@ -44,7 +43,6 @@ from sklearn import linear_model from sklearn.metrics import accuracy_score, r2_score from systemml.mllearn import LinearRegression, LogisticRegression, NaiveBayes, SVM -sc = SparkContext() sparkSession = SparkSession.builder.getOrCreate() # Currently not integrated with JUnit test http://git-wip-us.apache.org/repos/asf/systemml/blob/60d20c2b/src/main/python/tests/test_mllearn_numpy.py -- diff --git a/src/main/python/tests/test_mllearn_numpy.py b/src/main/python/tests/test_mllearn_numpy.py index 2a1bc3c..884dd36 100644 --- a/src/main/python/tests/test_mllearn_numpy.py +++ b/src/main/python/tests/test_mllearn_numpy.py @@ -33,7 +33,6 @@ sys.path.insert(0, path) import unittest import numpy as np -from pyspark.context import SparkContext from pyspark.ml import Pipeline from pyspark.ml.feature import HashingTF, Tokenizer from pyspark.sql import SparkSession @@ -44,9 +43,7 @@ from sklearn.metrics import accuracy_score, r2_score from systemml.mllearn import LinearRegression, LogisticRegression, NaiveBayes, SVM from sklearn import linear_model -sc = SparkContext() sparkSession = SparkSession.builder.getOrCreat
systemml git commit: [SYSTEMML-1907] Rename python package to systemml-*-python.tar.gz
Repository: systemml Updated Branches: refs/heads/gh-pages 288bc35c3 -> dbe9a03b2 [SYSTEMML-1907] Rename python package to systemml-*-python.tar.gz Switched Python artifact to tar.gz for PyPi upload compatibility with twine. Closes #666. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/dbe9a03b Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/dbe9a03b Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/dbe9a03b Branch: refs/heads/gh-pages Commit: dbe9a03b23a0614248890d38c12ea465cb3f3daa Parents: 288bc35 Author: Glenn Weidner Authored: Sat Sep 16 14:53:55 2017 -0700 Committer: Glenn Weidner Committed: Sat Sep 16 14:53:55 2017 -0700 -- beginners-guide-python.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/dbe9a03b/beginners-guide-python.md -- diff --git a/beginners-guide-python.md b/beginners-guide-python.md index 266d50f..173ea63 100644 --- a/beginners-guide-python.md +++ b/beginners-guide-python.md @@ -95,7 +95,7 @@ If you want to try out the bleeding edge version, please use following commands: git checkout https://github.com/apache/systemml.git cd systemml mvn clean package -P distribution -pip install target/systemml-1.0.0-SNAPSHOT-python.tgz +pip install target/systemml-1.0.0-SNAPSHOT-python.tar.gz ``` @@ -103,7 +103,7 @@ pip install target/systemml-1.0.0-SNAPSHOT-python.tgz git checkout https://github.com/apache/systemml.git cd systemml mvn clean package -P distribution -pip3 install target/systemml-1.0.0-SNAPSHOT-python.tgz +pip3 install target/systemml-1.0.0-SNAPSHOT-python.tar.gz ```
systemml-website git commit: [SYSTEMML-1907] Rename python package to systemml-*-python.tar.gz
Repository: systemml-website Updated Branches: refs/heads/master 9c86a9602 -> ada674608 [SYSTEMML-1907] Rename python package to systemml-*-python.tar.gz Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/ada67460 Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/ada67460 Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/ada67460 Branch: refs/heads/master Commit: ada674608be074f4c5f64a7fdd8ac39731544a95 Parents: 9c86a96 Author: Glenn Weidner Authored: Sun Sep 17 09:13:23 2017 -0700 Committer: Glenn Weidner Committed: Sun Sep 17 09:13:23 2017 -0700 -- _src/install-systemml.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/ada67460/_src/install-systemml.html -- diff --git a/_src/install-systemml.html b/_src/install-systemml.html index 06eae0a..c3f7820 100644 --- a/_src/install-systemml.html +++ b/_src/install-systemml.html @@ -112,7 +112,7 @@ ml.execute(helloScript) 1) Install python development build of SystemML: {% highlight bash %} -pip install https://sparktc.ibmcloud.com/repo/latest/systemml-1.0.0-SNAPSHOT-python.tgz{% endhighlight %} +pip install https://sparktc.ibmcloud.com/repo/latest/systemml-1.0.0-SNAPSHOT-python.tar.gz{% endhighlight %} 1) Download binary development build of SystemML (tgz or zip):
svn commit: r1808640 - /systemml/site/install-systemml.html
Author: gweidner Date: Sun Sep 17 16:21:45 2017 New Revision: 1808640 URL: http://svn.apache.org/viewvc?rev=1808640&view=rev Log: [SYSTEMML-1907] Rename python package to systemml-*-python.tar.gz Modified: systemml/site/install-systemml.html Modified: systemml/site/install-systemml.html URL: http://svn.apache.org/viewvc/systemml/site/install-systemml.html?rev=1808640&r1=1808639&r2=1808640&view=diff == --- systemml/site/install-systemml.html (original) +++ systemml/site/install-systemml.html Sun Sep 17 16:21:45 2017 @@ -234,7 +234,7 @@ ml.execute(helloScript) 1) Install python development build of SystemML: - pip install https://sparktc.ibmcloud.com/repo/latest/systemml-1.0.0-SNAPSHOT-python.tgz + pip install https://sparktc.ibmcloud.com/repo/latest/systemml-1.0.0-SNAPSHOT-python.tar.gz 1) Download binary development build of SystemML (tgz or zip):
systemml git commit: [SYSTEMML-1426] Support both ceil and ceiling built-in functions
Repository: systemml Updated Branches: refs/heads/master 43b573dfb -> 6a75104e5 [SYSTEMML-1426] Support both ceil and ceiling built-in functions Added support for ceiling built-in function while keeping existing ceil function with same internal compiler/runtime constants and opcode. Also included unit tests for ceiling. Closes #674. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/6a75104e Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/6a75104e Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/6a75104e Branch: refs/heads/master Commit: 6a75104e5e870b3dd69bcad493bd4fe9fadd7c98 Parents: 43b573d Author: Glenn Weidner Authored: Fri Sep 29 10:28:15 2017 -0700 Committer: Glenn Weidner Committed: Fri Sep 29 10:28:15 2017 -0700 -- docs/dml-language-reference.md | 2 +- .../RewriteAlgebraicSimplificationStatic.java | 4 +- .../sysml/parser/BuiltinFunctionExpression.java | 2 +- src/main/python/systemml/defmatrix.py | 5 +- .../functions/unary/matrix/RoundTest.java | 64 +++- .../scripts/functions/unary/matrix/Ceiling.R| 33 ++ .../scripts/functions/unary/matrix/Ceiling.dml | 25 7 files changed, 129 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/6a75104e/docs/dml-language-reference.md -- diff --git a/docs/dml-language-reference.md b/docs/dml-language-reference.md index c402acc..58ca306 100644 --- a/docs/dml-language-reference.md +++ b/docs/dml-language-reference.md @@ -821,7 +821,7 @@ is same as Function | Description | Parameters | Example | --- | -- | --- -exp(), log(), abs(), sqrt(), round(), floor(), ceil() | Apply mathematical function on input (cell wise if input is matrix) | Input: (), or ( ) Output: , or | sqrt(X) log(X,y) round(X) floor(X) ceil(X) +exp(), log(), abs(), sqrt(), round(), floor(), ceil(), ceiling() | Apply mathematical function on input (cell wise if input is matrix) | Input: ( ), or ( ) Output: , or | sqrt(X) log(X,y) round(X) floor(X) ceil(X) ceiling(X) sin(), cos(), tan(), sinh(), cosh(), tanh(), asin(), acos(), atan() | Apply trigonometric function on input (cell wise if input is matrix) | Input: ( ), or ( ) Output: , or | sin(X) sign() | Returns a matrix representing the signs of the input matrix elements, where 1 represents positive, 0 represents zero, and -1 represents negative | Input : (A ) Output : | A = matrix("-5 0 3 -3", rows=2, cols=2) signA = sign(A)Matrix signA: [[-1, 0], [1, -1]] http://git-wip-us.apache.org/repos/asf/systemml/blob/6a75104e/src/main/java/org/apache/sysml/hops/rewrite/RewriteAlgebraicSimplificationStatic.java -- diff --git a/src/main/java/org/apache/sysml/hops/rewrite/RewriteAlgebraicSimplificationStatic.java b/src/main/java/org/apache/sysml/hops/rewrite/RewriteAlgebraicSimplificationStatic.java index eadf492..d100989 100644 --- a/src/main/java/org/apache/sysml/hops/rewrite/RewriteAlgebraicSimplificationStatic.java +++ b/src/main/java/org/apache/sysml/hops/rewrite/RewriteAlgebraicSimplificationStatic.java @@ -170,7 +170,7 @@ public class RewriteAlgebraicSimplificationStatic extends HopRewriteRule hi = simplifyOrderedSort(hop, hi, i); //e.g., order(matrix())->seq; hi = removeUnnecessaryReorgOperation(hop, hi, i); //e.g., t(t(X))->X; rev(rev(X))->X potentially introduced by other rewrites hi = simplifyTransposeAggBinBinaryChains(hop, hi, i);//e.g., t(t(A)%*%t(B)+C) -> B%*%A+t(C) - hi = removeUnnecessaryMinus(hop, hi, i); //e.g., -(-X)->X; potentially introduced by simplfiy binary or dyn rewrites + hi = removeUnnecessaryMinus(hop, hi, i); //e.g., -(-X)->X; potentially introduced by simplify binary or dyn rewrites hi = simplifyGroupedAggregate(hi); //e.g., aggregate(target=X,groups=y,fn="count") -> aggregate(target=y,groups=y,fn="count") if(OptimizerUtils.ALLOW_OPERATOR_FUSION) { hi = fuseMinusNzBinaryOperation(hop, hi, i); //e.g., X-mean*ppred(X,0,!=) -> X -nz mean @@ -340,7 +340,7 @@ public class RewriteAlgebraicSimplificationStatic extends HopRewriteRule * rand*7 -> rand(min*7,max*7); rand+7 -> rand(min+7,max+7); rand-7 -> rand(min+(-7),max+(-7)) * 7*rand -> rand(min*7,max*7); 7+rand -> rand(min+7,max+7);
systemml git commit: [SYSTEMML-1426] Support both ceil and ceiling built-in functions
Repository: systemml Updated Branches: refs/heads/gh-pages 91190d6df -> b5a248a4b [SYSTEMML-1426] Support both ceil and ceiling built-in functions Added support for ceiling built-in function while keeping existing ceil function with same internal compiler/runtime constants and opcode. Also included unit tests for ceiling. Closes #674. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/b5a248a4 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/b5a248a4 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/b5a248a4 Branch: refs/heads/gh-pages Commit: b5a248a4b420bb24b350bd905632aa7c6cc07b3d Parents: 91190d6 Author: Glenn Weidner Authored: Fri Sep 29 10:28:15 2017 -0700 Committer: Glenn Weidner Committed: Fri Sep 29 10:28:15 2017 -0700 -- dml-language-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/b5a248a4/dml-language-reference.md -- diff --git a/dml-language-reference.md b/dml-language-reference.md index c402acc..58ca306 100644 --- a/dml-language-reference.md +++ b/dml-language-reference.md @@ -821,7 +821,7 @@ is same as Function | Description | Parameters | Example | --- | -- | --- -exp(), log(), abs(), sqrt(), round(), floor(), ceil() | Apply mathematical function on input (cell wise if input is matrix) | Input: (), or ( ) Output: , or | sqrt(X) log(X,y) round(X) floor(X) ceil(X) +exp(), log(), abs(), sqrt(), round(), floor(), ceil(), ceiling() | Apply mathematical function on input (cell wise if input is matrix) | Input: ( ), or ( ) Output: , or | sqrt(X) log(X,y) round(X) floor(X) ceil(X) ceiling(X) sin(), cos(), tan(), sinh(), cosh(), tanh(), asin(), acos(), atan() | Apply trigonometric function on input (cell wise if input is matrix) | Input: ( ), or ( ) Output: , or | sin(X) sign() | Returns a matrix representing the signs of the input matrix elements, where 1 represents positive, 0 represents zero, and -1 represents negative | Input : (A ) Output : | A = matrix("-5 0 3 -3", rows=2, cols=2) signA = sign(A)Matrix signA: [[-1, 0], [1, -1]]
systemml git commit: [HOTFIX] Fix javadoc reference link to resolve distribution build error
Repository: systemml Updated Branches: refs/heads/master 6a75104e5 -> e4c2f9e57 [HOTFIX] Fix javadoc reference link to resolve distribution build error Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/e4c2f9e5 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/e4c2f9e5 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/e4c2f9e5 Branch: refs/heads/master Commit: e4c2f9e57c854cb3b7c6546e863f49589ed2d33f Parents: 6a75104 Author: Glenn Weidner Authored: Fri Sep 29 12:37:41 2017 -0700 Committer: Glenn Weidner Committed: Fri Sep 29 12:37:41 2017 -0700 -- .../matrix/data/LibMatrixCuDNNConvolutionAlgorithm.java| 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/e4c2f9e5/src/main/java/org/apache/sysml/runtime/matrix/data/LibMatrixCuDNNConvolutionAlgorithm.java -- diff --git a/src/main/java/org/apache/sysml/runtime/matrix/data/LibMatrixCuDNNConvolutionAlgorithm.java b/src/main/java/org/apache/sysml/runtime/matrix/data/LibMatrixCuDNNConvolutionAlgorithm.java index 363cf78..2243b58 100644 --- a/src/main/java/org/apache/sysml/runtime/matrix/data/LibMatrixCuDNNConvolutionAlgorithm.java +++ b/src/main/java/org/apache/sysml/runtime/matrix/data/LibMatrixCuDNNConvolutionAlgorithm.java @@ -94,7 +94,7 @@ public class LibMatrixCuDNNConvolutionAlgorithm implements java.lang.AutoCloseab * Factory method to get the algorithm wrapper for convolution forward * * @param gCtx a valid {@link GPUContext} -* @param instName the invoking instruction's name for record {@link Statistics}. +* @param instName the invoking instruction's name for record {@link org.apache.sysml.utils.Statistics}. * @param Nnumber of input images * @param Cnumber of channels * @param Hheight of each image @@ -145,7 +145,7 @@ public class LibMatrixCuDNNConvolutionAlgorithm implements java.lang.AutoCloseab * Factory method to get the algorithm wrapper for convolution backward filter * * @param gCtx a valid {@link GPUContext} -* @param instName the invoking instruction's name for record {@link Statistics}. +* @param instName the invoking instruction's name for record {@link org.apache.sysml.utils.Statistics}. * @param Nnumber of input images * @param Cnumber of channels * @param Hheight of each image @@ -198,7 +198,7 @@ public class LibMatrixCuDNNConvolutionAlgorithm implements java.lang.AutoCloseab * Factory method to get the algorithm wrapper for convolution backward data * * @param gCtx a valid {@link GPUContext} -* @param instName the invoking instruction's name for record {@link Statistics}. +* @param instName the invoking instruction's name for record {@link org.apache.sysml.utils.Statistics}. * @param Nnumber of input images * @param Cnumber of channels * @param Hheight of each image
systemml git commit: [SYSTEMML-1942] Enable conditional spark tests
Repository: systemml Updated Branches: refs/heads/master 5dce90b3b -> 0b51bb18c [SYSTEMML-1942] Enable conditional spark tests Removed condition to run multiple spark tests in functions.unary.matrix package. Closes #677. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/0b51bb18 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/0b51bb18 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/0b51bb18 Branch: refs/heads/master Commit: 0b51bb18c6dac5095ab28e613ab2ae97f5a5ae74 Parents: 5dce90b Author: Glenn Weidner Authored: Sun Oct 1 14:05:14 2017 -0700 Committer: Glenn Weidner Committed: Sun Oct 1 14:05:14 2017 -0700 -- .../unary/matrix/EigenFactorizeTest.java| 46 +++- .../functions/unary/matrix/LUFactorizeTest.java | 50 - .../unary/matrix/MatrixInverseTest.java | 40 +++ .../functions/unary/matrix/MinusTest.java | 41 +++ .../functions/unary/matrix/QRSolverTest.java| 37 --- .../functions/unary/matrix/RoundTest.java | 110 --- 6 files changed, 163 insertions(+), 161 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/0b51bb18/src/test/java/org/apache/sysml/test/integration/functions/unary/matrix/EigenFactorizeTest.java -- diff --git a/src/test/java/org/apache/sysml/test/integration/functions/unary/matrix/EigenFactorizeTest.java b/src/test/java/org/apache/sysml/test/integration/functions/unary/matrix/EigenFactorizeTest.java index a8c1b86..f64b734 100644 --- a/src/test/java/org/apache/sysml/test/integration/functions/unary/matrix/EigenFactorizeTest.java +++ b/src/test/java/org/apache/sysml/test/integration/functions/unary/matrix/EigenFactorizeTest.java @@ -20,7 +20,7 @@ package org.apache.sysml.test.integration.functions.unary.matrix; import org.junit.Test; - +import org.apache.sysml.api.DMLScript; import org.apache.sysml.api.DMLScript.RUNTIME_PLATFORM; import org.apache.sysml.runtime.matrix.MatrixCharacteristics; import org.apache.sysml.test.integration.AutomatedTestBase; @@ -28,7 +28,6 @@ import org.apache.sysml.test.integration.TestConfiguration; public class EigenFactorizeTest extends AutomatedTestBase { - private final static String TEST_NAME1 = "eigen"; private final static String TEST_DIR = "functions/unary/matrix/"; private static final String TEST_CLASS_DIR = TEST_DIR + EigenFactorizeTest.class.getSimpleName() + "/"; @@ -39,61 +38,48 @@ public class EigenFactorizeTest extends AutomatedTestBase private final static int numEigenValuesToEvaluate = 15; @Override - public void setUp() - { - addTestConfiguration( - TEST_NAME1, - new TestConfiguration(TEST_CLASS_DIR, TEST_NAME1, - new String[] { "D" }) ); + public void setUp() { + addTestConfiguration(TEST_NAME1, + new TestConfiguration(TEST_CLASS_DIR, TEST_NAME1, new String[] { "D" }) ); } @Test - public void testEigenFactorizeDenseCP() - { + public void testEigenFactorizeDenseCP() { runTestEigenFactorize( rows1, RUNTIME_PLATFORM.SINGLE_NODE ); } @Test - public void testEigenFactorizeDenseSP() - { - if(rtplatform == RUNTIME_PLATFORM.SPARK) + public void testEigenFactorizeDenseSP() { runTestEigenFactorize( rows1, RUNTIME_PLATFORM.SPARK ); } @Test - public void testEigenFactorizeDenseMR() - { + public void testEigenFactorizeDenseMR() { runTestEigenFactorize( rows1, RUNTIME_PLATFORM.HADOOP ); } @Test - public void testEigenFactorizeDenseHybrid() - { + public void testEigenFactorizeDenseHybrid() { runTestEigenFactorize( rows1, RUNTIME_PLATFORM.HYBRID ); } @Test - public void testLargeEigenFactorizeDenseCP() - { + public void testLargeEigenFactorizeDenseCP() { runTestEigenFactorize( rows2, RUNTIME_PLATFORM.SINGLE_NODE ); } @Test - public void testLargeEigenFactorizeDenseSP() - { - if(rtplatform == RUNTIME_PLATFORM.SPARK) + public void testLargeEigenFactorizeDenseSP() { runTestEigenFactorize( rows2, RUNTIME_PLATFORM.SPARK ); } @Test - public void testLargeEigenFactorizeDenseMR() - { + public void testLargeEigenFactorizeDenseMR() { runTestEigenFactorize( rows2, RUNTIME_PLATFORM.HADOOP );
systemml git commit: [SYSTEMML-1929] Update Spark parameters in sparkDML.sh and docs
Repository: systemml Updated Branches: refs/heads/master bfb30b3af -> 0505fd38c [SYSTEMML-1929] Update Spark parameters in sparkDML.sh and docs Closes #670. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/0505fd38 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/0505fd38 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/0505fd38 Branch: refs/heads/master Commit: 0505fd38c3191551a14c9b21314b0c3432b47e2f Parents: bfb30b3 Author: Glenn Weidner Authored: Sat Oct 7 17:22:18 2017 -0700 Committer: Glenn Weidner Committed: Sat Oct 7 17:22:18 2017 -0700 -- docs/algorithms-classification.md | 88 ++-- docs/algorithms-clustering.md | 28 +++ docs/algorithms-descriptive-statistics.md | 28 +++ docs/algorithms-matrix-factorization.md | 36 docs/algorithms-regression.md | 72 docs/algorithms-survival-analysis.md| 32 +++ docs/spark-batch-mode.md| 8 +- docs/spark-mlcontext-programming-guide.md | 4 +- .../examples/mnist_lenet_distrib_sgd-train.dml | 2 +- scripts/perftest/python/run_perftest.py | 1 - scripts/sparkDML.sh | 17 ++-- src/main/resources/scripts/sparkDML.sh | 17 ++-- 12 files changed, 169 insertions(+), 164 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/0505fd38/docs/algorithms-classification.md -- diff --git a/docs/algorithms-classification.md b/docs/algorithms-classification.md index 1895103..62e40e7 100644 --- a/docs/algorithms-classification.md +++ b/docs/algorithms-classification.md @@ -160,9 +160,9 @@ val prediction = model.transform(X_test_df) fmt=[format] -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f MultiLogReg.dml -config SystemML-config.xml @@ -331,9 +331,9 @@ prediction.show() Log=/user/ml/log.csv -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f MultiLogReg.dml -config SystemML-config.xml @@ -527,9 +527,9 @@ val model = svm.fit(X_train_df) fmt=[format] -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f l2-svm.dml -config SystemML-config.xml @@ -574,9 +574,9 @@ val prediction = model.transform(X_test_df) fmt=[format] -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f l2-svm-predict.dml -config SystemML-config.xml @@ -658,9 +658,9 @@ more details on the Python API. Log=/user/ml/Log.csv -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f l2-svm.dml -config SystemML-config.xml @@ -692,9 +692,9 @@ more details on the Python API. confusion=/user/ml/confusion.csv -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin
systemml git commit: [SYSTEMML-1929] Update Spark parameters in sparkDML.sh and docs
Repository: systemml Updated Branches: refs/heads/gh-pages b5a248a4b -> e184e966e [SYSTEMML-1929] Update Spark parameters in sparkDML.sh and docs Closes #670. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/e184e966 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/e184e966 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/e184e966 Branch: refs/heads/gh-pages Commit: e184e966e2103062e5381c11a48c9e2bb53b5687 Parents: b5a248a Author: Glenn Weidner Authored: Sat Oct 7 17:22:18 2017 -0700 Committer: Glenn Weidner Committed: Sat Oct 7 17:22:18 2017 -0700 -- algorithms-classification.md | 88 +++ algorithms-clustering.md | 28 +- algorithms-descriptive-statistics.md | 28 +- algorithms-matrix-factorization.md | 36 ++--- algorithms-regression.md | 72 - algorithms-survival-analysis.md | 32 +-- spark-batch-mode.md | 8 +-- spark-mlcontext-programming-guide.md | 4 +- 8 files changed, 148 insertions(+), 148 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/e184e966/algorithms-classification.md -- diff --git a/algorithms-classification.md b/algorithms-classification.md index 1895103..62e40e7 100644 --- a/algorithms-classification.md +++ b/algorithms-classification.md @@ -160,9 +160,9 @@ val prediction = model.transform(X_test_df) fmt=[format] -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f MultiLogReg.dml -config SystemML-config.xml @@ -331,9 +331,9 @@ prediction.show() Log=/user/ml/log.csv -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f MultiLogReg.dml -config SystemML-config.xml @@ -527,9 +527,9 @@ val model = svm.fit(X_train_df) fmt=[format] -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f l2-svm.dml -config SystemML-config.xml @@ -574,9 +574,9 @@ val prediction = model.transform(X_test_df) fmt=[format] -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f l2-svm-predict.dml -config SystemML-config.xml @@ -658,9 +658,9 @@ more details on the Python API. Log=/user/ml/Log.csv -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f l2-svm.dml -config SystemML-config.xml @@ -692,9 +692,9 @@ more details on the Python API. confusion=/user/ml/confusion.csv -$SPARK_HOME/bin/spark-submit --master yarn-cluster +$SPARK_HOME/bin/spark-submit --master yarn + --deploy-mode cluster --conf spark.driver.maxResultSize=0 - --conf spark.akka.frameSize=128 SystemML.jar -f l2
systemml-website git commit: [SYSTEMML-1929] Update Spark parameters on install page
Repository: systemml-website Updated Branches: refs/heads/master 573cda770 -> 6fc5bada5 [SYSTEMML-1929] Update Spark parameters on install page Removed unnecessary spark.akka.framesize configuration parameter from Python Jupyter command line on install page. Project: http://git-wip-us.apache.org/repos/asf/systemml-website/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml-website/commit/6fc5bada Tree: http://git-wip-us.apache.org/repos/asf/systemml-website/tree/6fc5bada Diff: http://git-wip-us.apache.org/repos/asf/systemml-website/diff/6fc5bada Branch: refs/heads/master Commit: 6fc5bada5c31aa7f602fb9266713ddbdd38d840e Parents: 573cda7 Author: Glenn Weidner Authored: Sun Oct 8 18:50:45 2017 -0700 Committer: Glenn Weidner Committed: Sun Oct 8 18:50:45 2017 -0700 -- _src/install-systemml.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/systemml-website/blob/6fc5bada/_src/install-systemml.html -- diff --git a/_src/install-systemml.html b/_src/install-systemml.html index c3f7820..6665363 100644 --- a/_src/install-systemml.html +++ b/_src/install-systemml.html @@ -135,7 +135,7 @@ pip install https://sparktc.ibmcloud.com/repo/latest/systemml-1.0.0-SNAPSHOT-pyt {% highlight bash %} # Start Jupyter Notebook Server -PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark --master local[*] --conf "spark.driver.memory=12g" --conf spark.driver.maxResultSize=0 --conf spark.akka.frameSize=128 --conf spark.default.parallelism=100 +PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark --master local[*] --conf "spark.driver.memory=12g" --conf spark.driver.maxResultSize=0 --conf spark.default.parallelism=100 {% endhighlight %}
svn commit: r1811524 - /systemml/site/install-systemml.html
Author: gweidner Date: Mon Oct 9 02:07:25 2017 New Revision: 1811524 URL: http://svn.apache.org/viewvc?rev=1811524&view=rev Log: [SYSTEMML-1929] Update Spark parameters on install page Modified: systemml/site/install-systemml.html Modified: systemml/site/install-systemml.html URL: http://svn.apache.org/viewvc/systemml/site/install-systemml.html?rev=1811524&r1=1811523&r2=1811524&view=diff == --- systemml/site/install-systemml.html (original) +++ systemml/site/install-systemml.html Mon Oct 9 02:07:25 2017 @@ -256,7 +256,7 @@ ml.execute(helloScript) # Start Jupyter Notebook Server -PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark --master local[*] --conf "spark.driver.memory=12g" --conf spark.driver.maxResultSize=0 --conf spark.akka.frameSize=128 --conf spark.default.parallelism=100 +PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark --master local[*] --conf "spark.driver.memory=12g" --conf spark.driver.maxResultSize=0 --conf spark.default.parallelism=100
systemml git commit: [MINOR][DOC] Update description of statistical built-in functions sd and var
Repository: systemml Updated Branches: refs/heads/master be3e0c993 -> 31bbeb5b2 [MINOR][DOC] Update description of statistical built-in functions sd and var Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/31bbeb5b Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/31bbeb5b Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/31bbeb5b Branch: refs/heads/master Commit: 31bbeb5b27359bdff387b5b5a202418344feb720 Parents: be3e0c9 Author: Glenn Weidner Authored: Tue Oct 10 10:46:00 2017 -0700 Committer: Glenn Weidner Committed: Tue Oct 10 10:46:00 2017 -0700 -- docs/dml-language-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/31bbeb5b/docs/dml-language-reference.md -- diff --git a/docs/dml-language-reference.md b/docs/dml-language-reference.md index 58ca306..e377596 100644 --- a/docs/dml-language-reference.md +++ b/docs/dml-language-reference.md @@ -683,7 +683,7 @@ as.double(), as.integer(), as.logical() | A variable is cast as the Function | Description | Parameters | Example | --- | -- | --- mean() avg() | Return the mean value of all cells in matrix | Input: matrix Output: scalar | mean(X) -var() sd() | Return the variance/stdDev value of all cells in matrix | Input: matrix Output: scalar | var(X) sd(X) +var() sd() | Return the variance/stdDev value of all cells in matrix. Both use unbiased estimators with (n-1) denominator. | Input: matrix Output: scalar | var(X) sd(X) moment() | Returns the kth central moment of values in a column matrix V, where k = 2, 3, or 4. It can be used to compute statistical measures like Variance, Kurtosis, and Skewness. This function also takes an optional weights parameter W. | Input: (X <(n x 1) matrix>, [W <(n x 1) matrix>),] k) Output: | A = rand(rows=10,cols=1, pdf="normal") print("Variance from our (standard normal) random generator is approximately " + moment(A,2)) colSums() colMeans() colVars() colSds() colMaxs() colMins() | Column-wise computations -- for each column, compute the sum/mean/variance/stdDev/max/min of cell values | Input: matrix Output: (1 x n) matrix | colSums(X) colMeans(X) colVars(X) colSds(X) colMaxs(X) colMins(X) cov() | Returns the covariance between two 1-dimensional column matrices X and Y. The function takes an optional weights parameter W. All column matrices X, Y, and W (when specified) must have the exact same dimension. | Input: (X <(n x 1) matrix>, Y <(n x 1) matrix> [, W <(n x 1) matrix>)]) Output: | cov(X,Y) cov(X,Y,W)
systemml git commit: [MINOR][DOC] Update description of statistical built-in functions sd and var
Repository: systemml Updated Branches: refs/heads/gh-pages e184e966e -> 3144d4b9d [MINOR][DOC] Update description of statistical built-in functions sd and var Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/3144d4b9 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/3144d4b9 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/3144d4b9 Branch: refs/heads/gh-pages Commit: 3144d4b9d520e278db5c9a830ef8d3c7583ca0ab Parents: e184e96 Author: Glenn Weidner Authored: Tue Oct 10 10:46:00 2017 -0700 Committer: Glenn Weidner Committed: Tue Oct 10 10:46:00 2017 -0700 -- dml-language-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/3144d4b9/dml-language-reference.md -- diff --git a/dml-language-reference.md b/dml-language-reference.md index 58ca306..e377596 100644 --- a/dml-language-reference.md +++ b/dml-language-reference.md @@ -683,7 +683,7 @@ as.double(), as.integer(), as.logical() | A variable is cast as the Function | Description | Parameters | Example | --- | -- | --- mean() avg() | Return the mean value of all cells in matrix | Input: matrix Output: scalar | mean(X) -var() sd() | Return the variance/stdDev value of all cells in matrix | Input: matrix Output: scalar | var(X) sd(X) +var() sd() | Return the variance/stdDev value of all cells in matrix. Both use unbiased estimators with (n-1) denominator. | Input: matrix Output: scalar | var(X) sd(X) moment() | Returns the kth central moment of values in a column matrix V, where k = 2, 3, or 4. It can be used to compute statistical measures like Variance, Kurtosis, and Skewness. This function also takes an optional weights parameter W. | Input: (X <(n x 1) matrix>, [W <(n x 1) matrix>),] k) Output: | A = rand(rows=10,cols=1, pdf="normal") print("Variance from our (standard normal) random generator is approximately " + moment(A,2)) colSums() colMeans() colVars() colSds() colMaxs() colMins() | Column-wise computations -- for each column, compute the sum/mean/variance/stdDev/max/min of cell values | Input: matrix Output: (1 x n) matrix | colSums(X) colMeans(X) colVars(X) colSds(X) colMaxs(X) colMins(X) cov() | Returns the covariance between two 1-dimensional column matrices X and Y. The function takes an optional weights parameter W. All column matrices X, Y, and W (when specified) must have the exact same dimension. | Input: (X <(n x 1) matrix>, Y <(n x 1) matrix> [, W <(n x 1) matrix>)]) Output: | cov(X,Y) cov(X,Y,W)
systemml git commit: [SYSTEMML-848] Create Release Creation Process doc
Repository: systemml Updated Branches: refs/heads/master 75b93f261 -> bc781fcef [SYSTEMML-848] Create Release Creation Process doc Closes #697. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/bc781fce Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/bc781fce Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/bc781fce Branch: refs/heads/master Commit: bc781fcef778bed9cb6bff686f841c3040297213 Parents: 75b93f2 Author: Arvind Surve Authored: Tue Nov 7 16:29:58 2017 -0800 Committer: Glenn Weidner Committed: Tue Nov 7 16:29:58 2017 -0800 -- docs/release-creation-process.md | 135 ++ docs/release-process.md | 9 +-- 2 files changed, 138 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/systemml/blob/bc781fce/docs/release-creation-process.md -- diff --git a/docs/release-creation-process.md b/docs/release-creation-process.md new file mode 100644 index 000..edde97e --- /dev/null +++ b/docs/release-creation-process.md @@ -0,0 +1,135 @@ +--- +layout: global +title: SystemML Release Creation Process +description: Description of the SystemML release build process. +displayTitle: SystemML Release Creation Process +--- + + +* This will become a table of contents (this text will be scraped). +{:toc} + + Release Creation Guidelines Documentation +Prerequisite: [Project release guidelines](https://github.com/SparkTC/development-guidelines/blob/master/project-release-guidelines.md) + + + +Tips to prepare and release the build + +Step 0.Minimum changes and verification to be done before release build process starts. + + 1. ReadMe update and âMust Fixâ changes are already in. + 2. Performance Test is passing for dataset size of 80GB and below. + +Step 1: Prepare the release + + # Extract latest code to a directory + + + # Go to dev/release directory + cd /dev/release + + 1.a. Dry Run (This is trial build, will not commit anything in repository) + + e.g. (On Master branch with release candidate rc1, release version 0.15.0, and next development version 1.0.0-SNAPSHOT) + ./release-build.sh --release-prepare --releaseVersion="0.15.0" --developmentVersion="1.0.0-SNAPSHOT" --releaseRc="rc1" --tag="v0.15.0-rc1" --dryRun + + e.g. (On branch-0.15 branch with release candidate rc2, release version 0.15.0, and next development version 0.15.1-SNAPSHOT) + ./release-build.sh --release-prepare --releaseVersion="0.15.0" --developmentVersion="0.15.1-SNAPSHOT" --releaseRc="rc2" --tag="v0.15.0-rc2" --gitCommitHash="branch-0.15" --dryRun + + + 1.b. License verification + ./release-verify.sh --verifyLic + + 1.c. Run command to do release prepare step (This will commit changes to the repository) + This is same as step 1.a, without âdryRun option. + e.g. (On the Master branch) + ./release-build.sh --release-prepare --releaseVersion="0.15.0" --developmentVersion="1.0.0-SNAPSHOT" --releaseRc="rc1" --tag="v0.15.0-rc1" + + e.g. (On the branch-0.15 branch) + ./release-build.sh --release-prepare --releaseVersion="0.15.0" --developmentVersion="0.15.1-SNAPSHOT" --releaseRc="rc2" --tag="v0.15.0-rc2" --gitCommitHash="branch-0.15" + + 1.d Verify the release + This will verify release on Mac Operating System (OS), assuming these steps are run on Mac OS. It will verify licenses, notice and all other required verification only on Mac OS. + Verification of licenses and notice is required only on one platform. + + ./release-verify.sh --verifyAll + + +Step 2: Publish the release + + e.g. + ./release-build.sh --release-publish --gitTag="v0.15.0-rc1" + + +Step 3: Close the release candidate build on Nexus site + +Visit [NexusRepository](https://repository.apache.org/#stagingRepositories) site. + + Find out SystemML under (Staging Repositories) link. It should be in Open State (status). Close it (button on top left to middle) with proper comment. + Once it completes copying, URL will be updated with maven location to be sent in mail. + +Step 4. Send mail for voting (dev PMC) (d...@systemml.apache.org) + +Please check [Project release guidelines](https://github.com/SparkTC/development-guidelines/blob/master/project-release-guidelines.md) +or previous mail thread for format/content of the mail. + +Step 5: Create a branch based on release to be released. + # Create a branch based on TAG + Syntax: git branch + e.g.git branch branch-0.15 v0.15.0-rc1 + + # Push a branch to master repository + Syntax: git push origin + (origin is https://git-wip-us.apache.org/repos/asf