[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-28 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35728342 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,392 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-9408] [PySpark] Refactor linalg.py to /...

2015-07-28 Thread MechCoder
GitHub user MechCoder opened a pull request: https://github.com/apache/spark/pull/7731 [SPARK-9408] [PySpark] Refactor linalg.py to /linalg I refactored linalg.py to a folder /linalg so that future updates like `distributed.py` can be made easily. You can merge this pull request

[GitHub] spark pull request: [SPARK-9408] [PySpark] [MLlib] Refactor linalg...

2015-07-28 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7731#issuecomment-125713665 That does not work as well. I tried locally (I pushed it still). from pyspark.mllib.linalg import Vectors Vectors.dense([0.0, 1.0]) Do you

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-28 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7538#issuecomment-125698751 Oh, by refactoring I had meant the `ModelSummary` trait which can be inherited by these. Also the question about exposing the `numBins parameter` . It might be good

[GitHub] spark pull request: [SPARK-9408] [PySpark] [MLlib] Refactor linalg...

2015-07-28 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7731#issuecomment-125704342 @mengxr This breaks code on my machine, but I can't figure out why. :( --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-9408] [PySpark] [MLlib] Refactor linalg...

2015-07-28 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7731#issuecomment-125743419 I added those but still it does not work. I also changed , but it should not matter (but did not push) pyUDT to pyspark.mllib.linalg.local.MatrixUDT

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35615927 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,392 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35616011 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,392 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35616017 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,392 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35616054 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,392 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35616121 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,392 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7554#issuecomment-125182863 Alright, I made some changes to the design. The main problem was: RowMatrix initialization needs rows, numRows, numCols for creating a RowMatrix. However

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35569007 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35569987 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35572212 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35573201 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35570323 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35572067 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7554#issuecomment-125308486 Oh and would you want to change the project structure yourself or do you want to handle it yourself? --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7554#issuecomment-125307020 That's it from my side. I leave the discussion about conversion to and from different matrices to @mengxr --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35570745 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35572557 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35571749 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35572488 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35573286 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-27 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35570892 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-25 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7538#issuecomment-124853471 okay, thanks :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-25 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7538#issuecomment-124849911 @feynmanliang any news on this? thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-07-24 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/5748#discussion_r35439383 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -484,8 +480,9 @@ class Word2VecModel private[spark] ( * @return

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-07-24 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/5748#discussion_r35439718 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/feature/Word2VecSuite.scala --- @@ -37,6 +37,12 @@ class Word2VecSuite extends SparkFunSuite

[GitHub] spark pull request: [SPARK-9222] [MLlib] Make class instantiation ...

2015-07-24 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7573#issuecomment-124608830 I noticed that there are todo's in the LDAModel.scala file. Would you want me to fix them up, because it would be a good exercise? --- If your project is set up

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-07-24 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/5748#issuecomment-124580801 jenkins my friend. retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-07-24 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/5748#issuecomment-124600254 done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-07-24 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/5748#discussion_r35440919 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -484,8 +480,9 @@ class Word2VecModel private[spark] ( * @return

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-07-24 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/5748#discussion_r35437614 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -431,36 +422,41 @@ class Word2Vec extends Serializable with Logging

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-07-24 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/5748#issuecomment-124665485 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9222] [MLlib] Make class instantiation ...

2015-07-23 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7573#issuecomment-124336634 rebased and pushed by force --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-23 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7554#issuecomment-124122799 I have submitted a pull request across your branch, to remove `DistributedMatrices`. Would be great if we could take the discussion there (for now) --- If your

[GitHub] spark pull request: [SPARK-5991] [PySpark] [MLlib] Support model s...

2015-07-23 Thread MechCoder
GitHub user MechCoder opened a pull request: https://github.com/apache/spark/pull/7617 [SPARK-5991] [PySpark] [MLlib] Support model save/load in GMM This PR introduces save / load for GMM's in python API. Also I refactored `GaussianMixtureModel` and inherited it from

[GitHub] spark pull request: [SPARK-9223] [PySpark] [MLlib] Support model s...

2015-07-22 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7587#issuecomment-123783668 Jenkins my friend, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35239673 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35241707 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35235815 --- Diff: docs/mllib-data-types.md --- @@ -372,6 +372,29 @@ long m = mat.numRows(); long n = mat.numCols(); {% endhighlight %} /div

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35242716 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35236115 --- Diff: docs/mllib-data-types.md --- @@ -372,6 +372,29 @@ long m = mat.numRows(); long n = mat.numCols(); {% endhighlight %} /div

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35237840 --- Diff: docs/mllib-data-types.md --- @@ -431,7 +454,42 @@ long n = mat.numCols(); // Drop its row indices. RowMatrix rowMat = mat.toRowMatrix

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35241654 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35241549 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -1105,6 +1108,59 @@ private[python] class PythonMLLibAPI

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35245705 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35235476 --- Diff: docs/mllib-data-types.md --- @@ -372,6 +372,29 @@ long m = mat.numRows(); long n = mat.numCols(); {% endhighlight %} /div

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35240273 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35245404 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35239289 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7554#issuecomment-123825799 @dusenberrymw Thanks for your work! I have made some very minor comments. Also should the structure be `from pyspark.mllib.linalg.distributed import x` ? instead

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35249096 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35253870 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -1105,6 +1108,59 @@ private[python] class PythonMLLibAPI

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35253293 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35236567 --- Diff: docs/mllib-data-types.md --- @@ -431,7 +454,42 @@ long n = mat.numCols(); // Drop its row indices. RowMatrix rowMat = mat.toRowMatrix

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35237175 --- Diff: docs/mllib-data-types.md --- @@ -431,7 +454,42 @@ long n = mat.numCols(); // Drop its row indices. RowMatrix rowMat = mat.toRowMatrix

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35244034 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-22 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7554#discussion_r35252614 --- Diff: python/pyspark/mllib/linalg.py --- @@ -1152,9 +1156,385 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values): return

[GitHub] spark pull request: [SPARK-9223] [PySpark] [MLlib] Support model s...

2015-07-22 Thread MechCoder
GitHub user MechCoder opened a pull request: https://github.com/apache/spark/pull/7587 [SPARK-9223] [PySpark] [MLlib] Support model save/load in LDA Since save / load has been merged in LDA, it takes no time to write the wrappers in Python as well. You can merge this pull request

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-21 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/6948#issuecomment-123395868 Sounds good, --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-21 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/6948#issuecomment-123409119 Will follow up with both the PR's tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9222] [MLlib] Make class instantiation ...

2015-07-21 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7573#issuecomment-123559909 jenkins, my friend retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9222] [MLlib] Make class instantiation ...

2015-07-21 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7573#issuecomment-123463280 @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-9222] [MLlib] Make class instantiation ...

2015-07-21 Thread MechCoder
GitHub user MechCoder opened a pull request: https://github.com/apache/spark/pull/7573 [SPARK-9222] [MLlib] Make class instantiation variables in DistributedLDAModel private[clustering] This makes it easier to test all the class variables of the DistributedLDAmodel. You can merge

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-21 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7538#discussion_r35087353 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -407,6 +449,60 @@ private[classification] class

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-21 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7538#issuecomment-123221795 Thanks a lot for your kind reviews :) Maybe it might make sense to make a HasSummary trait with the hasSummary and setSummary Yes, indeed. Where

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-21 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7538#issuecomment-123222001 Btw, I assume it had been decided not to add hinge loss, log loss etc ? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-21 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7538#discussion_r35083990 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -252,7 +254,13 @@ class LogisticRegression(override val

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-21 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7538#issuecomment-123304015 I've addressed your comments about the dataframe storage. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-21 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7538#discussion_r35098464 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -286,6 +294,40 @@ class LogisticRegressionModel private

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-21 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7538#discussion_r35099192 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -407,6 +449,60 @@ private[classification] class

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-20 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7538#discussion_r34998173 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -407,6 +449,60 @@ private[classification] class

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-20 Thread MechCoder
GitHub user MechCoder opened a pull request: https://github.com/apache/spark/pull/7538 [SPARK-9112] [ML] Implement Stats for LogisticRegression I have added support for stats in LogisticRegression. The API is similar to that of LinearRegression

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-20 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7538#issuecomment-122898117 @feynmanliang --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-20 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7538#discussion_r34998300 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -407,6 +449,60 @@ private[classification] class

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-20 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7538#discussion_r34998302 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -407,6 +449,60 @@ private[classification] class

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-20 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7538#discussion_r34998365 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -701,4 +701,19 @@ class LogisticRegressionSuite

[GitHub] spark pull request: [SPARK-6485] [MLlib] [Python] Add CoordinateMa...

2015-07-20 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7554#issuecomment-123168892 I'll try to give a first pass later today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9112] [ML] Implement Stats for Logistic...

2015-07-20 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7538#discussion_r35071668 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -286,6 +294,40 @@ class LogisticRegressionModel private

[GitHub] spark pull request: [SPARK-8996] [MLlib] [PySpark] Python API for ...

2015-07-18 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7430#issuecomment-122596147 done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-17 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/6948#issuecomment-122254893 @jkbradley I will not be available till Tuesday, so if you have any minor comments feel free to address them and commit. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-8996] [MLlib] [PySpark] Python API for ...

2015-07-17 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7430#issuecomment-122246592 I've addressed your comments. please review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8996] [MLlib] [PySpark] Python API for ...

2015-07-16 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7430#issuecomment-122036351 Hmm. I did try that out, but there was some problem with python to java conversion. I'll have a deeper look. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-16 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/6948#discussion_r34759686 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala --- @@ -184,6 +199,82 @@ class LocalLDAModel private[clustering

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-16 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/6948#discussion_r34759954 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala --- @@ -184,6 +199,82 @@ class LocalLDAModel private[clustering

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-16 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/6948#discussion_r34760895 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala --- @@ -184,6 +199,82 @@ class LocalLDAModel private[clustering

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-16 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/6948#issuecomment-121856660 should look ok now, I think --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-16 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/6948#issuecomment-121861219 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8996] [MLlib] [PySpark] Python API for ...

2015-07-16 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7430#issuecomment-122043920 It would also be really helpful if you could give a quick pass over the rest of the PR --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-8996] [MLlib] [PySpark] Python API for ...

2015-07-16 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/7430#discussion_r34861716 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -1093,6 +1093,27 @@ private[python] class PythonMLLibAPI

[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...

2015-07-15 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121534525 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-15 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/6948#discussion_r34731112 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala --- @@ -184,6 +199,81 @@ class LocalLDAModel private[clustering

[GitHub] spark pull request: [SPARK-8996] [MLlib] [PySpark] Python API for ...

2015-07-15 Thread MechCoder
GitHub user MechCoder opened a pull request: https://github.com/apache/spark/pull/7430 [SPARK-8996] [MLlib] [PySpark] Python API for Kolmogorov-Smirnov Test Python API for the KS-test Statistics.kolmogorovSmirnovTest(data, distName, *params) I'm not quite sure how

[GitHub] spark pull request: [SPARK-5989] [MLlib] Model save/load for LDA

2015-07-15 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/6948#issuecomment-121767752 @jkbradley done ! btw is there a preferred way to manually inspect the Parquet files generated? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...

2015-07-15 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121662258 thanks for the review ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...

2015-07-14 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121179494 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8706] [PySpark] [Project infra] Add pyl...

2015-07-13 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/7241#issuecomment-121131134 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

<    1   2   3   4   5   6   7   8   9   10   >