Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-155148314
@yu-iskw Thanks for this! Quick request: Could you please send a little
follow-up PR to document (in the Python doc) what is being returned?
---
If your project is
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-155175314
@jkbradley sure!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-155232384
@jkbradley I send the PR at https://github.com/apache/spark/pull/9577.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154585403
@jkbradley @davies could you review it? I modified the type conversion
using `SerDe.dumps`.
---
If your project is set up for it, you can reply to this email and have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154632203
**[Test build #2002 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2002/consoleFull)**
for PR 8643 at commit
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154629369
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154643744
Thank you for merging it and your great support!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8643
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44207597
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -773,10 +776,10 @@ def train(cls, rdd, k=10, maxIterations=20,
docConcentration=-1.0,
:param
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44207604
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -745,9 +749,8 @@ def load(cls, sc, path):
raise TypeError("sc should be a SparkContext,
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154615987
LGTM, but a few minor comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154618704
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154642699
LGTM, merging this into master and 1.6 branch, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154617157
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154640453
**[Test build #2002 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2002/consoleFull)**
for PR 8643 at commit
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154640689
@davies thanks for the review. I fixed them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44207559
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -687,9 +687,14 @@ class LDAModel(JavaModelWrapper):
... [2, SparseVector(2, {0: 1.0})],
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44172629
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44164309
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44168908
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44169316
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44171965
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154526569
Build started sha1 is merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154526739
**[Test build #45250 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45250/consoleFull)**
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154527979
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45250/
---
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154527959
**[Test build #45250 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45250/consoleFull)**
for PR 8643 at commit
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r44185186
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154526527
Build triggered. sha1 is merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154529620
Build started sha1 is merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154546841
**[Test build #45258 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45258/consoleFull)**
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154529543
Build triggered. sha1 is merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154531334
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45252/
---
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154532750
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154537228
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user shaneknapp commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154543591
jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154544425
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154544398
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154558189
**[Test build #45258 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45258/consoleFull)**
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-154558405
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43723078
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43723787
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43823836
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43657655
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43658405
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43720867
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user yu-iskw commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43722360
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43720929
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -690,6 +690,21 @@ class LDAModel(JavaModelWrapper):
>>> model = LDA.train(rdd, k=2)
>>>
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8643#discussion_r43720248
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala ---
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153237999
**[Test build #44881 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44881/consoleFull)**
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153247346
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153247107
**[Test build #44881 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44881/consoleFull)**
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153247342
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153247670
I reverted DataFrame serialization to Java Any types.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152976265
Build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152976149
**[Test build #44799 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44799/consoleFull)**
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152976267
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153237504
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-153237516
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152938719
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152938736
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-152939997
**[Test build #44799 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44799/consoleFull)**
for PR 8643 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150372009
**[Test build #44182 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44182/consoleFull)**
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150371516
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150372929
@jkbradley sorry for the delay of my update. I tried to use DataFrame
serialization at
https://github.com/yu-iskw/spark/commit/2f701930677c7d359c0a706d16c2438509528b77.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150392863
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150392759
**[Test build #44182 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44182/consoleFull)**
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-150392862
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user yu-iskw commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-140981031
@jkbradley thank you for the comment. Just to be sure,
`LDAModelWrapper`.`describeTopics()` should return a DataFrame and then extract
the return value from the
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-141212929
@yu-iskw I think it's OK to use DataFrame internally in spark.mllib. It
already has the dependency, and it would be a private API.
---
If your project is set up for
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-139610362
@yu-iskw Rather than using Java Any types and the old serialization
patterns, would it be easier to convert to a local DataFrame? We should be
able to take advantage
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138304711
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138304729
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138327468
[Test build #42099 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42099/console)
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138327571
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138327569
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user yu-iskw opened a pull request:
https://github.com/apache/spark/pull/8643
[SPARK-8467][MLlib][PySpark] Add LDAModel.describeTopics() in Python
Could @jkbradley and @davies review it?
- Create a wrapper class: `LDAModelWrapper` for `LDAModel`. Because we
can't
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138306190
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138306187
[Test build #42096 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42096/console)
for PR 8643 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138306191
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138318657
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138318637
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138305856
[Test build #42096 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42096/consoleFull)
for PR 8643 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8643#issuecomment-138319885
[Test build #42099 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42099/consoleFull)
for PR 8643 at commit
83 matches
Mail list logo
