[GitHub] spark pull request: [SPARK-4387][PySpark] Refactoring python profi...

2015-01-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3901#issuecomment-71923298 @davies Sure, let's pull this in for 1.3. Sorry for letting this hang in the review queue for so long. This looks good to me, so I'm going to merge it to `master`.

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23726705 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,116 @@ object KafkaUtils {

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71923634 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4387][PySpark] Refactoring python profi...

2015-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3901 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71923618 [Test build #26248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26248/consoleFull) for PR 4222 at commit

[GitHub] spark pull request: [SPARK-5441][pyspark] Make SerDeUtil PairRDD t...

2015-01-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4236#issuecomment-71924451 Thanks for adding tests. This looks good to me, so I'm going to merge it into `master` (1.3.0) and mark it for later backport into `branch-1.2` (I'd commit it now,

[GitHub] spark pull request: [SPARK-1405] [mllib] Latent Dirichlet Allocati...

2015-01-28 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4047#discussion_r23727215 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala --- @@ -0,0 +1,265 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-5388] Provide a stable application subm...

2015-01-28 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4216#issuecomment-71924525 I made a comment at one spot in the code, but throughout I find the name stable confusing. It implies the other one is unstable, and without the context from the JIRA,

[GitHub] spark pull request: [SPARK-4955]With executor dynamic scaling enab...

2015-01-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3962#issuecomment-71910884 LGTM actually. I'm going to merge this into master after adding the comment I talked about earlier. Thanks for explaining your reasoning @lianhuiwang. --- If your

[GitHub] spark pull request: [SPARK-5188][BUILD] make-distribution.sh shoul...

2015-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3988 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-1934 [CORE] this reference escape to ...

2015-01-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4225#issuecomment-71912133 Thanks @srowen and @zsxwing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-4687. [WIP] Add an addDirectory API

2015-01-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3670#discussion_r23722407 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -24,29 +24,37 @@ import java.net.URI import java.util.{Arrays, Properties, UUID}

[GitHub] spark pull request: [SPARK-5437] Fix DriverSuite and SparkSubmitSu...

2015-01-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4230#issuecomment-7191 Merging into master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5434] [EC2] Preserve spaces in EC2 path

2015-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4224 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-4687. [WIP] Add an addDirectory API

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3670#issuecomment-71914187 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5461] [graphx] Add isCheckpointed, getC...

2015-01-28 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4253#issuecomment-71915043 Unrelated failure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4989][CORE] backport for branch-1.2 cat...

2015-01-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3969#issuecomment-71916721 LGTM I'll merge this after the 1.2.1 release is out of the window. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-4989][CORE] backport for branch-1.1 cat...

2015-01-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3970#issuecomment-71916636 Hey @liyezhang556520 would you mind closing this it's now merged --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-4687. [WIP] Add an addDirectory API

2015-01-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3670#discussion_r23723912 --- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala --- @@ -72,4 +74,87 @@ class SparkContextSuite extends FunSuite with LocalSparkContext

[GitHub] spark pull request: [SPARK-5441][pyspark] Make SerDeUtil PairRDD t...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4236#issuecomment-71920676 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5441][pyspark] Make SerDeUtil PairRDD t...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4236#issuecomment-71920662 [Test build #26243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26243/consoleFull) for PR 4236 at commit

[GitHub] spark pull request: [SPARK-5388] Provide a stable application subm...

2015-01-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4216#issuecomment-71921055 Hi @andrewor14, Although this patch adds many lines, its scope is actually limited only to standalone cluster mode, and the default submit behavior is actually

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23726607 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,116 @@ object KafkaUtils {

[GitHub] spark pull request: [SPARK-1405] [mllib] Latent Dirichlet Allocati...

2015-01-28 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4047#discussion_r23726961 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDA.scala --- @@ -0,0 +1,472 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-5461] [graphx] Add isCheckpointed, getC...

2015-01-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4253#issuecomment-71923990 lgtm. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-71923968 Kinda diverging into off-topic territory, but... It seems like having multiple apps futzing w/ the system properties at the same time is bound to create

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23726952 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,116 @@ object KafkaUtils {

[GitHub] spark pull request: [SPARK-5441][pyspark] Make SerDeUtil PairRDD t...

2015-01-28 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4236#discussion_r23727281 --- Diff: core/src/test/scala/org/apache/spark/api/python/SerDeUtilSuite.scala --- @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-5461] [graphx] Add isCheckpointed, getC...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4253#issuecomment-71924683 [Test build #568 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/568/consoleFull) for PR 4253 at commit

[GitHub] spark pull request: Replaced deprecated FileSystem.isDirectory() w...

2015-01-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4197#issuecomment-71924766 Just for completeness: `FileStatus.isDir()` is also deprecated... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-5388] Provide a stable application subm...

2015-01-28 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/4216#discussion_r23719735 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -121,6 +122,17 @@ private[spark] class Master( throw new

[GitHub] spark pull request: [SPARK-5461] [graphx] Add isCheckpointed, getC...

2015-01-28 Thread jkbradley
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/4253 [SPARK-5461] [graphx] Add isCheckpointed, getCheckpointedFiles methods to Graph Added the 2 methods to Graph and GraphImpl. Both make calls to the underlying vertex and edge RDDs. This

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-28 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4151#issuecomment-71911954 @davies I merged your changes and moved `Identifiable` to `util.py`. Could you make a final pass? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4151#issuecomment-71912647 [Test build #26246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26246/consoleFull) for PR 4151 at commit

[GitHub] spark pull request: [SPARK-5434] [EC2] Preserve spaces in EC2 path

2015-01-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4224#issuecomment-71913993 Ok I'm merging this into master and marking it as backport needed (since we're in the middle of a 1.2.1 release) --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-5176] The thrift server does not suppor...

2015-01-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4137#issuecomment-71916234 @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-4687. [WIP] Add an addDirectory API

2015-01-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3670#discussion_r23723648 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -555,17 +584,42 @@ private[spark] object Utils extends Logging {

[GitHub] spark pull request: [SPARK-5176] The thrift server does not suppor...

2015-01-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4137#issuecomment-71916253 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-4687. [WIP] Add an addDirectory API

2015-01-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/3670#issuecomment-71917373 Seems like tests are failing because of the `isDirectory` issue: [error] /home/jenkins/workspace/SparkPullRequestBuilder/core/src/main/scala/org/apache/spark

[GitHub] spark pull request: [WIP] [SPARK-3996]: Shade Jetty in Spark deliv...

2015-01-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4252#issuecomment-71919727 Latest patch LGTM. Thanks for the extra cleanup too! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-28 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4155#discussion_r23725993 --- Diff: core/src/main/scala/org/apache/spark/SparkHadoopWriter.scala --- @@ -106,18 +107,30 @@ class SparkHadoopWriter(@transient jobConf: JobConf)

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23726392 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,116 @@ object KafkaUtils {

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4151#issuecomment-71922343 [Test build #26244 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26244/consoleFull) for PR 4151 at commit

[GitHub] spark pull request: [SPARK-4259][MLlib]: Add Power Iteration Clust...

2015-01-28 Thread fjiang6
GitHub user fjiang6 opened a pull request: https://github.com/apache/spark/pull/4254 [SPARK-4259][MLlib]: Add Power Iteration Clustering Algorithm with Gaussian Similarity Function Add single pseudo-eigenvector PIC Including documentations, one property file and updated pom.xml

[GitHub] spark pull request: [SPARK-4387][PySpark] Refactoring python profi...

2015-01-28 Thread orenmazor
Github user orenmazor commented on the pull request: https://github.com/apache/spark/pull/3901#issuecomment-71925039 :dancer: :shipit: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4151#issuecomment-71925161 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5465] [SQL] Fixes filter push-down for ...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4255#issuecomment-71938129 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5097][SQL] Made DataFrame dsl usable in...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4241#issuecomment-71940279 [Test build #26263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26263/consoleFull) for PR 4241 at commit

[GitHub] spark pull request: [SPARK-2996] Implement userClassPathFirst for ...

2015-01-28 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/3233#issuecomment-71943787 The new spark.driver.userClassPathFirst property seems a little strange to me in that, IIUC, it only takes effect when the driver is started through the application

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-28 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4151#discussion_r23736564 --- Diff: python/pyspark/ml/param/shared.py --- @@ -0,0 +1,260 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-71945158 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5461] [graphx] Add isCheckpointed, getC...

2015-01-28 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4253#issuecomment-71946532 That broke some stuff...will fix soon --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-5212][SQL] Add support of schema-less, ...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4014#issuecomment-71947781 [Test build #26268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26268/consoleFull) for PR 4014 at commit

[GitHub] spark pull request: [SPARK-4259][MLlib]: Add Power Iteration Clust...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4254#issuecomment-71947798 [Test build #26272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26272/consoleFull) for PR 4254 at commit

[GitHub] spark pull request: [SPARK-5212][SQL] Add support of schema-less, ...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4014#issuecomment-71947786 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4256#issuecomment-71951033 [Test build #26267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26267/consoleFull) for PR 4256 at commit

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-28 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4151#issuecomment-71951198 Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4256#issuecomment-71951042 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5440][pyspark] Add toLocalIterator to p...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4237#issuecomment-71910948 [Test build #26240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26240/consoleFull) for PR 4237 at commit

[GitHub] spark pull request: [SPARK-5440][pyspark] Add toLocalIterator to p...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4237#issuecomment-71910964 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP] [SPARK-3996]: Shade Jetty in Spark deliv...

2015-01-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4252#issuecomment-71912254 Hi @pwendell , Did you check that jetty classes only show up in spark-core's jar? I see a jetty dependency in `streaming/pom.xml`, so I'm afraid that would cause

[GitHub] spark pull request: SPARK-4687. [WIP] Add an addDirectory API

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3670#issuecomment-71914178 [Test build #26241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26241/consoleFull) for PR 3670 at commit

[GitHub] spark pull request: [SPARK-5461] [graphx] Add isCheckpointed, getC...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4253#issuecomment-71914257 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4222#issuecomment-71914714 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5417] Remove redundant executor-id set(...

2015-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4213 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5388] Provide a stable application subm...

2015-01-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/4216#discussion_r23724419 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/SubmitRestProtocolMessage.scala --- @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5097][SQL] Address DataFrame code revie...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4241#issuecomment-71919673 [Test build #26251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26251/consoleFull) for PR 4241 at commit

[GitHub] spark pull request: [WIP] [SPARK-3996]: Shade Jetty in Spark deliv...

2015-01-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4252#discussion_r23725257 --- Diff: pom.xml --- @@ -337,25 +337,38 @@ /exclusion /exclusions /dependency + + !-- Shaded deps marked

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71922614 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5461] [graphx] Add isCheckpointed, getC...

2015-01-28 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4253#issuecomment-71923563 @rxin Added more doc. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-28 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23727132 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,116 @@ object KafkaUtils {

[GitHub] spark pull request: [SPARK-1405] [mllib] Latent Dirichlet Allocati...

2015-01-28 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4047#discussion_r23727115 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDA.scala --- @@ -0,0 +1,472 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-71937434 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71941597 [Test build #569 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/569/consoleFull) for PR 3715 at commit

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread brkyvz
GitHub user brkyvz opened a pull request: https://github.com/apache/spark/pull/4256 [SPARK-3977] Conversion methods for BlockMatrix to other Distributed Matrices The conversion methods for `BlockMatrix`. Conversions go through `CoordinateMatrix` in order to cause a shuffle so that

[GitHub] spark pull request: [SPARK-5388] Provide a stable application subm...

2015-01-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4216#discussion_r23736007 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/SubmitRestProtocolMessage.scala --- @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4151#issuecomment-71942733 [Test build #26266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26266/consoleFull) for PR 4151 at commit

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4256#issuecomment-7194 [Test build #26267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26267/consoleFull) for PR 4256 at commit

[GitHub] spark pull request: [SPARK-5212][SQL] Add support of schema-less, ...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4014#issuecomment-71943344 [Test build #26268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26268/consoleFull) for PR 4014 at commit

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-71945152 [Test build #26262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26262/consoleFull) for PR 4215 at commit

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4256#issuecomment-71946157 [Test build #26270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26270/consoleFull) for PR 4256 at commit

[GitHub] spark pull request: [SPARK-4382] Add locations parameter to Twitte...

2015-01-28 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/3246#issuecomment-71946175 @tdas Do you have time to look at this to see if it is ready to merge? Thanks. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-28 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23737948 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -144,4 +150,116 @@ object KafkaUtils {

[GitHub] spark pull request: [SPARK-2996] Implement userClassPathFirst for ...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3233#issuecomment-71947316 [Test build #26271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26271/consoleFull) for PR 3233 at commit

[GitHub] spark pull request: [SPARK-2996] Implement userClassPathFirst for ...

2015-01-28 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/3233#discussion_r23738607 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -375,4 +390,64 @@ private[spark] object SparkConf { def isSparkPortConf(name:

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4256#issuecomment-71949428 [Test build #26265 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26265/consoleFull) for PR 4256 at commit

[GitHub] spark pull request: [SPARK-5445][SQL] Made DataFrame dsl usable in...

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4241#issuecomment-71950869 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4256#discussion_r23739791 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala --- @@ -182,6 +184,38 @@ class BlockMatrix( this }

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4256#discussion_r23739809 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/IndexedRowMatrix.scala --- @@ -75,6 +77,23 @@ class IndexedRowMatrix( new

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4256#discussion_r23739799 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/IndexedRowMatrix.scala --- @@ -24,6 +24,8 @@ import org.apache.spark.rdd.RDD

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4256#discussion_r23739796 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/CoordinateMatrix.scala --- @@ -98,6 +97,59 @@ class CoordinateMatrix(

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4256#discussion_r23739788 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala --- @@ -182,6 +184,38 @@ class BlockMatrix( this }

[GitHub] spark pull request: [SPARK-3977] Conversion methods for BlockMatri...

2015-01-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4256#discussion_r23739793 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/CoordinateMatrix.scala --- @@ -98,6 +97,59 @@ class CoordinateMatrix(

[GitHub] spark pull request: [SPARK-5445][SQL] Made DataFrame dsl usable in...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4241#issuecomment-71950863 [Test build #26263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26263/consoleFull) for PR 4241 at commit

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4151 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5465] [SQL] Fixes filter push-down for ...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4255#issuecomment-71938124 [Test build #26254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26254/consoleFull) for PR 4255 at commit

[GitHub] spark pull request: [SPARK-5388] Provide a stable application subm...

2015-01-28 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/4216#issuecomment-71938252 If you guys have concerns about the protocol, you should comment on it or on the JIRA; the protocol was posted 5 days ago. Really what matters for this release is

[GitHub] spark pull request: SPARK-4687. [WIP] Add an addDirectory API

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3670#issuecomment-71938967 [Test build #26257 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26257/consoleFull) for PR 3670 at commit

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-01-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-71938989 [Test build #26262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26262/consoleFull) for PR 4215 at commit

[GitHub] spark pull request: SPARK-4687. [WIP] Add an addDirectory API

2015-01-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3670#issuecomment-71938975 Test PASSed. Refer to this link for build results (access rights to CI server needed):

<    1   2   3   4   5   6   7   8   >