[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9051#discussion_r44323340 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala --- @@ -145,9 +145,20 @@ private[ui] class StageTableBase( case None =>

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread feynmanliang
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/9513#discussion_r44323247 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9051#discussion_r44323362 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala --- @@ -145,9 +145,20 @@ private[ui] class StageTableBase( case None =>

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-155175125 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-155175092 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9051#issuecomment-155176563 This patch looks alright. I left a few minor comments but I'll merge it after they are addressed. Separately we should address @tgravescs' request for SPARK-11185 at

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread feynmanliang
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/9513#discussion_r44324308 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-11564][SQL][follow-up] clean up java tu...

2015-11-09 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9567#discussion_r44324313 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/Encoder.scala --- @@ -50,92 +50,10 @@ object Encoder { def DOUBLE:

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-155177514 **[Test build #45406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45406/consoleFull)** for PR 5267 at commit

[GitHub] spark pull request: [SPARK-11463] [PYSPARK] only install signal in...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9574#issuecomment-155182146 **[Test build #45405 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45405/consoleFull)** for PR 9574 at commit

[GitHub] spark pull request: [SPARK-10371] [SQL] Implement subexpr eliminat...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9480#issuecomment-155185089 **[Test build #45384 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45384/consoleFull)** for PR 9480 at commit

[GitHub] spark pull request: [SPARK-10371] [SQL] Implement subexpr eliminat...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9480#issuecomment-155185293 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11578] [SQL] User API for Typed Aggrega...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9555#issuecomment-155188203 **[Test build #45409 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45409/consoleFull)** for PR 9555 at commit

[GitHub] spark pull request: [SPARK-11578] [SQL] User API for Typed Aggrega...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9555#issuecomment-155188209 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11564][SQL][follow-up] clean up java tu...

2015-11-09 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9567#discussion_r44328940 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/Encoder.scala --- @@ -50,92 +50,10 @@ object Encoder { def DOUBLE:

[GitHub] spark pull request: [SPARK-11140] [core] Transfer files using netw...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9530#issuecomment-155160422 **[Test build #45383 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45383/consoleFull)** for PR 9530 at commit

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155160682 **[Test build #45398 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45398/consoleFull)** for PR 8984 at commit

[GitHub] spark pull request: [SPARK-9830] [SQL] Remove AggregateExpression1...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9556#issuecomment-155160400 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11140] [core] Transfer files using netw...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9530#issuecomment-155160617 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155164455 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-11572] Exit AsynchronousListenerBus thr...

2015-11-09 Thread vonnagy
Github user vonnagy commented on the pull request: https://github.com/apache/spark/pull/9546#issuecomment-155166356 I was unable to duplicate the issue I had with the `listenerThread.join` so it seems that your change solved the issue. If I encounter the issue again then I

[GitHub] spark pull request: [SPARK-11581][Docs] Example mllib code in docu...

2015-11-09 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9560#issuecomment-155166778 LGTM, merging with master, branch-1.6, branch-1.5, and branch-1.4 Thanks! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9513#issuecomment-155166627 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9513#issuecomment-155166554 **[Test build #45388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45388/consoleFull)** for PR 9513 at commit

[GitHub] spark pull request: [SPARK-11577][SQL] Handle code review comments...

2015-11-09 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9551#issuecomment-155169381 This has been merged already. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44320580 --- Diff: core/src/main/scala/org/apache/spark/util/CheckpointingIterator.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155169298 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155168576 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44321308 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -258,11 +258,14 @@ abstract class RDD[T: ClassTag]( * subclasses of RDD.

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44321996 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -258,11 +258,14 @@ abstract class RDD[T: ClassTag]( * subclasses of RDD.

[GitHub] spark pull request: [SPARK-11359][STREAMING][KINESIS] Checkpoint t...

2015-11-09 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9421#issuecomment-155175426 @zsxwing It should be acceptable as well. Think about it like this: We have 2 receivers, A and B: t0 -> A receives batch with seq number x_0, B receives batch with

[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9051#discussion_r44323350 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala --- @@ -145,9 +145,20 @@ private[ui] class StageTableBase( case None =>

[GitHub] spark pull request: [SPARK-8467][MLlib][PySpark] Add LDAModel.desc...

2015-11-09 Thread yu-iskw
Github user yu-iskw commented on the pull request: https://github.com/apache/spark/pull/8643#issuecomment-155175314 @jkbradley sure! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10930] History "Stages" page "duration"...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9051#discussion_r44323731 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala --- @@ -145,9 +145,20 @@ private[ui] class StageTableBase( case None =>

[GitHub] spark pull request: [SPARK-11564][SQL][follow-up] clean up java tu...

2015-11-09 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9567#discussion_r44323839 --- Diff: sql/core/src/test/java/test/org/apache/spark/sql/JavaDatasetSuite.java --- @@ -299,59 +299,33 @@ public void testJoin() { @Test

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread feynmanliang
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/9513#discussion_r44323843 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-9926] [SPARK-10340] [SQL] Use S3 bulk l...

2015-11-09 Thread piaozhexiu
Github user piaozhexiu commented on a diff in the pull request: https://github.com/apache/spark/pull/8512#discussion_r44330648 --- Diff: core/pom.xml --- @@ -40,6 +40,11 @@ ${avro.mapred.classifier} + com.amazonaws + aws-java-sdk

[GitHub] spark pull request: [SPARK-11539] debian packaging

2015-11-09 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9507#issuecomment-155161478 With my apologies, would you mind closing this PR? I believe this was already removed from Spark. This PR will remain anyhow to show what you put together. --- If your

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread olarayej
Github user olarayej commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155161252 @shivaram @felixcheung I followed the same steps described by @shivaram. What's confusing for us is that every time we run a fetch followed by a merge, it

[GitHub] spark pull request: [SPARK-11598] [SQL] enable tests for ShuffledH...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9573#issuecomment-155163698 **[Test build #45397 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45397/consoleFull)** for PR 9573 at commit

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155165539 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155165525 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44320330 --- Diff: core/src/main/scala/org/apache/spark/util/CheckpointingIterator.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44320223 --- Diff: core/src/main/scala/org/apache/spark/util/CheckpointingIterator.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44320866 --- Diff: core/src/main/scala/org/apache/spark/rdd/ReliableRDDCheckpointData.scala --- @@ -63,11 +98,20 @@ private[spark] class

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-11-09 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/5267#discussion_r44322215 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/BisectingKMeans.scala --- @@ -0,0 +1,489 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155172521 **[Test build #45387 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45387/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44322179 --- Diff: core/src/main/scala/org/apache/spark/util/CheckpointingIterator.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-11578] [SQL] User API for Typed Aggrega...

2015-11-09 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9555#discussion_r44323023 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SumOf.scala --- @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-155174143 **[Test build #45396 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45396/consoleFull)** for PR 5267 at commit

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread feynmanliang
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/9513#discussion_r44323496 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-9830] [SQL] Remove AggregateExpression1...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9556#issuecomment-155182875 **[Test build #45407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45407/consoleFull)** for PR 9556 at commit

[GitHub] spark pull request: [SPARK-9830] [SQL] Remove AggregateExpression1...

2015-11-09 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9556#discussion_r44327164 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -146,148 +146,105 @@ private[sql] abstract class SparkStrategies

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread feynmanliang
Github user feynmanliang commented on the pull request: https://github.com/apache/spark/pull/9513#issuecomment-155185664 Second pass. Most significant comments are about completely removing `Vector` from the public API and debating `DistributedLDAModel < LDAModel` vs `abstract class

[GitHub] spark pull request: SPARK-6541 - Sort executors by ID (numeric)

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9165#issuecomment-155185724 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-6541 - Sort executors by ID (numeric)

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9165#issuecomment-155185761 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11578] [SQL] User API for Typed Aggrega...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9555#issuecomment-155187488 **[Test build #45409 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45409/consoleFull)** for PR 9555 at commit

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-155190761 **[Test build #45406 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45406/consoleFull)** for PR 5267 at commit

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155164443 **[Test build #45398 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45398/consoleFull)** for PR 8984 at commit

[GitHub] spark pull request: [SPARK-11598] [SQL] enable tests for ShuffledH...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9573#issuecomment-155164550 **[Test build #45397 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45397/consoleFull)** for PR 9573 at commit

[GitHub] spark pull request: [SPARK-11598] [SQL] enable tests for ShuffledH...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9573#issuecomment-155164557 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11581][Docs] Example mllib code in docu...

2015-11-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9560 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11598] [SQL] enable tests for ShuffledH...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9573#issuecomment-155168228 **[Test build #45403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45403/consoleFull)** for PR 9573 at commit

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44320118 --- Diff: core/src/main/scala/org/apache/spark/util/CheckpointingIterator.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44320738 --- Diff: core/src/main/scala/org/apache/spark/rdd/ReliableRDDCheckpointData.scala --- @@ -63,11 +98,20 @@ private[spark] class

[GitHub] spark pull request: [SPARK-11463] [PYSPARK] only install signal in...

2015-11-09 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/9574 [SPARK-11463] [PYSPARK] only install signal in main thread Only install signal in main thread, or it will fail to create context in not-main thread. You can merge this pull request into a Git

[GitHub] spark pull request: [SPARK-11463] [PYSPARK] only install signal in...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9574#issuecomment-155170467 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44321310 --- Diff: core/src/main/scala/org/apache/spark/util/CheckpointingIterator.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9513#issuecomment-155170628 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-155170436 **[Test build #45404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45404/consoleFull)** for PR 9526 at commit

[GitHub] spark pull request: [SPARK-11463] [PYSPARK] only install signal in...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9574#issuecomment-155170496 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11597][SQL][WIP] improve performance of...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9572#issuecomment-155172704 **[Test build #45394 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45394/consoleFull)** for PR 9572 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155172748 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11259] [ML] Params.validateParams() sho...

2015-11-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/9224#discussion_r44322374 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Predictor.scala --- @@ -113,6 +113,7 @@ abstract class Predictor[ private[ml] def

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155172649 Yeah something seems to be messed up. You shouldn't get other files changed if you do a fetch + merge as long as the rest of your tree is synced to the same place.

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9428#discussion_r44322400 --- Diff: core/src/main/scala/org/apache/spark/util/CheckpointingIterator.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread feynmanliang
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/9513#discussion_r44323084 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5267#issuecomment-155174569 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11057] [SQL] Add correlation and covari...

2015-11-09 Thread NarineK
Github user NarineK commented on the pull request: https://github.com/apache/spark/pull/9366#issuecomment-155171975 In general I think that currently there are some issues in the StatFunctions.scala: It seems that all computations both for covariance and correlation are

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread feynmanliang
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/9513#discussion_r44324500 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-11359][STREAMING][KINESIS] Checkpoint t...

2015-11-09 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9421#discussion_r44324452 --- Diff: extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisCheckpointerSuite.scala --- @@ -0,0 +1,156 @@ +/* + * Licensed

[GitHub] spark pull request: [SPARK-11578] [SQL] User API for Typed Aggrega...

2015-11-09 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9555#discussion_r44326231 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/expressions/Aggregator.scala --- @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-6541 - Sort executors by ID (numeric)

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9165#issuecomment-155186811 **[Test build #45408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45408/consoleFull)** for PR 9165 at commit

[GitHub] spark pull request: [SPARK-11578] [SQL] User API for Typed Aggrega...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9555#issuecomment-155186853 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11578] [SQL] User API for Typed Aggrega...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9555#issuecomment-155186821 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155159546 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155159619 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11598] [SQL] enable tests for ShuffledH...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9573#issuecomment-155159575 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11598] [SQL] enable tests for ShuffledH...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9573#issuecomment-155159524 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11564][SQL][follow-up] clean up java tu...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9567#issuecomment-155165945 **[Test build #45375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45375/consoleFull)** for PR 9567 at commit

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9513#issuecomment-155165981 I had not written tests very carefully, so they had some bugs. Updated now. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155165489 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10863][SPARKR] Method coltypes() to get...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8984#issuecomment-155165511 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11140] [core] Transfer files using netw...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9530#issuecomment-155166790 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9513#issuecomment-155166751 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11140] [core] Transfer files using netw...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9530#issuecomment-155167098 **[Test build #45402 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45402/consoleFull)** for PR 9530 at commit

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9513#issuecomment-155166791 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11140] [core] Transfer files using netw...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9530#issuecomment-155166742 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5565] [ML] LDA wrapper for Pipelines AP...

2015-11-09 Thread feynmanliang
Github user feynmanliang commented on a diff in the pull request: https://github.com/apache/spark/pull/9513#discussion_r44321879 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/LDA.scala --- @@ -0,0 +1,740 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-6517][mllib] Implement the Algorithm of...

2015-11-09 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/5267#discussion_r44321863 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/BisectingKMeans.scala --- @@ -0,0 +1,489 @@ +/* + * Licensed to the Apache Software

<    2   3   4   5   6   7   8   9   10   11   >