[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3381#issuecomment-63772820 [Test build #23665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23665/consoleFull) for PR 3381 at commit

[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3381#issuecomment-63773096 [Test build #23665 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23665/consoleFull) for PR 3381 at commit

[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3381#issuecomment-63773100 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: add jackson-core-asl 1.8.8 dependency

2014-11-20 Thread devlatte
Github user devlatte commented on the pull request: https://github.com/apache/spark/pull/3379#issuecomment-63773937 It might be related with this issue. https://issues.apache.org/jira/browse/SPARK-3602 --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-4486][MLLIB] Improve GradientBoosting A...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3374#issuecomment-63774229 [Test build #23663 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23663/consoleFull) for PR 3374 at commit

[GitHub] spark pull request: [SPARK-4486][MLLIB] Improve GradientBoosting A...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3374#issuecomment-63774234 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4505][Core] Add a ClassTag parameter to...

2014-11-20 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/3378#issuecomment-63774979 We should definitely add a ClassTag since this can be used for primitive types. However, there might be places where we create a lot of CompactBuffers. I haven't had a

[GitHub] spark pull request: [SPARK-4505][Core] Add a ClassTag parameter to...

2014-11-20 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/3378#issuecomment-63775682 Cogroup uses `CompactBuffer`. However, it cannot add ClassTag due to its signature: ```Scala class CoGroupedRDD[K](@transient var rdds: Seq[RDD[_ :

[GitHub] spark pull request: [SPARK-4486][MLLIB] Improve GradientBoosting A...

2014-11-20 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3374#issuecomment-63777328 @manishamde @jkbradley Thanks! Merged into master and branch-1.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-4439] [MLlib] add python api for random...

2014-11-20 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3320#issuecomment-63777609 @davies We updated the `RandomForest` API in #3374 . Now `RandomForest` returns a `RandomForestModel`. Could you rebase and update this PR? Thanks! --- If your project

[GitHub] spark pull request: [SPARK-4486][MLLIB] Improve GradientBoosting A...

2014-11-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3374 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4481][Streaming][Doc] Fix the wrong des...

2014-11-20 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3376#issuecomment-63779898 I have merged this. Thanks for the backport! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4481][Streaming][Doc] Fix the wrong des...

2014-11-20 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/3376#issuecomment-63780029 Github wont close this automatically, so could you please close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-4510][MLlib]: Add k-medoids Partitionin...

2014-11-20 Thread fjiang6
GitHub user fjiang6 opened a pull request: https://github.com/apache/spark/pull/3382 [SPARK-4510][MLlib]: Add k-medoids Partitioning Around Medoids (PAM) algorithm PAM (k-medoids) including the test case and an example. Passed the style checks Tested and compared with

[GitHub] spark pull request: [SPARK-4481][Streaming][Doc] Fix the wrong des...

2014-11-20 Thread zsxwing
Github user zsxwing closed the pull request at: https://github.com/apache/spark/pull/3376 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4510][MLlib]: Add k-medoids Partitionin...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3382#issuecomment-63781094 [Test build #23666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23666/consoleFull) for PR 3382 at commit

[GitHub] spark pull request: [SPARK-4510][MLlib]: Add k-medoids Partitionin...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3382#issuecomment-63791400 [Test build #23666 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23666/consoleFull) for PR 3382 at commit

[GitHub] spark pull request: [SPARK-4510][MLlib]: Add k-medoids Partitionin...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3382#issuecomment-63791407 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3938][SQL] Names in-memory columnar RDD...

2014-11-20 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/3383 [SPARK-3938][SQL] Names in-memory columnar RDD with corresponding table name This PR enables the Web UI storage tab to show the in-memory table name instead of the mysterious query plan string as

[GitHub] spark pull request: [SPARK-3938][SQL] Names in-memory columnar RDD...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3383#issuecomment-63792868 [Test build #23667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23667/consoleFull) for PR 3383 at commit

[GitHub] spark pull request: [SPARK-3938][SQL] Names in-memory columnar RDD...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3383#issuecomment-63793690 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3938][SQL] Names in-memory columnar RDD...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3383#issuecomment-63793685 [Test build #23667 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23667/consoleFull) for PR 3383 at commit

[GitHub] spark pull request: add jackson-core-asl 1.8.8 dependency

2014-11-20 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3379#issuecomment-63796464 This is basically the same as https://issues.apache.org/jira/browse/SPARK-3955 actually, which already has an open PR: https://github.com/apache/spark/pull/2818 Maybe

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63803519 [Test build #23668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23668/consoleFull) for PR 3237 at commit

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63808836 [Test build #23669 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23669/consoleFull) for PR 3237 at commit

[GitHub] spark pull request: [SPARK-3938][SQL] Names in-memory columnar RDD...

2014-11-20 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/3383#issuecomment-63810342 +10 (haven't looked at PR itself) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63810730 [Test build #23670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23670/consoleFull) for PR 3237 at commit

[GitHub] spark pull request: [SPARK-3938][SQL] Names in-memory columnar RDD...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3383#issuecomment-63814934 [Test build #23671 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23671/consoleFull) for PR 3383 at commit

[GitHub] spark pull request: Merge pull request #1 from apache/master

2014-11-20 Thread codeAshu
GitHub user codeAshu opened a pull request: https://github.com/apache/spark/pull/3384 Merge pull request #1 from apache/master updating Fork You can merge this pull request into a Git repository by running: $ git pull https://github.com/codeAshu/spark master Alternatively you

[GitHub] spark pull request: Merge pull request #1 from apache/master

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3384#issuecomment-63815959 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Merge pull request #1 from apache/master

2014-11-20 Thread codeAshu
Github user codeAshu closed the pull request at: https://github.com/apache/spark/pull/3384 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Merge pull request #1 from apache/master

2014-11-20 Thread codeAshu
Github user codeAshu commented on the pull request: https://github.com/apache/spark/pull/3384#issuecomment-63816089 sorry my bad --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: Branch 1.2

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3385#issuecomment-63817444 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Branch 1.2

2014-11-20 Thread codeAshu
GitHub user codeAshu opened a pull request: https://github.com/apache/spark/pull/3385 Branch 1.2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/spark branch-1.2 Alternatively you can review and apply these changes as

[GitHub] spark pull request: Branch 1.2

2014-11-20 Thread codeAshu
Github user codeAshu closed the pull request at: https://github.com/apache/spark/pull/3385 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63817831 [Test build #23668 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23668/consoleFull) for PR 3237 at commit

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63817840 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...

2014-11-20 Thread chenghao-intel
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/3386 [Spark-4512] [SQL] Unresolved Attribute Exception in Sort By It will cause exception while do query like: SELECT key+key FROM src sort by value; This fix is inspired by #3363 ,

[GitHub] spark pull request: Add example that reads a local file, writes to...

2014-11-20 Thread rnowling
Github user rnowling commented on the pull request: https://github.com/apache/spark/pull/3347#issuecomment-63820267 @andrewor14 , thanks for the comments! I believe I fixed everything except for the changing the name of the example. I wanted some more feedback. I wrote the

[GitHub] spark pull request: Add example that reads a local file, writes to...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3347#issuecomment-63820290 [Test build #23672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23672/consoleFull) for PR 3347 at commit

[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-63820723 [Test build #23673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23673/consoleFull) for PR 3386 at commit

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63821052 [Test build #23674 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23674/consoleFull) for PR 3222 at commit

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread giwa
Github user giwa commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63822056 @tdas @watermen I believe python code become like this. ``` def pprint(self, num): Print the first num elements of each

[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-63822505 [Test build #23673 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23673/consoleFull) for PR 3386 at commit

[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-63822515 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63824115 [Test build #23670 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23670/consoleFull) for PR 3237 at commit

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63824131 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63825274 [Test build #23669 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23669/consoleFull) for PR 3237 at commit

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63825280 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3938][SQL] Names in-memory columnar RDD...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3383#issuecomment-63827207 [Test build #23671 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23671/consoleFull) for PR 3383 at commit

[GitHub] spark pull request: [SPARK-3938][SQL] Names in-memory columnar RDD...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3383#issuecomment-63827215 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4488][PySpark] Add control over map-sid...

2014-11-20 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/3366#issuecomment-63831692 @davies Could you help reviewing this patch? Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: Add example that reads a local file, writes to...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3347#issuecomment-63835431 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: Add example that reads a local file, writes to...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3347#issuecomment-63835422 [Test build #23672 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23672/consoleFull) for PR 3347 at commit

[GitHub] spark pull request: [SPARK-3325][Streaming] Add a parameter to the...

2014-11-20 Thread watermen
Github user watermen commented on the pull request: https://github.com/apache/spark/pull/3237#issuecomment-63836267 @tdas See test build #23670, I had to add exclusions for ```scala

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63836399 [Test build #23674 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23674/consoleFull) for PR 3222 at commit

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-63836409 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2014-11-20 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/3381#issuecomment-63841756 looks like a compilation error, and the reason why no unit test results were stored was that none were run (unless i'm missing something). On Wed, Nov

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63844350 I looked through this and took a spin locally, LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63851287 Hey @JoshRosen I'll take a look at this right now. Is it still WIP by the way? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2014-11-20 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/3381#issuecomment-63851763 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20664971 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -144,11 +146,30 @@ class JobProgressListener(conf: SparkConf)

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20665049 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -166,6 +188,21 @@ class JobProgressListener(conf: SparkConf)

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20665136 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala --- @@ -40,9 +40,22 @@ private[jobs] object UIData { class JobUIData(

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20665208 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -214,6 +264,14 @@ class JobProgressListener(conf: SparkConf)

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20665341 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -214,6 +264,14 @@ class JobProgressListener(conf: SparkConf)

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20665377 --- Diff: core/src/test/scala/org/apache/spark/ui/UISeleniumSuite.scala --- @@ -17,16 +17,18 @@ package org.apache.spark.ui -import

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20665632 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -144,11 +146,30 @@ class JobProgressListener(conf: SparkConf)

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20665705 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -56,7 +56,11 @@ case class SparkListenerTaskEnd( extends

[GitHub] spark pull request: [SPARK-4439] [MLlib] add python api for random...

2014-11-20 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3320#issuecomment-63853606 @mengxr done. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20665717 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20665799 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63854023 All superficial comments on the code, but I tried it out and still got this somewhat icky situation where a completed stage has fewer succeeded tasks than the

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20666049 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala --- @@ -195,9 +180,10 @@ private[ui] class StageTableBase( private[ui]

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63855330 Also, I thought more about having this in 1.2, and I'm -0.5 on putting this in 1.2. Given all of the subtleties you ended up running into with this, Josh, I don't

[GitHub] spark pull request: [SPARK-4439] [MLlib] add python api for random...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3320#issuecomment-63856051 [Test build #23677 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23677/consoleFull) for PR 3320 at commit

[GitHub] spark pull request: Add example that reads a local file, writes to...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3347#issuecomment-63856412 [Test build #23675 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23675/consoleFull) for PR 3347 at commit

[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3381#issuecomment-63856447 [Test build #23676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23676/consoleFull) for PR 3381 at commit

[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3381#issuecomment-63857314 [Test build #23676 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23676/consoleFull) for PR 3381 at commit

[GitHub] spark pull request: [SPARK-4508] [SQL] build native date type to c...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3381#issuecomment-63857315 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4488][PySpark] Add control over map-sid...

2014-11-20 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3366#issuecomment-63859043 What's the cases that we should disable map side aggregation? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20669493 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -56,7 +56,11 @@ case class SparkListenerTaskEnd( extends

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20669574 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -56,7 +56,11 @@ case class SparkListenerTaskEnd( extends

[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...

2014-11-20 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-63862394 @CodingCat I think we discussed in https://issues.apache.org/jira/browse/SPARK-3628 that it would be best to do this only for result stages first. Can you do that? The

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3009#issuecomment-63862396 Hi Kay, The behavior you noticed is intentional. If a job completes but doesn't run all of it's stages, it ends up showing as finished with a partially

[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...

2014-11-20 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2524#discussion_r20670036 --- Diff: core/src/main/scala/org/apache/spark/Accumulators.scala --- @@ -282,7 +285,6 @@ private object Accumulators { return ret }

[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...

2014-11-20 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/2524#discussion_r20670065 --- Diff: core/src/main/scala/org/apache/spark/Accumulators.scala --- @@ -226,9 +227,12 @@ GrowableAccumulableParam[R % Growable[T] with TraversableOnce[T]

[GitHub] spark pull request: [SPARK-4513][SQL] Support relational operator ...

2014-11-20 Thread ravipesala
GitHub user ravipesala opened a pull request: https://github.com/apache/spark/pull/3387 [SPARK-4513][SQL] Support relational operator '=' in Spark SQL The relational operator '=' is not working in Spark SQL. Same works in Spark HiveQL You can merge this pull request into a Git

[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...

2014-11-20 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-63863153 BTW when doing this only for result stages, my suggestion is to use the data structures within the stage instead of having a second HashMap. I believe I mentioned this

[GitHub] spark pull request: [SPARK-4513][SQL] Support relational operator ...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3387#issuecomment-63863591 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: add Sphinx as a dependency of building docs

2014-11-20 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/3388 add Sphinx as a dependency of building docs You can merge this pull request into a Git repository by running: $ git pull https://github.com/davies/spark doc_readme Alternatively you can

[GitHub] spark pull request: add Sphinx as a dependency of building docs

2014-11-20 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/3388#issuecomment-63863907 cc @pwendell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: add Sphinx as a dependency of building docs

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3388#issuecomment-63864808 [Test build #23678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23678/consoleFull) for PR 3388 at commit

[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] elimin...

2014-11-20 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-63866302 @mateiz , I see...I had the impression that we agreed on still support shuffle stage deduplication finally... OK, I can shrink this patch to only support

[GitHub] spark pull request: [SPARK-4439] [MLlib] add python api for random...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3320#issuecomment-63869584 [Test build #23677 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23677/consoleFull) for PR 3320 at commit

[GitHub] spark pull request: [SPARK-4439] [MLlib] add python api for random...

2014-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3320#issuecomment-63869599 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3974][MLlib] Distributed Block Matrix A...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3200#issuecomment-63870038 [Test build #23679 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23679/consoleFull) for PR 3200 at commit

[GitHub] spark pull request: Add example that reads a local file, writes to...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3347#issuecomment-63870055 [Test build #23675 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23675/consoleFull) for PR 3347 at commit

[GitHub] spark pull request: [SPARK-4145] [WIP] Web UI job pages

2014-11-20 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3009#discussion_r20673960 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -144,11 +146,30 @@ class JobProgressListener(conf: SparkConf)

[GitHub] spark pull request: [SPARK-4477] [PySpark] remove numpy from RDDSa...

2014-11-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3351#issuecomment-63871878 [Test build #23680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23680/consoleFull) for PR 3351 at commit

  1   2   3   4   >