[GitHub] spark pull request: SPARK-1868: Users should be allowed to cogroup...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/813#issuecomment-46144569 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2148 Add link to requirements for custom...

2014-06-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1092 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Updating docs to include missing information a...

2014-06-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1089 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-2309: apply output dir existence checkin...

2014-06-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1088#issuecomment-46145753 I think the title should say: `SPARK-2039` instead of `SPARK-2309` --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-2309: apply output dir existence checkin...

2014-06-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1088#issuecomment-46145766 LGTM - I can merge this and I'll just fix the title. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-2309: apply output dir existence checkin...

2014-06-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1088 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1946] Submit stage after (configured nu...

2014-06-16 Thread li-zhihui
Github user li-zhihui commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r13793268 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -48,6 +48,10 @@ class

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-46146650 @darabos hey that's a flaky unit test we fixed a while back. The issue is that your patch doesn't merge cleanly with master, so it's not getting the fix. Do you mind

[GitHub] spark pull request: Compression should be a setting for individual...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1091#issuecomment-46147420 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-2146. Fix takeOrdered doc

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1086#issuecomment-46147737 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-2146. Fix takeOrdered doc

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1086#issuecomment-46147727 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1868: Users should be allowed to cogroup...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/813#issuecomment-46148949 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1868: Users should be allowed to cogroup...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/813#issuecomment-46148950 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15811/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-2149. Univariate kernel density estimati...

2014-06-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/1093#discussion_r13794553 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/KernelDensity.scala --- @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: Compression should be a setting for individual...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1091#issuecomment-46150263 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-2118] spark class should complain if to...

2014-06-16 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1068#issuecomment-46150437 @pwendell Did I miss anything ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-2146. Fix takeOrdered doc

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1086#issuecomment-46150728 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15813/ --- If your

[GitHub] spark pull request: SPARK-2146. Fix takeOrdered doc

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1086#issuecomment-46150727 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: SPARK-2149. Univariate kernel density estimati...

2014-06-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/1093#discussion_r13794772 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/KernelDensity.scala --- @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: SPARK-2149. Univariate kernel density estimati...

2014-06-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/1093#discussion_r13794760 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/KernelDensity.scala --- @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-2109] Setting SPARK_MEM for bin/pyspark...

2014-06-16 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/1050#discussion_r13794758 --- Diff: bin/run-example --- @@ -27,10 +27,12 @@ if [ -n $1 ]; then EXAMPLE_CLASS=$1 shift else - echo Usage: ./bin/run-example

[GitHub] spark pull request: SPARK-2309: apply output dir existence checkin...

2014-06-16 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/1088#issuecomment-46161534 @pwendell , ah, sorry for the mistake thanks for fixing this --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-1946] Submit stage after (configured ra...

2014-06-16 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r13808991 --- Diff: yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala --- @@ -77,6 +77,12 @@ private[spark] class

[GitHub] spark pull request: [SPARK-1930] The Container is running beyond p...

2014-06-16 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/894#discussion_r13810314 --- Diff: docs/running-on-yarn.md --- @@ -67,6 +67,20 @@ Most of the configs are the same for Spark on YARN as for other deployment modes The

[GitHub] spark pull request: [SPARK-1930] The Container is running beyond p...

2014-06-16 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/894#discussion_r13810329 --- Diff: docs/running-on-yarn.md --- @@ -67,6 +67,20 @@ Most of the configs are the same for Spark on YARN as for other deployment modes The

[GitHub] spark pull request: [SPARK-2051]In yarn.ClientBase spark.yarn.dist...

2014-06-16 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/969#issuecomment-46191458 @witgo As I stated above, the config for client mode is not meant for the end user, it is for internal use that is why I consider this an improvement and not a bug.

[GitHub] spark pull request: [SPARK-1930] The Container is running beyond p...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-46192112 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1930] The Container is running beyond p...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-46192097 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-46192771 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-46192755 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1779] add warning when memoryFraction i...

2014-06-16 Thread scwf
Github user scwf commented on a diff in the pull request: https://github.com/apache/spark/pull/714#discussion_r13811545 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -236,6 +236,27 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging {

[GitHub] spark pull request: spark.storage.memoryFraction indicates the max...

2014-06-16 Thread scwf
Github user scwf closed the pull request at: https://github.com/apache/spark/pull/567 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: add yarn.version for profile yarn and yarn-alp...

2014-06-16 Thread scwf
Github user scwf closed the pull request at: https://github.com/apache/spark/pull/625 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-46198003 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15815/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-46198001 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1930] The Container is running beyond p...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-46198004 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15814/ --- If your project

[GitHub] spark pull request: [SPARK-1930] The Container is running beyond p...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/894#issuecomment-46198002 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-2051]In yarn.ClientBase spark.yarn.dist...

2014-06-16 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/969#issuecomment-46198873 @tgravescs The code should be like this? ClientArguments: ```scala // -archives/--files via spark submit or yarn-client defaults to use file:// if not

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-46199453 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-46199444 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2109] Setting SPARK_MEM for bin/pyspark...

2014-06-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1050#issuecomment-46199870 @ScrapCodes: in that case, wouldn't it be better to just use echo everywhere? Yeah, it's a little more verbose, but doesn't suffer from the indentation issue.

[GitHub] spark pull request: Follow up of PR #1071 for Java API

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1085#issuecomment-46200054 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Follow up of PR #1071 for Java API

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1085#issuecomment-46200040 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-2150: Provide direct link to finished ap...

2014-06-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1094#issuecomment-46201382 Hi @rahulsinghaliitd, Thanks for filing this; this is something I had on my radar but hadn't gotten down to it yet. While I also want the feature, I

[GitHub] spark pull request: [SPARK-2051]In yarn.ClientBase spark.yarn.dist...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/969#issuecomment-46202989 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2051]In yarn.ClientBase spark.yarn.dist...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/969#issuecomment-46203003 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2109] Setting SPARK_MEM for bin/pyspark...

2014-06-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1050#issuecomment-46203327 Yeah, I personally prefer echoes. The indentation thing is a little hard to track sometimes, especially because this is bash. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-1779] add warning when memoryFraction i...

2014-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/714#discussion_r13816162 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -236,6 +236,18 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request: [SPARK-1779] add warning when memoryFraction i...

2014-06-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/714#issuecomment-46203619 Ah yes, this is much better. There is a missing indent. After you fix that I think this is good to go. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-46204734 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15816/ --- If your project

[GitHub] spark pull request: SPARK-2035: Store call stack for stages, displ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/981#issuecomment-46204733 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: Fix for Spark-2151

2014-06-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/1095#issuecomment-46206220 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-2141] Adding getPersistentRddIds and un...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1082#issuecomment-46209130 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Follow up of PR #1071 for Java API

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1085#issuecomment-46209843 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15817/ --- If your

[GitHub] spark pull request: Follow up of PR #1071 for Java API

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1085#issuecomment-46209841 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-2051]In yarn.ClientBase spark.yarn.dist...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/969#issuecomment-46209840 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2051]In yarn.ClientBase spark.yarn.dist...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/969#issuecomment-46209842 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15818/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1090#discussion_r13819928 --- Diff: core/src/main/scala/org/apache/spark/rdd/SortedParitionsRDD.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1090#discussion_r13820071 --- Diff: core/src/main/scala/org/apache/spark/rdd/SortedParitionsRDD.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1090#discussion_r13820032 --- Diff: core/src/main/scala/org/apache/spark/rdd/SortedParitionsRDD.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1090#discussion_r13820570 --- Diff: core/src/main/scala/org/apache/spark/rdd/SortedParitionsRDD.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1090#discussion_r13820746 --- Diff: core/src/main/scala/org/apache/spark/rdd/SortedParitionsRDD.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1090#discussion_r13820794 --- Diff: core/src/main/scala/org/apache/spark/rdd/SortedParitionsRDD.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1090#discussion_r13820829 --- Diff: core/src/main/scala/org/apache/spark/rdd/SortedParitionsRDD.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-2053][SQL] Add Catalyst expressions for...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1055#issuecomment-46213424 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread ash211
Github user ash211 commented on a diff in the pull request: https://github.com/apache/spark/pull/1090#discussion_r13820938 --- Diff: core/src/main/scala/org/apache/spark/rdd/SortedParitionsRDD.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-2010] Support for nested data in PySpar...

2014-06-16 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1041#issuecomment-46213668 Thanks. I've merged this in master branch-1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1090#discussion_r13820994 --- Diff: core/src/main/scala/org/apache/spark/rdd/SortedParitionsRDD.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-2053][SQL] Add Catalyst expressions for...

2014-06-16 Thread concretevitamin
Github user concretevitamin commented on a diff in the pull request: https://github.com/apache/spark/pull/1055#discussion_r13821017 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala --- @@ -917,6 +919,19 @@ private[hive] object HiveQl { case

[GitHub] spark pull request: [SPARK-2010] Support for nested data in PySpar...

2014-06-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1041 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-2150: Provide direct link to finished ap...

2014-06-16 Thread rahulsinghaliitd
Github user rahulsinghaliitd commented on the pull request: https://github.com/apache/spark/pull/1094#issuecomment-46213984 Hi @vanzin, I don't have any preference as to how the URL is formatted and I think that changing the URL can be a separate activity. I am hoping this

[GitHub] spark pull request: SPARK-2149. Univariate kernel density estimati...

2014-06-16 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/1093#discussion_r13821125 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/KernelDensity.scala --- @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-2053][SQL] Add Catalyst expressions for...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1055#issuecomment-46213443 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2141] Adding getPersistentRddIds and un...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1082#issuecomment-46214558 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15819/ --- If your

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1090#issuecomment-46214599 Hi @msiddalingaiah, thanks for adding this much-needed functionality. I haven't looked too closely into the details, but it seems that this shares much of the logic

[GitHub] spark pull request: [SPARK-2141] Adding getPersistentRddIds and un...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1082#issuecomment-46214557 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-983] Support external sorting

2014-06-16 Thread msiddalingaiah
Github user msiddalingaiah commented on the pull request: https://github.com/apache/spark/pull/1090#issuecomment-46218706 Thanks. It's not clear to me if it can use ExternalAppendInlyMap as the underlying buffer either. There was some discussion about how to handle memory management

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-06-16 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r13823899 --- Diff: external/flume/src/main/scala/org/apache/spark/streaming/flume/EventTransformer.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-06-16 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r13823887 --- Diff: external/flume/src/main/scala/org/apache/spark/streaming/flume/EventTransformer.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222354 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222342 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222401 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15821/ --- If your project is set up for it, you can

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222400 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222714 Have made a few changes: * Removed the special SchemaRDD (JsonRDD) for JSON datasets. Now, when users call `jsonFile` and `jsonRDD`, a SchemaRDD is returned. * Added

[GitHub] spark pull request: [SPARK-1201] Do not fully materialize partitio...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1083#issuecomment-46222984 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1201] Do not fully materialize partitio...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1083#issuecomment-46222973 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2053][SQL] Add Catalyst expressions for...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1055#issuecomment-46223170 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2053][SQL] Add Catalyst expressions for...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1055#issuecomment-46223171 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15820/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-1930] The Container is running beyond p...

2014-06-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/894 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Follow up of PR #1071 for Java API

2014-06-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1085 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Follow up of PR #1071 for Java API

2014-06-16 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1085#issuecomment-46224276 Thanks! Merged into master and 1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-06-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r13825922 --- Diff: external/flume/src/main/scala/org/apache/spark/streaming/flume/EventTransformer.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-06-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r13825936 --- Diff: external/flume/src/main/scala/org/apache/spark/streaming/flume/EventTransformer.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-06-16 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/807#discussion_r13826375 --- Diff: external/flume/src/main/scala/org/apache/spark/streaming/flume/EventTransformer.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-1768] History server enhancements.

2014-06-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/718#issuecomment-46225589 No changes, just rebased on top of current master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13826595 --- Diff: docs/sql-programming-guide.md --- @@ -17,20 +17,20 @@ Spark. At the core of this component is a new type of RDD,

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13826671 --- Diff: docs/sql-programming-guide.md --- @@ -62,10 +62,10 @@ descendants. To create a basic SQLContext, all you need is a SparkContext. {%

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13827260 --- Diff: docs/sql-programming-guide.md --- @@ -98,7 +98,9 @@ sqlCtx = SQLContext(sc) /div -## Running SQL on RDDs +# Data Sources

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13827230 --- Diff: docs/sql-programming-guide.md --- @@ -98,7 +98,9 @@ sqlCtx = SQLContext(sc) /div -## Running SQL on RDDs +# Data Sources

[GitHub] spark pull request: [SPARK-1201] Do not fully materialize partitio...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1083#issuecomment-46229031 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

  1   2   3   >