[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash

2015-11-21 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158641892 Hmm, do we need to make the hash function result consistent with Hive's? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9885#issuecomment-158645412 **[Test build #46480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46480/consoleFull)** for PR 9885 at commit

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-21 Thread maver1ck
Github user maver1ck commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-158649521 Hi, How can I run this ? Spark 1.6.0-preview1 Compiled with: mvn -e -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package I'm

[GitHub] spark pull request: [SPARK-11898] [MLlib] Use broadcast for the gl...

2015-11-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9878#issuecomment-158649425 I think this looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3580][CORE] Add Consistent Method To Ge...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9767#issuecomment-158652339 **[Test build #2094 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2094/consoleFull)** for PR 9767 at commit

[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash

2015-11-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158649170 Yeah, the point appears to be to match Hive's, so it has to be documented and tested as such. Otherwise a generic unspecified 'hash' function doesn't help much. --- If

[GitHub] spark pull request: [SPARK-3580][CORE] Add Consistent Method To Ge...

2015-11-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9767#issuecomment-158650446 I think this looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3580][CORE] Add Consistent Method To Ge...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9767#issuecomment-158650509 **[Test build #2094 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2094/consoleFull)** for PR 9767 at commit

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9885#issuecomment-158662487 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9885#issuecomment-158662450 **[Test build #46481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46481/consoleFull)** for PR 9885 at commit

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9885#issuecomment-158662488 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10864] [Web UI] app name is hidden if w...

2015-11-21 Thread ajbozarth
Github user ajbozarth commented on the pull request: https://github.com/apache/spark/pull/9874#issuecomment-158673063 Here's a few screenshots. I included a before, an after with the same width as the before, an after right after it wraps and an after at the minimum width.

[GitHub] spark pull request: [SPARK-11880][Windows][Spark Submit] bin/load-...

2015-11-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9863#issuecomment-158637912 LGTM though how did this ever work then? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9026] Refactor SimpleFutureAction.onCom...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7385#issuecomment-158658397 **[Test build #2093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2093/consoleFull)** for PR 7385 at commit

[GitHub] spark pull request: [SPARK-11880][Windows][Spark Submit] bin/load-...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9863#issuecomment-158638902 **[Test build #2092 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2092/consoleFull)** for PR 9863 at commit

[GitHub] spark pull request: [SPARK-9026] Refactor SimpleFutureAction.onCom...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7385#issuecomment-158639432 **[Test build #2093 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2093/consoleFull)** for PR 7385 at commit

[GitHub] spark pull request: [SPARK-4424] Remove spark.driver.allowMultiple...

2015-11-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9865#issuecomment-158638957 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/9885#discussion_r45546783 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala --- @@ -84,6 +84,8 @@ class VectorAssembler(override val uid: String)

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread BenFradet
Github user BenFradet commented on a diff in the pull request: https://github.com/apache/spark/pull/9885#discussion_r45547407 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala --- @@ -84,6 +84,8 @@ class VectorAssembler(override val uid: String)

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9885#issuecomment-158659615 **[Test build #46481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46481/consoleFull)** for PR 9885 at commit

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-21 Thread nburoojy
Github user nburoojy commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-158661662 This is a wrapper around the Hive collect fns. Try compiling with `-Phive -Phive-thriftserver` --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-11880][Windows][Spark Submit] bin/load-...

2015-11-21 Thread toddwan
Github user toddwan commented on the pull request: https://github.com/apache/spark/pull/9863#issuecomment-158661692 I guess users on Windows platform seldom touch `spark-env.cmd`, and have lots of workarounds if they ran into this issue. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread BenFradet
GitHub user BenFradet opened a pull request: https://github.com/apache/spark/pull/9885 [SPARK-11902] [ML] Unhandled case in VectorAssembler#transform There is an unhandled case in the transform method of VectorAssembler if one of the input columns doesn't have one of the supported

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9885#issuecomment-158649804 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9885#issuecomment-158649771 **[Test build #46480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46480/consoleFull)** for PR 9885 at commit

[GitHub] spark pull request: [SPARK-11902] [ML] Unhandled case in VectorAss...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9885#issuecomment-158649802 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11859][Mesos] SparkContext accepts inva...

2015-11-21 Thread toddwan
GitHub user toddwan opened a pull request: https://github.com/apache/spark/pull/9886 [SPARK-11859][Mesos] SparkContext accepts invalid Master URLs in the form zk://host:port for a multi-master Mesos cluster using ZooKeeper * According to below doc and validation logic in

[GitHub] spark pull request: [SPARK-9301] [SQL] Add collect_set and collect...

2015-11-21 Thread maver1ck
Github user maver1ck commented on the pull request: https://github.com/apache/spark/pull/9526#issuecomment-158662413 Thanks. And when I compile with hive is there a chance to do sth like this ? "select id, collect_list(table.*) as data from table group by id" ? --- If your

[GitHub] spark pull request: [SPARK-10864] [Web UI] app name is hidden if w...

2015-11-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9874#issuecomment-158636924 This is probably fine but yeah would be good to see screenshots. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-11551][DOC][Example]Replace example cod...

2015-11-21 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/9735#issuecomment-158640370 @somideshmukh Do you still have time on this? I can help if you are busy. Pls let me know. :) --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash

2015-11-21 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158649259 Thanks. I will update this later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-11796] [test-maven] [WIP] Fixing httpcl...

2015-11-21 Thread markgrover
Github user markgrover commented on the pull request: https://github.com/apache/spark/pull/9876#issuecomment-158651416 OK, I will do that. I am afk for some time but will take care of this when I am back. Thanks. On Nov 20, 2015 10:12 PM, "Josh Rosen"

[GitHub] spark pull request: [SPARK-11859][Mesos] SparkContext accepts inva...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9886#issuecomment-158661212 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11859][Mesos] SparkContext accepts inva...

2015-11-21 Thread dragos
Github user dragos commented on the pull request: https://github.com/apache/spark/pull/9886#issuecomment-158667956 @andrewor14 I wonder if we shouldn't first warn about this, and defer the actual failure until 2.0. There might be people relying on this loophole. If I understand

[GitHub] spark pull request: [SPARK-2365] Add IndexedRDD, an efficient upda...

2015-11-21 Thread josephlijia
Github user josephlijia commented on the pull request: https://github.com/apache/spark/pull/1297#issuecomment-158647538 When we looked up one certain key-value by IndexedRDD, we found that it was even slower than ordinary RDD. We use 100, keys in our experiment. When we tested it

[GitHub] spark pull request: [SPARK-11880][Windows][Spark Submit] bin/load-...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9863#issuecomment-158650306 **[Test build #2092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2092/consoleFull)** for PR 9863 at commit

[GitHub] spark pull request: [SPARK-5095][MESOS] Support capping cores and ...

2015-11-21 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/4027#issuecomment-158668110 @andrewor14 I've updated the patch now. Originally you suggested me to look at deploy/master.scala to try to use the same configurations like spark.executor.cores. But

[GitHub] spark pull request: [SPARK-6990] [Build] Add Java linting script; ...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9867#issuecomment-158671998 **[Test build #46482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46482/consoleFull)** for PR 9867 at commit

[GitHub] spark pull request: [SPARK-11482][SQL] Make maven repo for Hive me...

2015-11-21 Thread xguo27
Github user xguo27 commented on the pull request: https://github.com/apache/spark/pull/9543#issuecomment-158694089 Sorry about the failure, can we re-test please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11881][SQL] Fix for postgresql fetchsiz...

2015-11-21 Thread mariusvniekerk
Github user mariusvniekerk commented on the pull request: https://github.com/apache/spark/pull/9861#issuecomment-158701845 Not entirely sure why this causes NPE exceptions in some of the unit tests... --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-11905] [SQL] Support Persist/Cache and ...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9889#issuecomment-158705743 **[Test build #46485 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46485/consoleFull)** for PR 9889 at commit

[GitHub] spark pull request: [SPARK-11778][SQL]:add regression test

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9890#issuecomment-158716152 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11778][SQL]:add regression test

2015-11-21 Thread huaxingao
GitHub user huaxingao opened a pull request: https://github.com/apache/spark/pull/9890 [SPARK-11778][SQL]:add regression test Fix regression test for SPARK-11778. @marmbrus Could you please take a look? Thank you very much!! You can merge this pull request into a Git

[GitHub] spark pull request: [SPARK-11837] [EC2] python3 compatibility for ...

2015-11-21 Thread mortada
Github user mortada commented on the pull request: https://github.com/apache/spark/pull/9797#issuecomment-158695890 @JoshRosen Jenkins seemed to have failed again, but this PR should be good to go --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-11905] [SQL] Support Persist/Cache and ...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9889#issuecomment-158705761 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11905] [SQL] Support Persist/Cache and ...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9889#issuecomment-158705760 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11856][SQL] add type cast if the real t...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9840#issuecomment-158735571 **[Test build #46486 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46486/consoleFull)** for PR 9840 at commit

[GitHub] spark pull request: [SPARK-11856][SQL] add type cast if the real t...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9840#issuecomment-158735602 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11856][SQL] add type cast if the real t...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9840#issuecomment-158735601 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11856][SQL] add type cast if the real t...

2015-11-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/9840#discussion_r4624 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -124,17 +124,46 @@ object ScalaReflection extends

[GitHub] spark pull request: [SPARK-11905] Support Persist/Cache and Unpers...

2015-11-21 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/9889 [SPARK-11905] Support Persist/Cache and Unpersist in Dataset APIs Persist and Unpersist exist in both RDD and Dataframe APIs. I think they are still very critical in Dataset APIs. Not sure my

[GitHub] spark pull request: [SPARK-11856][SQL] add type cast if the real t...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9840#issuecomment-158714817 **[Test build #46486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46486/consoleFull)** for PR 9840 at commit

[GitHub] spark pull request: [WIP] [SPARK-11327] [MESOS] Dispatcher does no...

2015-11-21 Thread jayv
Github user jayv commented on the pull request: https://github.com/apache/spark/pull/9752#issuecomment-158715518 I will get to it on Monday. - Jo Voordeckers On Sat, Nov 21, 2015 at 2:31 PM, Iulian Dragos wrote: > @jayv

[GitHub] spark pull request: [SPARK-5337][Mesos][Standalone] respect spark....

2015-11-21 Thread dragos
Github user dragos commented on a diff in the pull request: https://github.com/apache/spark/pull/8610#discussion_r45551887 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -639,10 +640,11 @@ private[deploy] class Master( // in the queue,

[GitHub] spark pull request: [SPARK-11482][SQL] Make maven repo for Hive me...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9543#issuecomment-158689288 **[Test build #2095 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2095/consoleFull)** for PR 9543 at commit

[GitHub] spark pull request: [SPARK-9026] Modifications to JobWaiter, Futur...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9264#issuecomment-158689502 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11482][SQL] Make maven repo for Hive me...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9543#issuecomment-158689476 **[Test build #2095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2095/consoleFull)** for PR 9543 at commit

[GitHub] spark pull request: [SPARK-9026] Modifications to JobWaiter, Futur...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9264#issuecomment-158689474 **[Test build #46483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46483/consoleFull)** for PR 9264 at commit

[GitHub] spark pull request: [SPARK-11904] [PySpark] reduceByKeyAndWindow d...

2015-11-21 Thread dtolpin
GitHub user dtolpin opened a pull request: https://github.com/apache/spark/pull/9888 [SPARK-11904] [PySpark] reduceByKeyAndWindow does not require checkpointing when invFunc is None when invFunc is None, `reduceByKeyAndWindow(func, None, winsize, slidesize)` is equivalent to

[GitHub] spark pull request: [SPARK-11904] [PySpark] reduceByKeyAndWindow d...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9887#issuecomment-158688910 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11899][SQL] API audit for GroupedDatase...

2015-11-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9880#discussion_r45552122 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/GroupedDataset.scala --- @@ -36,11 +37,13 @@ import org.apache.spark.sql.execution.QueryExecution *

[GitHub] spark pull request: [SPARK-9026] Modifications to JobWaiter, Futur...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9264#issuecomment-158689503 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB][SPARK-7615, SPARK-7617, SPARK-7618]: A...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6245#issuecomment-158689542 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11899][SQL] API audit for GroupedDatase...

2015-11-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9880#issuecomment-158679126 LGTM aside from one minor variable naming nit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11859][Mesos] SparkContext accepts inva...

2015-11-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9886#issuecomment-158680586 LGTM, though I tend to agree there's a little risk here in making something that shouldn't work actually not work. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-9026] Modifications to JobWaiter, Futur...

2015-11-21 Thread reggert
Github user reggert commented on the pull request: https://github.com/apache/spark/pull/9264#issuecomment-158680579 I've come up with a reusable way to make use of semaphores to control timing of tasks during unit tests. Please see the `Smuggle` class and let me know what you think.

[GitHub] spark pull request: [SPARK-11904] [PySpark] reduceByKeyAndWindow d...

2015-11-21 Thread dtolpin
GitHub user dtolpin opened a pull request: https://github.com/apache/spark/pull/9887 [SPARK-11904] [PySpark] reduceByKeyAndWindow does not require checkpointing when invFunc is None when invFunc is None, `reduceByKeyAndWindow(func, None, winsize, slidesize)` is equivalent to

[GitHub] spark pull request: [SPARK-11904] [PySpark] reduceByKeyAndWindow d...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9888#issuecomment-158690246 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9026] Modifications to JobWaiter, Futur...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9264#issuecomment-158682569 **[Test build #46483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46483/consoleFull)** for PR 9264 at commit

[GitHub] spark pull request: [SPARK-6990] [Build] Add Java linting script; ...

2015-11-21 Thread dskrvk
Github user dskrvk commented on the pull request: https://github.com/apache/spark/pull/9867#issuecomment-158683916 Added some more commits so that new changes are in line with the style guide. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-11899][SQL] API audit for GroupedDatase...

2015-11-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9880 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11904] [PySpark] reduceByKeyAndWindow d...

2015-11-21 Thread dtolpin
Github user dtolpin commented on the pull request: https://github.com/apache/spark/pull/9887#issuecomment-158689771 I did something wrong with rebasing, will remove and redo. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [WIP] [SPARK-11327] [MESOS] Dispatcher does no...

2015-11-21 Thread dragos
Github user dragos commented on the pull request: https://github.com/apache/spark/pull/9752#issuecomment-158687388 @jayv will you have time to update this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11904] [PySpark] reduceByKeyAndWindow d...

2015-11-21 Thread dtolpin
Github user dtolpin closed the pull request at: https://github.com/apache/spark/pull/9887 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11905] [SQL] Support Persist/Cache and ...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9889#issuecomment-158698890 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11905] [SQL] Support Persist/Cache and ...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9889#issuecomment-158698891 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11905] [SQL] Support Persist/Cache and ...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9889#issuecomment-158698821 **[Test build #46485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46485/consoleFull)** for PR 9889 at commit

[GitHub] spark pull request: [SPARK-11482][SQL] Make maven repo for Hive me...

2015-11-21 Thread xguo27
Github user xguo27 commented on the pull request: https://github.com/apache/spark/pull/9543#issuecomment-158678073 @rxin Thanks, Reynold! Somehow no test was triggered. Not sure why. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-11628][SQL] support column datatype of ...

2015-11-21 Thread xguo27
Github user xguo27 commented on the pull request: https://github.com/apache/spark/pull/9612#issuecomment-158678513 @cloud-fan I have added a few tests per your suggestion. Do they look good to you? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-11899][SQL] API audit for GroupedDatase...

2015-11-21 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9880#discussion_r45550214 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/GroupedDataset.scala --- @@ -36,11 +37,13 @@ import org.apache.spark.sql.execution.QueryExecution

[GitHub] spark pull request: [SPARK-6990] [Build] Add Java linting script; ...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9867#issuecomment-158683704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6990] [Build] Add Java linting script; ...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9867#issuecomment-158683645 **[Test build #46482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46482/consoleFull)** for PR 9867 at commit

[GitHub] spark pull request: [SPARK-6990] [Build] Add Java linting script; ...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9867#issuecomment-158683705 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5095][MESOS] Support capping cores and ...

2015-11-21 Thread dragos
Github user dragos commented on the pull request: https://github.com/apache/spark/pull/4027#issuecomment-158686949 @tnachen I think this trade-off has been discussed in [this comment](https://github.com/apache/spark/pull/4027#issuecomment-92553493) and the following three. Since

[GitHub] spark pull request: [SPARK-11900][SQL] Add since version for all e...

2015-11-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9881 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11901][SQL] API audit for Aggregator.

2015-11-21 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/9882 [SPARK-11901][SQL] API audit for Aggregator. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark SPARK-11901 Alternatively you can review

[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158613140 **[Test build #46475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46475/consoleFull)** for PR 9883 at commit

[GitHub] spark pull request: [SPARK-11871] Add save/load for MLPC

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9854#issuecomment-158617009 **[Test build #46478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46478/consoleFull)** for PR 9854 at commit

[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158625782 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158625768 **[Test build #46475 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46475/consoleFull)** for PR 9883 at commit

[GitHub] spark pull request: [SPARK-8233][SQL] Add misc function hash

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9883#issuecomment-158625781 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11891] Model export/import for RFormula...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9884#issuecomment-158613288 **[Test build #46476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46476/consoleFull)** for PR 9884 at commit

[GitHub] spark pull request: [SPARK-5095][MESOS] Support capping cores and ...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4027#issuecomment-158614042 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11899][SQL] API audit for GroupedDatase...

2015-11-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9880#issuecomment-158616553 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11901][SQL] API audit for Aggregator.

2015-11-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9882 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11871] Add save/load for MLPC

2015-11-21 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/9854#issuecomment-158616947 Pls ignore my first comment, I have splited the test of model and classifier separately. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-11891] Model export/import for RFormula...

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9884#issuecomment-158626876 **[Test build #46479 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46479/consoleFull)** for PR 9884 at commit

[GitHub] spark pull request: [SPARK-11901][SQL] API audit for Aggregator.

2015-11-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9882#issuecomment-158612426 **[Test build #46474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46474/consoleFull)** for PR 9882 at commit

[GitHub] spark pull request: [SPARK-11900][SQL] Add since version for all e...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9881#issuecomment-158615833 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11900][SQL] Add since version for all e...

2015-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9881#issuecomment-158615832 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

  1   2   >