[GitHub] spark pull request: [SPARK-14394][SQL] Generate AggregateHashMap c...

2016-04-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12161#discussion_r58978809 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregateHashMap.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14451][SQL] Move encoder definition int...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12231#issuecomment-207203325 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14298] [ML] [MLlib] LDA should support ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12089#issuecomment-207203368 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14451][SQL] Move encoder definition int...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12231#issuecomment-207203324 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207203276 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207203274 **[Test build #55311 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55311/consoleFull)** for PR 12215 at commit [`30eb58d`](https://g

[GitHub] spark pull request: [SPARK-14298] [ML] [MLlib] LDA should support ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12089#issuecomment-207203292 **[Test build #55310 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55310/consoleFull)** for PR 12089 at commit [`4af96d8`](https://g

[GitHub] spark pull request: [SPARK-14451][SQL] Move encoder definition int...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12231#issuecomment-207203244 **[Test build #55299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55299/consoleFull)** for PR 12231 at commit [`ca1b47e`](https://g

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207203275 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14394][SQL] Generate AggregateHashMap c...

2016-04-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12161#discussion_r58978741 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregateHashMap.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14394][SQL] Generate AggregateHashMap c...

2016-04-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12161#discussion_r58978723 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregateHashMap.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207203126 **[Test build #55311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55311/consoleFull)** for PR 12215 at commit [`30eb58d`](https://gi

[GitHub] spark pull request: [SPARK-14394][SQL] Generate AggregateHashMap c...

2016-04-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12161#discussion_r58978685 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregateHashMap.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14394][SQL] Generate AggregateHashMap c...

2016-04-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12161#discussion_r58978611 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregateHashMap.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14394][SQL] Generate AggregateHashMap c...

2016-04-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12161#discussion_r58978578 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregateHashMap.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207202641 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-14394][SQL] Generate AggregateHashMap c...

2016-04-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12161#discussion_r58978492 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregateHashMap.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14394][SQL] Generate AggregateHashMap c...

2016-04-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12161#discussion_r58978122 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregateHashMap.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207200571 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207200574 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207200075 **[Test build #55294 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55294/consoleFull)** for PR 12215 at commit [`e2237bd`](https://g

[GitHub] spark pull request: [SPARK-14275][SQL] Reimplement TypedAggregateE...

2016-04-07 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12067#issuecomment-207199587 Well it's not cheating if the user doesn't need to explicitly reuse. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-207198913 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r58977765 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,8 +62,14 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]]) ex

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-207198914 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/5

[GitHub] spark pull request: [WIP][SPARK-14408][CORE] Changed RDD.treeAggre...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12217#issuecomment-207197276 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [WIP][SPARK-14408][CORE] Changed RDD.treeAggre...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12217#issuecomment-207197273 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14460] [SQL] properly handling of colum...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12252#issuecomment-207197063 **[Test build #55309 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55309/consoleFull)** for PR 12252 at commit [`88935c5`](https://gi

[GitHub] spark pull request: [SPARK-14298] [ML] [MLlib] LDA should support ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12089#issuecomment-207197076 **[Test build #55310 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55310/consoleFull)** for PR 12089 at commit [`4af96d8`](https://gi

[GitHub] spark pull request: [WIP][SPARK-14408][CORE] Changed RDD.treeAggre...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12217#issuecomment-207197072 **[Test build #55296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55296/consoleFull)** for PR 12217 at commit [`02d107a`](https://g

[GitHub] spark pull request: [SPARK-14460] [SQL] properly handling of colum...

2016-04-07 Thread bomeng
GitHub user bomeng opened a pull request: https://github.com/apache/spark/pull/12252 [SPARK-14460] [SQL] properly handling of column name contains space ## What changes were proposed in this pull request? Although it is not recommended, table can be created with column name

[GitHub] spark pull request: [SPARK-14298] [ML] [MLlib] LDA should support ...

2016-04-07 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/12089#issuecomment-207196623 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207196573 **[Test build #55308 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55308/consoleFull)** for PR 12247 at commit [`d161f3a`](https://gi

[GitHub] spark pull request: [SPARK-14357] [CORE] Properly handle the root ...

2016-04-07 Thread jasonmoore2k
Github user jasonmoore2k commented on the pull request: https://github.com/apache/spark/pull/12228#issuecomment-207196510 @andrewor14, think it's safe to just retest? I don't think the fatal error was because of these changes. I don't have the permissions to start a test run. ---

[GitHub] spark pull request: [SPARK-6717][ML] Clear shuffle files after che...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11919#issuecomment-207196167 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-6717][ML] Clear shuffle files after che...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11919#issuecomment-207196165 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-6717][ML] Clear shuffle files after che...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11919#issuecomment-207195938 **[Test build #55295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55295/consoleFull)** for PR 11919 at commit [`c1493a1`](https://g

[GitHub] spark pull request: [SPARK-14275][SQL] Reimplement TypedAggregateE...

2016-04-07 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12067#issuecomment-207195627 And I think "reuse a single object" should help, as then we only need to create one object for one partition. But it's like cheating, because RDD doesn't reuse the ob

[GitHub] spark pull request: [SPARK-14474][SQL]Move FileSource offset log i...

2016-04-07 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/12247#issuecomment-207195658 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-207195538 **[Test build #55266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55266/consoleFull)** for PR 9565 at commit [`648c7b2`](https://git

[GitHub] spark pull request: [SPARK-14275][SQL] Reimplement TypedAggregateE...

2016-04-07 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12067#issuecomment-207195178 In the benchmark, for RDD, we first apply a function to turn a long into a `Data`, then do aggregate. For Dataset, we first turn a long to a `UTFString`, then turn th

[GitHub] spark pull request: [WIP, DO-NOT-MERGE][SPARK-14473][SQL] Define a...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12246#issuecomment-207195067 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [WIP, DO-NOT-MERGE][SPARK-14473][SQL] Define a...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12246#issuecomment-207195056 **[Test build #55306 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55306/consoleFull)** for PR 12246 at commit [`6e8c957`](https://g

[GitHub] spark pull request: [WIP, DO-NOT-MERGE][SPARK-14473][SQL] Define a...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12246#issuecomment-207195066 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13597][PySpark][ML] Python API for Gene...

2016-04-07 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/11468#issuecomment-207194961 @vectorijk This PR looks good overall, please address my last comments. Thanks! --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark pull request: [SPARK-13597][PySpark][ML] Python API for Gene...

2016-04-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/11468#discussion_r58976307 --- Diff: python/pyspark/ml/regression.py --- @@ -934,6 +935,146 @@ def predict(self, features): return self._call_java("predict", features)

[GitHub] spark pull request: [SPARK-14352][SQL] approxQuantile should suppo...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12135#issuecomment-207194500 **[Test build #55307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55307/consoleFull)** for PR 12135 at commit [`c9ebfef`](https://gi

[GitHub] spark pull request: [WIP, DO-NOT-MERGE][SPARK-14473][SQL] Define a...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12246#issuecomment-207194503 **[Test build #55306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55306/consoleFull)** for PR 12246 at commit [`6e8c957`](https://gi

[GitHub] spark pull request: [SPARK-14423][YARN] Avoid same name files adde...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12203#issuecomment-207194264 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14423][YARN] Avoid same name files adde...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12203#issuecomment-207194261 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207194124 **[Test build #55305 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55305/consoleFull)** for PR 12215 at commit [`30eb58d`](https://g

[GitHub] spark pull request: [SPARK-14357] [CORE] Properly handle the root ...

2016-04-07 Thread jasonmoore2k
Github user jasonmoore2k commented on the pull request: https://github.com/apache/spark/pull/12228#issuecomment-207194131 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-14423][YARN] Avoid same name files adde...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12203#issuecomment-207193962 **[Test build #55302 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55302/consoleFull)** for PR 12203 at commit [`e1b09c4`](https://g

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207194136 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207194133 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207193356 **[Test build #55305 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55305/consoleFull)** for PR 12215 at commit [`30eb58d`](https://gi

[GitHub] spark pull request: [SPARK-14275][SQL] Reimplement TypedAggregateE...

2016-04-07 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12067#issuecomment-207193129 The part I don't get is that even in the RDD case, we'd need to create an object per row. This is equivalent to the "deserialization" in aggregator, since they both just c

[GitHub] spark pull request: [SPARK-14357] [CORE] Properly handle the root ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12228#issuecomment-207193145 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14435][BUILD] Shade Kryo in our custom ...

2016-04-07 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/12215#issuecomment-207193168 org.spark-project.hive:hive-exec:1.2.1.spark2 is now on Maven Central, so this PR is no longer WIP. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-14357] [CORE] Properly handle the root ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12228#issuecomment-207193142 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14357] [CORE] Properly handle the root ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12228#issuecomment-207193064 **[Test build #55293 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55293/consoleFull)** for PR 12228 at commit [`d5ebd55`](https://g

[GitHub] spark pull request: [SPARK-13597][PySpark][ML] Python API for Gene...

2016-04-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/11468#discussion_r58975761 --- Diff: python/pyspark/ml/regression.py --- @@ -934,6 +935,146 @@ def predict(self, features): return self._call_java("predict", features)

[GitHub] spark pull request: [SPARK-14275][SQL] Reimplement TypedAggregateE...

2016-04-07 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12067#issuecomment-207192990 if we can reuse a single object and mutate the object in place, would it be the same speed? --- If your project is set up for it, you can reply to this email and have yo

[GitHub] spark pull request: [SPARK-14465][BUILD] Checkstyle should check a...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12242#issuecomment-207191987 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13842][PYSPARK] pyspark.sql.types.Struc...

2016-04-07 Thread skparkes
Github user skparkes commented on the pull request: https://github.com/apache/spark/pull/12251#issuecomment-207192169 It appears the user documentation is nicely scraped from the docstrings, so I went ahead and expanded the docstring slightly (and added a couple `doctest`s).

[GitHub] spark pull request: [SPARK-14465][BUILD] Checkstyle should check a...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12242#issuecomment-207191985 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14275][SQL] Reimplement TypedAggregateE...

2016-04-07 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/12067#issuecomment-207191252 @rxin , because aggregator needs to deserialize internal row to object fist, then call aggregator methods. --- If your project is set up for it, you can reply to th

[GitHub] spark pull request: [SPARK-14465][BUILD] Checkstyle should check a...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12242#issuecomment-207191161 **[Test build #55303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55303/consoleFull)** for PR 12242 at commit [`832b801`](https://gi

[GitHub] spark pull request: [SPARK-14415][SQL] All functions should show u...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12185#issuecomment-207191187 **[Test build #55304 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55304/consoleFull)** for PR 12185 at commit [`e41326b`](https://gi

[GitHub] spark pull request: [SPARK-12569][PySpark][ML]:DecisionTreeRegress...

2016-04-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12116#discussion_r58975356 --- Diff: python/pyspark/ml/regression.py --- @@ -425,6 +429,9 @@ class DecisionTreeRegressor(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredi

[GitHub] spark pull request: [SPARK-13538][ML] Add GaussianMixture to ML

2016-04-07 Thread zhengruifeng
Github user zhengruifeng commented on the pull request: https://github.com/apache/spark/pull/11419#issuecomment-207190370 @jkbradley Thanks. BTW, I have three minor PRs for DOC, and there is a whiile since I open them. Do you mind if I cc you at those PRs and you give a glimpse in you

[GitHub] spark pull request: [SPARK-14423][YARN] Avoid same name files adde...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12203#issuecomment-207189976 **[Test build #55302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55302/consoleFull)** for PR 12203 at commit [`e1b09c4`](https://gi

[GitHub] spark pull request: [SPARK-12569][PySpark][ML]:DecisionTreeRegress...

2016-04-07 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/12116#issuecomment-207189042 +1 @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-207188890 **[Test build #55301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55301/consoleFull)** for PR 9565 at commit [`2a0c319`](https://gith

[GitHub] spark pull request: [SPARK-12569][PySpark][ML]:DecisionTreeRegress...

2016-04-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12116#discussion_r58975014 --- Diff: python/pyspark/ml/regression.py --- @@ -454,12 +461,12 @@ def __init__(self, featuresCol="features", labelCol="label", predictionCol="pred

[GitHub] spark pull request: [SPARK-14465][BUILD] Checkstyle should check a...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12242#issuecomment-207188874 **[Test build #55256 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55256/consoleFull)** for PR 12242 at commit [`582177d`](https://g

[GitHub] spark pull request: [SPARK-12569][PySpark][ML]:DecisionTreeRegress...

2016-04-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12116#discussion_r58974996 --- Diff: python/pyspark/ml/regression.py --- @@ -454,12 +461,12 @@ def __init__(self, featuresCol="features", labelCol="label", predictionCol="pred

[GitHub] spark pull request: [SPARK-12569][PySpark][ML]:DecisionTreeRegress...

2016-04-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12116#discussion_r58974920 --- Diff: python/pyspark/ml/regression.py --- @@ -433,12 +440,12 @@ class DecisionTreeRegressor(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredi

[GitHub] spark pull request: [SPARK-14477][BUILD] Allow custom mirrors for ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12250#issuecomment-207188553 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12569][PySpark][ML]:DecisionTreeRegress...

2016-04-07 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/12116#discussion_r58974903 --- Diff: python/pyspark/ml/regression.py --- @@ -433,12 +440,12 @@ class DecisionTreeRegressor(JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredi

[GitHub] spark pull request: [SPARK-14477][BUILD] Allow custom mirrors for ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12250#issuecomment-207188555 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14477][BUILD] Allow custom mirrors for ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12250#issuecomment-207188418 **[Test build #55284 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55284/consoleFull)** for PR 12250 at commit [`fcbcf26`](https://g

[GitHub] spark pull request: [SPARK-14357] [CORE] Properly handle the root ...

2016-04-07 Thread jasonmoore2k
Github user jasonmoore2k commented on the pull request: https://github.com/apache/spark/pull/12228#issuecomment-207188334 Flaky test? The second test that is running has passed that one. --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request: [SPARK-13842][PYSPARK] pyspark.sql.types.Struc...

2016-04-07 Thread skparkes
Github user skparkes commented on the pull request: https://github.com/apache/spark/pull/12251#issuecomment-207188208 I updated the pull request description, but I also realize your comment could have meant the user documentation. I'll spend a moment to see if I can find the appropri

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-207188224 **[Test build #55300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55300/consoleFull)** for PR 9565 at commit [`648c7b2`](https://gith

[GitHub] spark pull request: [SPARK-13048][ML][MLLIB] keepLastCheckpoint op...

2016-04-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12166 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2016-04-07 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-207187370 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-14132][SPARK-14133][SQL] Alter table pa...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12220#issuecomment-207187142 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14132][SPARK-14133][SQL] Alter table pa...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12220#issuecomment-207187148 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13048][ML][MLLIB] keepLastCheckpoint op...

2016-04-07 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/12166#issuecomment-207186805 Merging with master @holdenk @hhbyyh Thanks for taking a look! --- If your project is set up for it, you can reply to this email and have your reply appear on Git

[GitHub] spark pull request: [SPARK-14132][SPARK-14133][SQL] Alter table pa...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12220#issuecomment-207186730 **[Test build #55286 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55286/consoleFull)** for PR 12220 at commit [`220141d`](https://g

[GitHub] spark pull request: [SPARK-14357] [CORE] Properly handle the root ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12228#issuecomment-207184857 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14357] [CORE] Properly handle the root ...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12228#issuecomment-207184850 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207184221 LGTM. This PR dramatically improves our s3 performance at Netflix. @andrewor14 @srowen @JoshRosen @davies @marmbrus @yhuai, any further feedback? Thanks. ---

[GitHub] spark pull request: [SPARK-14275][SQL] Reimplement TypedAggregateE...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12067#issuecomment-207184158 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14357] [CORE] Properly handle the root ...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12228#issuecomment-207184154 **[Test build #55288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55288/consoleFull)** for PR 12228 at commit [`64a4499`](https://g

[GitHub] spark pull request: [SPARK-14275][SQL] Reimplement TypedAggregateE...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12067#issuecomment-207184159 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14275][SQL] Reimplement TypedAggregateE...

2016-04-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12067#issuecomment-207183673 **[Test build #55289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55289/consoleFull)** for PR 12067 at commit [`5f6510e`](https://g

[GitHub] spark pull request: [SPARK-13842][PYSPARK] pyspark.sql.types.Struc...

2016-04-07 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12251#issuecomment-207182939 Can you update the description to include the ways that are being expanded? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-04-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-207181095 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

<    1   2   3   4   5   6   7   8   9   >