[GitHub] spark pull request #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.s...

2016-10-18 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/15529 [SPARK-17751] [SQL] [Backport-2.0] Remove spark.sql.eagerAnalysis and Output the Plan if Existed in AnalysisException ### What changes were proposed in this pull request? This PR is to backp

[GitHub] spark issue #15496: [SPARK-17950] [Python] Match SparseVector behavior with ...

2016-10-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15496 As I say on the JIRA, if I understand this correctly, this turns O(n) operations into O(n^2) etc. I don't think that actually helps anything. --- If your project is set up for it, you can reply to t

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2016-10-18 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r83792860 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ColumnPruningSuite.scala --- @@ -266,8 +266,8 @@ class ColumnPruningSuite

[GitHub] spark issue #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.sql.eage...

2016-10-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15529 cc @hvanhovell @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2016-10-18 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r83792912 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -536,14 +536,14 @@ class FilterPushdownSu

[GitHub] spark issue #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.sql.eage...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15529 **[Test build #67115 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67115/consoleFull)** for PR 15529 at commit [`ebc6f73`](https://github.com/apache/spark/commit/e

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2016-10-18 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r83793000 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -1016,6 +1016,8 @@ class FilterPushdownSu

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2016-10-18 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r83793204 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizeCodegenSuite.scala --- @@ -51,10 +51,10 @@ class OptimizeCodegenSu

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-18 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/9 @sethah agree with that yes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2016-10-18 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r83793396 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveDDLCommandSuite.scala --- @@ -49,6 +49,11 @@ class HiveDDLCommandSuite extends PlanTest {

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-18 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/9 @yinxusen for cutting lineage do you intend to check whether the `initialModel` itself has an `initialModel`, and if so clear it? I think that can be a reasonable solution if we have a clean

[GitHub] spark issue #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls in cat...

2016-10-18 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/15417 @gatorsmile I've update the PR description and left some comments on changed codes, please comment if there were anything that still unclear. Thank you! --- If your project is set up for it, y

[GitHub] spark issue #15525: [SPARK-17985][CORE] Bump commons-lang3 version to 3.5.

2016-10-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15525 I think this is OK. The general issue is whether it will be compatible with other transitive dependencies, but I believe it would remain so. --- If your project is set up for it, you can reply to th

[GitHub] spark issue #15525: [SPARK-17985][CORE] Bump commons-lang3 version to 3.5.

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15525 **[Test build #67111 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67111/consoleFull)** for PR 15525 at commit [`f318dff`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15525: [SPARK-17985][CORE] Bump commons-lang3 version to 3.5.

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67111/ Test PASSed. ---

[GitHub] spark issue #15525: [SPARK-17985][CORE] Bump commons-lang3 version to 3.5.

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15525 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-10-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13775#discussion_r83796409 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -118,6 +120,11 @@ class OrcFileFormat extends FileFormat with Data

[GitHub] spark pull request #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-10-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13775#discussion_r83796484 --- Diff: sql/hive/src/main/java/org/apache/hadoop/hive/ql/io/orc/VectorizedSparkOrcNewRecordReader.java --- @@ -0,0 +1,318 @@ +/* + * Licensed to th

[GitHub] spark issue #15520: [SPARK-13747][SQL]Fix concurrent executions in ForkJoinP...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15520 **[Test build #67112 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67112/consoleFull)** for PR 15520 at commit [`6aa9e2f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15520: [SPARK-13747][SQL]Fix concurrent executions in ForkJoinP...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15520 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67112/ Test PASSed. ---

[GitHub] spark issue #15520: [SPARK-13747][SQL]Fix concurrent executions in ForkJoinP...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15520 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15519: [WIP][SQL][STREAMING][TEST] Fix flaky tests in Streaming...

2016-10-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15519 Merging this to master and branch 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15327: [SPARK-16575][CORE] partition calculation mismatch with ...

2016-10-18 Thread fidato13
Github user fidato13 commented on the issue: https://github.com/apache/spark/pull/15327 @srowen @rxin @zsxwing Can you please have a look and advise. Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark issue #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrdering` gro...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15480 **[Test build #67116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67116/consoleFull)** for PR 15480 at commit [`1ae9935`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #15519: [SQL][STREAMING][TEST] Fix flaky tests in Streami...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15519 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15528: [SPARK-17899][SQL][follow-up] debug mode should work for...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15528 **[Test build #67114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67114/consoleFull)** for PR 15528 at commit [`5994f07`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15528: [SPARK-17899][SQL][follow-up] debug mode should work for...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15528 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67114/ Test PASSed. ---

[GitHub] spark issue #15528: [SPARK-17899][SQL][follow-up] debug mode should work for...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15528 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactore...

2016-10-18 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/15530 [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored StreamingQueryListener APIs ## What changes were proposed in this pull request? As per @rxin request, here are further API changes - Cha

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15530 **[Test build #67117 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67117/consoleFull)** for PR 15530 at commit [`08a75ea`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrderi...

2016-10-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15480#discussion_r83803058 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala --- @@ -118,7 +118,45 @@ object GenerateOrderi

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15530 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67117/ Test FAILed. ---

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15530 **[Test build #67117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67117/consoleFull)** for PR 15530 at commit [`08a75ea`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15530 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15531: [SQL][STREAMING][TEST] Remove Option.contains for...

2016-10-18 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/15531 [SQL][STREAMING][TEST] Remove Option.contains for Scala 2.10 compatibility ## What changes were proposed in this pull request? Scala 2.10 does not have Option.contains, which broke Scala 2.10

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Remove Option.contains for Scala ...

2016-10-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15531 I will merge this as soon as compilation passes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15530 **[Test build #67119 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67119/consoleFull)** for PR 15530 at commit [`c263914`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Remove Option.contains for Scala ...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15531 **[Test build #67118 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67118/consoleFull)** for PR 15531 at commit [`cd54a4b`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15490: [SPARK-10541] [Web UI] Allow ApplicationHistoryProviders...

2016-10-18 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/15490 @ajbozarth I'm not a spark committer, I'm not capaclbe of getting stuff in. I did dd one comment to some of the code, otherwise nothing I have issues with. LGTM --- If your project is set up

[GitHub] spark issue #15274: [SPARK-17699] Support for parsing JSON string columns

2016-10-18 Thread DanielMe
Github user DanielMe commented on the issue: https://github.com/apache/spark/pull/15274 @yhuai thanks! My impression was that `get_json_object` does not convert json arrays to `ArrayType`s, maybe I misunderstood the way it's supposed to be used though. --- If your project is set up

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Follow up to remove Option.contai...

2016-10-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15531 Ha we have been bit by this like 3 times in the past month. Time to begin agitating to remove 2.10 support. --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Follow up to remove Option.contai...

2016-10-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15531 We should ping the dev list for something like this. For now, I am merging this fix. Compilation has passed in 2.11 in Jenkins, and I locally compiled and tested in 2.10. --- If your project is

[GitHub] spark issue #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.sql.eage...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15529 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67115/ Test PASSed. ---

[GitHub] spark issue #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.sql.eage...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15529 **[Test build #67115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67115/consoleFull)** for PR 15529 at commit [`ebc6f73`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.sql.eage...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15529 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.sql.eage...

2016-10-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15529 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark issue #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to run in th...

2016-10-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15338 Agree that env variables are not a great way to do anything, but that ship has sailed. I don't think we'd remove the 10 that are already here, and if this changed, would be a total rewrite. For that

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83812701 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1448,14 +1452,70 @@ private[spark] object Utils extends Logging { CallSite(short

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83811366 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1448,14 +1452,70 @@ private[spark] object Utils extends Logging { CallSite(short

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83811950 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1448,14 +1452,70 @@ private[spark] object Utils extends Logging { CallSite(short

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83811589 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1448,14 +1452,70 @@ private[spark] object Utils extends Logging { CallSite(short

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83812257 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -274,65 +276,108 @@ class UtilsSuite extends SparkFunSuite with ResetSystemPropertie

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83813908 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -274,65 +276,108 @@ class UtilsSuite extends SparkFunSuite with ResetSystemPropertie

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83811382 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1448,14 +1452,70 @@ private[spark] object Utils extends Logging { CallSite(short

[GitHub] spark pull request #15432: [SPARK-17854][SQL] rand/randn allows null/long as...

2016-10-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15432#discussion_r83814390 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/randomExpressions.scala --- @@ -50,22 +54,27 @@ abstract class RDG extends Le

[GitHub] spark pull request #15432: [SPARK-17854][SQL] rand/randn allows null/long as...

2016-10-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15432#discussion_r83814907 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/randomExpressions.scala --- @@ -77,18 +86,28 @@ case class Rand(seed: Long) e

[GitHub] spark pull request #15531: [SQL][STREAMING][TEST] Follow up to remove Option...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15531 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15530 @rxin @marmbrus Please take a look at the API changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15432: [SPARK-17854][SQL] rand/randn allows null/long as...

2016-10-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15432#discussion_r83817947 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/randomExpressions.scala --- @@ -50,22 +54,27 @@ abstract class RDG exten

[GitHub] spark issue #15517: [SPARK-17972][SQL] Build Datasets upon `withCachedData` ...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15517 **[Test build #67120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67120/consoleFull)** for PR 15517 at commit [`e1283a8`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #15514: [SPARK-17960][PySpark] [Upgrade to Py4J 0.10.4]

2016-10-18 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15514 Awesome, thanks for taking this on @jagadeesanas2, this looks pretty reasonable although it might be helpful if we mentioned briefly the PR description what the Py4J changes were in this release sin

[GitHub] spark issue #15517: [SPARK-17972][SQL] Build Datasets upon `withCachedData` ...

2016-10-18 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/15517 The previous test failure was because we replace the analyzed plan with `withCacheData`, while cache manager uses the original analyzed plan as keys. Force-pushed a new and much simpler ap

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-18 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r83820944 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,14 +74,26 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark issue #15450: [SPARK-3261] [MLLIB] KMeans clusterer can return duplica...

2016-10-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15450 @sethah I should say I am not trying to handle cases where clusters start separate and converge to nearly the same point. I don't that's something we should even try to do. To elaborate, he

[GitHub] spark pull request #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KM...

2016-10-18 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/13557#discussion_r83822272 --- Diff: python/pyspark/ml/clustering.py --- @@ -17,9 +17,11 @@ from pyspark import since, keyword_only from pyspark.ml.util import * -fr

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #67121 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67121/consoleFull)** for PR 11105 at commit [`436ff00`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KM...

2016-10-18 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/13557#discussion_r83822305 --- Diff: python/pyspark/ml/clustering.py --- @@ -201,7 +203,74 @@ def computeCost(self, dataset): """ return self._call_java("compu

[GitHub] spark issue #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrdering` gro...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15480 **[Test build #67116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67116/consoleFull)** for PR 15480 at commit [`1ae9935`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrdering` gro...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15480 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrdering` gro...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15480 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67116/ Test PASSed. ---

[GitHub] spark issue #15436: [SPARK-17875] [BUILD] Remove unneeded direct dependence ...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15436 **[Test build #3360 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3360/consoleFull)** for PR 15436 at commit [`f49f6a6`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15382: [SPARK-17810] [SQL] Default spark.sql.warehouse.d...

2016-10-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15382#discussion_r83823942 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -741,7 +741,7 @@ private[sql] class SQLConf extends Serializable with Ca

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #67121 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67121/consoleFull)** for PR 11105 at commit [`436ff00`](https://github.com/apache/spark/commit/

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67121/ Test FAILed. ---

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Follow up to remove Option.contai...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15531 **[Test build #67118 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67118/consoleFull)** for PR 15531 at commit [`cd54a4b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Follow up to remove Option.contai...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15531 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67118/ Test PASSed. ---

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Follow up to remove Option.contai...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15531 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15508: [DO-NOT-MERGE]

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15508 **[Test build #67122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67122/consoleFull)** for PR 15508 at commit [`59b805e`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15530 **[Test build #67119 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67119/consoleFull)** for PR 15530 at commit [`c263914`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15530 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15530 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67119/ Test PASSed. ---

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-18 Thread lins05
Github user lins05 commented on the issue: https://github.com/apache/spark/pull/15377 Thanks @jerryshao @srowen . I've updated the code like what you suggested. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark issue #15508: [DO-NOT-MERGE]

2016-10-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15508 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if th

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #67124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67124/consoleFull)** for PR 15377 at commit [`b745394`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #15508: [DO-NOT-MERGE]

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15508 **[Test build #67123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67123/consoleFull)** for PR 15508 at commit [`2f5664b`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #15487: [SPARK-17940][SQL] Fixed a typo in LAST function and imp...

2016-10-18 Thread lins05
Github user lins05 commented on the issue: https://github.com/apache/spark/pull/15487 @HyukjinKwon I've updated the usage string. Now it looks like this: ``` spark-sql> describe function first; Function: first Class: org.apache.spark.sql.catalyst.expressions.aggregate.Fi

[GitHub] spark issue #15487: [SPARK-17940][SQL] Fixed a typo in LAST function and imp...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15487 **[Test build #67125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67125/consoleFull)** for PR 15487 at commit [`4c3f043`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15517: [SPARK-17972][SQL] Build Datasets upon `withCachedData` ...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15517 **[Test build #67120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67120/consoleFull)** for PR 15517 at commit [`e1283a8`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15517: [SPARK-17972][SQL] Build Datasets upon `withCachedData` ...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15517 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15517: [SPARK-17972][SQL] Build Datasets upon `withCachedData` ...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15517 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67120/ Test FAILed. ---

[GitHub] spark issue #15487: [SPARK-17940][SQL] Fixed a typo in LAST function and imp...

2016-10-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15487 @lins05 This is being discussed in https://github.com/apache/spark/pull/15513. One idea is, we might have to a bit wait until it is confirmed (although I guess it is a soft-yes). Alternatively,

[GitHub] spark pull request #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrderi...

2016-10-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15480#discussion_r83837138 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -537,7 +537,6 @@ class CodegenContext {

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15505 **[Test build #67126 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67126/consoleFull)** for PR 15505 at commit [`80eed8f`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #15508: [DO-NOT-MERGE]

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15508 **[Test build #67122 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67122/consoleFull)** for PR 15508 at commit [`59b805e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15508: [DO-NOT-MERGE]

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15508 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67122/ Test FAILed. ---

[GitHub] spark issue #15508: [DO-NOT-MERGE]

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15508 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15508: [DO-NOT-MERGE]

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15508 **[Test build #67123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67123/consoleFull)** for PR 15508 at commit [`2f5664b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15508: [DO-NOT-MERGE]

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15508 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15508: [DO-NOT-MERGE]

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15508 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67123/ Test PASSed. ---

  1   2   3   4   5   6   >