[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15535 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67140/ Test FAILed. ---

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15535 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15407: [SPARK-17841][STREAMING][KAFKA] drain commitQueue

2016-10-18 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15407 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

[GitHub] spark pull request #15407: [SPARK-17841][STREAMING][KAFKA] drain commitQueue

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15407 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15490: [SPARK-10541] [Web UI] Allow ApplicationHistoryPr...

2016-10-18 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/15490#discussion_r83950235 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -262,6 +263,17 @@ private[history] class FsHistoryProvider(con

[GitHub] spark issue #15407: [SPARK-17841][STREAMING][KAFKA] drain commitQueue

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15407 Thanks for the explanation. Merging in master/branch-2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15530 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #15526: [SPARK-17986] [ML] SQLTransformer should remove temporar...

2016-10-18 Thread drewrobb
Github user drewrobb commented on the issue: https://github.com/apache/spark/pull/15526 @yanboliang would you be able to review this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15535 **[Test build #67141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67141/consoleFull)** for PR 15535 at commit [`9549e58`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15535 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #15525: [SPARK-17985][CORE] Bump commons-lang3 version to 3.5.

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15525 Actually I'm seeing the following exceptions locally as well as on Jenkins for Hadoop 2.2: ``` [error] /scratch/rxin/spark/core/src/main/scala/org/apache/spark/util/Utils.scala:1489: valu

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15530 @rxin Can you take a look once again. Note that there is a separate PR for the same changes in branch-2.0 (with different Mima excludes) - https://github.com/apache/spark/pull/15535/files --- If y

[GitHub] spark issue #15450: [SPARK-3261] [MLLIB] KMeans clusterer can return duplica...

2016-10-18 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15450 I don't feel strongly either way, but I don't like the potential of this: scala model.getK scala> 3 model.clusterCenters.length scala> 1 Should we conside

[GitHub] spark issue #15521: [SPARK-17980] [SQL] Fix refreshByPath for converted Hive...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15521 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15521: [SPARK-17980] [SQL] Fix refreshByPath for converted Hive...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15521 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67137/ Test PASSed. ---

[GitHub] spark issue #15521: [SPARK-17980] [SQL] Fix refreshByPath for converted Hive...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15521 **[Test build #67137 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67137/consoleFull)** for PR 15521 at commit [`5088ae0`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15535 **[Test build #67140 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67140/consoleFull)** for PR 15535 at commit [`0fe56d2`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.s...

2016-10-18 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/15529 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.sql.eage...

2016-10-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15529 Thanks! Close it now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15499: [SPARK-17955][SQL] Make DataFrameReader.jdbc call...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15499 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15512: [SPARK-17930][CORE]The SerializerInstance instanc...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15512 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactore...

2016-10-18 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/15535 [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored StreamingQueryListener APIs for branch-2.0 This is the branch-2.0 PR of #15530 to make the APIs consistent with the master. Since these APIs are exp

[GitHub] spark issue #15499: [SPARK-17955][SQL] Make DataFrameReader.jdbc call DataFr...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15499 Merging in master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15512: [SPARK-17930][CORE]The SerializerInstance instance used ...

2016-10-18 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15512 LGTM. Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request #15481: [SPARK-17929] [CORE] Fix deadlock when CoarseGrai...

2016-10-18 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15481#discussion_r83946127 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -92,19 +92,19 @@ class CoarseGrainedSchedulerBac

[GitHub] spark pull request #15481: [SPARK-17929] [CORE] Fix deadlock when CoarseGrai...

2016-10-18 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15481#discussion_r83946120 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -392,9 +392,23 @@ class CoarseGrainedSchedulerBa

[GitHub] spark issue #15514: [SPARK-17960][PySpark] [Upgrade to Py4J 0.10.4]

2016-10-18 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15514 cc @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15285 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15285 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67134/ Test PASSed. ---

[GitHub] spark pull request #15522: [MINOR][DOC] Add more built-in sources in sql-pro...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15522 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15285 **[Test build #67134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67134/consoleFull)** for PR 15285 at commit [`4ead779`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15525: [SPARK-17985][CORE] Bump commons-lang3 version to...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15525 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15522: [MINOR][DOC] Add more built-in sources in sql-programmin...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15522 Merging in master/branch-2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15533: [SPARK-17974] (try 2) Refactor FileCatalog classe...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15533 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15285 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67135/ Test PASSed. ---

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15285 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15525: [SPARK-17985][CORE] Bump commons-lang3 version to 3.5.

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15525 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15533: [SPARK-17974] (try 2) Refactor FileCatalog classes to si...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15533 Merging in master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15285 **[Test build #67135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67135/consoleFull)** for PR 15285 at commit [`1e3302d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15534: [SPARK-17917][Scheduler] Fire SparkListenerTasksStarved ...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15534 The user can track this pretty easily themselves, can't they? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark issue #15533: [SPARK-17974] (try 2) Refactor FileCatalog classes to si...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15533 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67136/ Test PASSed. ---

[GitHub] spark issue #15533: [SPARK-17974] (try 2) Refactor FileCatalog classes to si...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15533 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15533: [SPARK-17974] (try 2) Refactor FileCatalog classes to si...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15533 **[Test build #67136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67136/consoleFull)** for PR 15533 at commit [`699bce5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15530 **[Test build #67139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67139/consoleFull)** for PR 15530 at commit [`0257b9e`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #15496: [SPARK-17950] [Python] Match SparseVector behavior with ...

2016-10-18 Thread itg-abby
Github user itg-abby commented on the issue: https://github.com/apache/spark/pull/15496 It is possible that I am missing something or that I have unintentionally obfuscated this pull request, I will try summarizing my understanding/purpose and see if it sheds any light: Dense

[GitHub] spark issue #13526: [SPARK-15780][SQL] Support mapValues on KeyValueGroupedD...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13526 cc @cloud-fan This looks good to me -- but I don't remember why we didn't merge it earlier already. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15285 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #14947: [SPARK-17388][SQL] Support for inferring type dat...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14947 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15522: [MINOR][DOC] Add more built-in sources in sql-pro...

2016-10-18 Thread weiqingy
Github user weiqingy commented on a diff in the pull request: https://github.com/apache/spark/pull/15522#discussion_r83941926 --- Diff: docs/sql-programming-guide.md --- @@ -422,8 +422,8 @@ In the simplest form, the default data source (`parquet` unless otherwise config You ca

[GitHub] spark issue #14947: [SPARK-17388][SQL] Support for inferring type date/times...

2016-10-18 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14947 Merging this into master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15285 LGTM. Merging to master and 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #15467: [SPARK-17912][SQL] Refactor code generation to get data ...

2016-10-18 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/15467 @ericl thank you very much --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15285 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15285 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67133/ Test PASSed. ---

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15285 **[Test build #67133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67133/consoleFull)** for PR 15285 at commit [`b3041b7`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15475: [SPARK-17991] [SQL] Enable metastore partition pruning b...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15475 **[Test build #67138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67138/consoleFull)** for PR 15475 at commit [`2581380`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #15475: [SPARK-17991] [SQL] Enable metastore partition pruning b...

2016-10-18 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15475 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15532: [SPARK-17989][SQL] Check ascendingOrder type in s...

2016-10-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15532#discussion_r83930227 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -124,7 +124,14 @@ case class SortArray(base:

[GitHub] spark issue #14947: [SPARK-17388][SQL] Support for inferring type date/times...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14947 **[Test build #67131 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67131/consoleFull)** for PR 14947 at commit [`bd5a63d`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15382: [SPARK-17810] [SQL] Default spark.sql.warehouse.d...

2016-10-18 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15382#discussion_r83928417 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -741,7 +741,7 @@ private[sql] class SQLConf extends Serializable with C

[GitHub] spark issue #14947: [SPARK-17388][SQL] Support for inferring type date/times...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14947 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67131/ Test PASSed. ---

[GitHub] spark issue #14947: [SPARK-17388][SQL] Support for inferring type date/times...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14947 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15532: [SPARK-17989][SQL] Check ascendingOrder type in s...

2016-10-18 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/15532#discussion_r83927813 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -124,7 +124,14 @@ case class SortArray

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r83927478 --- Diff: R/pkg/R/DataFrame.R --- @@ -1166,26 +1166,33 @@ setMethod("take", collect(limited) }) -#' Head -#'

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r83927201 --- Diff: R/pkg/R/DataFrame.R --- @@ -1166,26 +1166,33 @@ setMethod("take", collect(limited) }) -#' Head -#'

[GitHub] spark pull request #11336: [SPARK-9325][SPARK-R] collect() head() and show()...

2016-10-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11336#discussion_r83926993 --- Diff: R/pkg/R/DataFrame.R --- @@ -1166,26 +1166,33 @@ setMethod("take", collect(limited) }) -#' Head -#'

[GitHub] spark issue #11336: [SPARK-9325][SPARK-R] collect() head() and show() for Co...

2016-10-18 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/11336 could you update the JIRA and PR to reflect what's being added here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark issue #15534: [SPARK-17917][Scheduler] Fire SparkListenerTasksStarved ...

2016-10-18 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/15534 You are introducing 2 new events but they are not being consumed. Do you plan to use those anywhere ? Whats is use case that you are trying to solve ? --- If your project is set up for it, you

[GitHub] spark issue #15534: [SPARK-17917][Scheduler] Fire SparkListenerTasksStarved ...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15534 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #15450: [SPARK-3261] [MLLIB] KMeans clusterer can return duplica...

2016-10-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15450 Heh, I believe the PC term is 'corner cases'. I agree. There's not much point in clustering data to k centroids when there are <= k distinct points. I think that's all the more reasons to not make th

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15534: [SPARK-17917][Scheduler] Fire SparkListenerTasksS...

2016-10-18 Thread mariobriggs
GitHub user mariobriggs opened a pull request: https://github.com/apache/spark/pull/15534 [SPARK-17917][Scheduler] Fire SparkListenerTasksStarved event when submitted tasks are starved beyond the configured starvation timeout ## What changes were proposed in this pull request?

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #67132 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67132/consoleFull)** for PR 15009 at commit [`58c6bac`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67132/ Test FAILed. ---

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15285 **[Test build #67135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67135/consoleFull)** for PR 15285 at commit [`1e3302d`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #15382: [SPARK-17810] [SQL] Default spark.sql.warehouse.d...

2016-10-18 Thread koertkuipers
Github user koertkuipers commented on a diff in the pull request: https://github.com/apache/spark/pull/15382#discussion_r83921525 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -741,7 +741,7 @@ private[sql] class SQLConf extends Serializable wi

[GitHub] spark pull request #15516: [SPARK-17961][SparkR][SQL] Add storageLevel to Da...

2016-10-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15516#discussion_r83921132 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -795,6 +795,8 @@ test_that("cache(), persist(), and unpersist() on a DataFrame", { pers

[GitHub] spark issue #15521: [SPARK-17980] [SQL] Fix refreshByPath for converted Hive...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15521 **[Test build #67137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67137/consoleFull)** for PR 15521 at commit [`5088ae0`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #15516: [SPARK-17961][SparkR][SQL] Add storageLevel to Da...

2016-10-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15516#discussion_r83920746 --- Diff: R/pkg/R/DataFrame.R --- @@ -654,6 +654,33 @@ setMethod("unpersist", x }) +#' StorageLevel +#'

[GitHub] spark pull request #15521: [SPARK-17980] [SQL] Fix refreshByPath for convert...

2016-10-18 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15521#discussion_r83919797 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala --- @@ -49,13 +49,18 @@ class TableFileCatalog(

[GitHub] spark pull request #15521: [SPARK-17980] [SQL] Fix refreshByPath for convert...

2016-10-18 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15521#discussion_r83919804 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -185,9 +185,10 @@ class CacheManager extends Logging { plan

[GitHub] spark pull request #15521: [SPARK-17980] [SQL] Fix refreshByPath for convert...

2016-10-18 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15521#discussion_r83919788 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetadataCacheSuite.scala --- @@ -91,6 +95,19 @@ class HiveMetadataCacheSuite extends QueryTest

[GitHub] spark issue #15533: [SPARK-17974] (try 2) Refactor FileCatalog classes to si...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15533 Can you put the proper description here still? It becomes part of the commit msg. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark issue #15532: [SPARK-17989][SQL] Check ascendingOrder type in sort_arr...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15532 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67129/ Test PASSed. ---

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Follow up to remove Option.contai...

2016-10-18 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15531 @srowen Unfortunately the 'Some' in the == is usually missed out, resulting in bugs (and we have had a fair share of them in the past - some more severe than others). Given that there is no supp

[GitHub] spark issue #15533: [SPARK-17974] (try 2) Refactor FileCatalog classes to si...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15533 **[Test build #67136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67136/consoleFull)** for PR 15533 at commit [`699bce5`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #15532: [SPARK-17989][SQL] Check ascendingOrder type in sort_arr...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15532 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15533: [SPARK-17974] (try 2) Refactor FileCatalog classe...

2016-10-18 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/15533 [SPARK-17974] (try 2) Refactor FileCatalog classes to simplify the inheritance tree This is just https://github.com/apache/spark/pull/15518 which got reverted for a scalastyle bug. @rxin You can m

[GitHub] spark issue #15532: [SPARK-17989][SQL] Check ascendingOrder type in sort_arr...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15532 **[Test build #67129 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67129/consoleFull)** for PR 15532 at commit [`5f73245`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Follow up to remove Option.contai...

2016-10-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15531 Let the religious wars commence. If Option is a "collection of 0 or 1 things" then `.contains` at least makes some sense, if a bit obtuse. I still think `== Some(x)` is clearer anyway. --- If your

[GitHub] spark issue #15490: [SPARK-10541] [Web UI] Allow ApplicationHistoryProviders...

2016-10-18 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/15490 Oh, I see, new UI had meant I'd left the comment partially incomplete. Sorry. Just the one: printing out the actual log dir location. That makes it much easier to identify a configuration prob

[GitHub] spark pull request #15490: [SPARK-10541] [Web UI] Allow ApplicationHistoryPr...

2016-10-18 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/15490#discussion_r83530966 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -262,6 +263,17 @@ private[history] class FsHistoryProvider

[GitHub] spark pull request #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactore...

2016-10-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15530#discussion_r83915869 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryListener.scala --- @@ -41,24 +41,24 @@ abstract class StreamingQueryListener {

[GitHub] spark issue #15531: [SQL][STREAMING][TEST] Follow up to remove Option.contai...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15531 I disagree. It makes it very confusing, especially without explicit types, to know what exactly is happening. Not everything should be a monad. --- If your project is set up for it, you can reply to

[GitHub] spark pull request #15528: [SPARK-17899][SQL][follow-up] debug mode should w...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15528 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15285 **[Test build #67134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67134/consoleFull)** for PR 15285 at commit [`4ead779`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83914442 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1452,52 +1452,54 @@ private[spark] object Utils extends Logging { CallSite(short

[GitHub] spark issue #15528: [SPARK-17899][SQL][follow-up] debug mode should work for...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15528 Merging in master. Do we need this for branch-2.0 as well? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83914169 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1452,52 +1452,54 @@ private[spark] object Utils extends Logging { CallSite(short

[GitHub] spark issue #15529: [SPARK-17751] [SQL] [Backport-2.0] Remove spark.sql.eage...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15529 Can you close the ticket? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

<    1   2   3   4   5   6   >