[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14638 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69362/ Test PASSed. ---

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14638 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #16071: [SPARK-18635] [SQL] Partition name/values not esc...

2016-11-29 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16071#discussion_r90155215 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/PartitionProviderCompatibilitySuite.scala --- @@ -205,6 +205,58 @@ class PartitionProviderCompatib

[GitHub] spark pull request #16071: [SPARK-18635] [SQL] Partition name/values not esc...

2016-11-29 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16071#discussion_r90155112 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/PartitionProviderCompatibilitySuite.scala --- @@ -205,6 +205,58 @@ class PartitionProviderCompatib

[GitHub] spark issue #16071: [SPARK-18635] [SQL] Partition name/values not escaped co...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16071 **[Test build #69370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69370/consoleFull)** for PR 16071 at commit [`4162229`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #16057: [SPARK-18624][SQL] Implicit cast complex types

2016-11-29 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16057 If this is going to support ArrayType only, can you update the pr title to reflect it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KMeans of...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13557 **[Test build #69368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69368/consoleFull)** for PR 13557 at commit [`032bb9d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KMeans of...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13557 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16069: [WIP][SPARK-18638][BUILD] Upgrade sbt to 0.13.13

2016-11-29 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/16069 Let's update Zinc here. I wasn't aware that the 0.13.13 version Zinc was available (its release cycle is slightly behind SBT's). --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KMeans of...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13557 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69368/ Test PASSed. ---

[GitHub] spark pull request #16072: [SPARK-18639] Build only a single pip package

2016-11-29 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/16072 [SPARK-18639] Build only a single pip package ## What changes were proposed in this pull request? We current build 5 separate pip binary tar balls, doubling the release script runtime. It'd be bet

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-29 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16052 OK --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] spark pull request #16073: [SPARK-18640] Add synchronization to TaskSchedule...

2016-11-29 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/16073 [SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors ## What changes were proposed in this pull request? The method `TaskSchedulerImpl.runningTasksByExecutors()` acc

[GitHub] spark issue #16072: [SPARK-18639] Build only a single pip package

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16072 **[Test build #69371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69371/consoleFull)** for PR 16072 at commit [`88b53c3`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #16057: [SPARK-18624][SQL] Implicit cast complex types

2016-11-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16057#discussion_r90157966 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -714,6 +714,17 @@ object TypeCoercion { /

[GitHub] spark pull request #16072: [SPARK-18639] Build only a single pip package

2016-11-29 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/16072#discussion_r90158161 --- Diff: dev/create-release/release-build.sh --- @@ -150,6 +150,7 @@ if [[ "$1" == "package" ]]; then NAME=$1 FLAGS=$2 ZINC_PORT

[GitHub] spark issue #16073: [SPARK-18640] Add synchronization to TaskScheduler.runni...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16073 **[Test build #69372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69372/consoleFull)** for PR 16073 at commit [`c866a1a`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #16063: [SPARK-18622][SQL] Fix the datatype of the Sum aggregate...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16063 **[Test build #69363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69363/consoleFull)** for PR 16063 at commit [`027c31f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16063: [SPARK-18622][SQL] Fix the datatype of the Sum aggregate...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16063 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69363/ Test FAILed. ---

[GitHub] spark issue #16063: [SPARK-18622][SQL] Fix the datatype of the Sum aggregate...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16063 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15877: [SPARK-18429] [SQL] implement a new Aggregate for...

2016-11-29 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/15877#discussion_r90158620 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CountMinSketchAggSuite.scala --- @@ -0,0 +1,320 @@ +/* + * L

[GitHub] spark pull request #16074: [SPARK-18145] Update documentation for hive parti...

2016-11-29 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/16074 [SPARK-18145] Update documentation for hive partition management in 2.1 ## What changes were proposed in this pull request? This documents the partition handling changes for Spark 2.1 and how

[GitHub] spark issue #16074: [SPARK-18145] Update documentation for hive partition ma...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16074 **[Test build #69373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69373/consoleFull)** for PR 16074 at commit [`49eff17`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15982: [SPARK-18546][core] Fix merging shuffle spills when usin...

2016-11-29 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15982 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15982: [SPARK-18546][core] Fix merging shuffle spills when usin...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15982 **[Test build #69375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69375/consoleFull)** for PR 15982 at commit [`1025c6b`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #16074: [SPARK-18145] Update documentation for hive partition ma...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16074 **[Test build #69374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69374/consoleFull)** for PR 16074 at commit [`9ee308a`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #16069: [WIP][SPARK-18638][BUILD] Upgrade sbt to 0.13.13

2016-11-29 Thread weiqingy
Github user weiqingy commented on the issue: https://github.com/apache/spark/pull/16069 @JoshRosen OK. I will update Zinc to 0.3.11, the latest stable version (The version of Zinc downloaded from "`Download the latest stable version`" in [Zinc github](https://github.com/typesafehub/zi

[GitHub] spark issue #15972: [SPARK-18319][ML][QA2.1] 2.1 QA: API: Experimental, Deve...

2016-11-29 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15972 LGTM merging with master and branch-2.1 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request #15972: [SPARK-18319][ML][QA2.1] 2.1 QA: API: Experimenta...

2016-11-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15972 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #16074: [SPARK-18145] Update documentation for hive partition ma...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16074 **[Test build #69373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69373/consoleFull)** for PR 16074 at commit [`49eff17`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16074: [SPARK-18145] Update documentation for hive partition ma...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16074 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69373/ Test PASSed. ---

[GitHub] spark issue #16074: [SPARK-18145] Update documentation for hive partition ma...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16074 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16074: [SPARK-18145] Update documentation for hive partition ma...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16074 **[Test build #69374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69374/consoleFull)** for PR 16074 at commit [`9ee308a`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16074: [SPARK-18145] Update documentation for hive partition ma...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16074 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69374/ Test PASSed. ---

[GitHub] spark issue #16074: [SPARK-18145] Update documentation for hive partition ma...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16074 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16063: [SPARK-18622][SQL] Fix the datatype of the Sum aggregate...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16063 **[Test build #69364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69364/consoleFull)** for PR 16063 at commit [`0d54e88`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16063: [SPARK-18622][SQL] Fix the datatype of the Sum aggregate...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16063 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16063: [SPARK-18622][SQL] Fix the datatype of the Sum aggregate...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16063 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69364/ Test PASSed. ---

[GitHub] spark issue #16069: [WIP][SPARK-18638][BUILD] Upgrade sbt to 0.13.13

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16069 **[Test build #69365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69365/consoleFull)** for PR 16069 at commit [`7ae86b7`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16069: [WIP][SPARK-18638][BUILD] Upgrade sbt to 0.13.13

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69365/ Test PASSed. ---

[GitHub] spark pull request #16075: [SPARK-18516][Structured Streaming] Follow up PR ...

2016-11-29 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/16075 [SPARK-18516][Structured Streaming] Follow up PR to add StreamingQuery.status to Python ## What changes were proposed in this pull request? - Add StreamingQueryStatus.json - Make it not case

[GitHub] spark issue #16069: [WIP][SPARK-18638][BUILD] Upgrade sbt to 0.13.13

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16069 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #16075: [SPARK-18516][Structured Streaming] Follow up PR ...

2016-11-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16075#discussion_r90163483 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryListenerSuite.scala --- @@ -159,28 +164,6 @@ class StreamingQueryListenerSuite e

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16066 **[Test build #69367 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69367/consoleFull)** for PR 16066 at commit [`5f8e3d0`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16075: [SPARK-18516][Structured Streaming] Follow up PR to add ...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16075 **[Test build #69376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69376/consoleFull)** for PR 16075 at commit [`f936595`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16066 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16066 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69367/ Test PASSed. ---

[GitHub] spark issue #15821: [SPARK-13534][WIP][PySpark] Using Apache Arrow to increa...

2016-11-29 Thread mariusvniekerk
Github user mariusvniekerk commented on the issue: https://github.com/apache/spark/pull/15821 So this is very cool stuff. Would it be reasonable to add some api pieces so that on the python side things like DataFrame.mapPartitions makes use of Apache Arrow to lower the seri

[GitHub] spark issue #15821: [SPARK-13534][WIP][PySpark] Using Apache Arrow to increa...

2016-11-29 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15821 @mariusvniekerk I think just getting this working for local connection is going to be hard so breaking up using arrow on the driver side into a separate follow up piece of work would make sense. -

[GitHub] spark issue #16061: [SPARK-18278] [Scheduler] Support native submission of s...

2016-11-29 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/16061 Another external scheduler backend I'm aware of is Two Sigma's scheduler backend for the system they've created called [Cook](https://github.com/twosigma/Cook). See [CoarseCookSchedulerBackend.scal

[GitHub] spark pull request #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint In...

2016-11-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16067#discussion_r90164208 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/ConstraintPropagationSuite.scala --- @@ -351,6 +351,15 @@ class ConstraintPropa

[GitHub] spark issue #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint Inference...

2016-11-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16067 Also cc @cloud-fan @hvanhovell @sameeragarwal @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request #15998: [SPARK-18572][SQL] Add a method `listPartitionNam...

2016-11-29 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/15998#discussion_r90164420 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalog.scala --- @@ -189,11 +189,28 @@ abstract class ExternalCatalog {

[GitHub] spark pull request #15998: [SPARK-18572][SQL] Add a method `listPartitionNam...

2016-11-29 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/15998#discussion_r90164431 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -730,6 +730,23 @@ class SessionCatalog( }

[GitHub] spark issue #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint Inference...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16067 **[Test build #69377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69377/consoleFull)** for PR 16067 at commit [`a835f80`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15998 **[Test build #69378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69378/consoleFull)** for PR 15998 at commit [`48ae2a3`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #15998: [SPARK-18572][SQL] Add a method `listPartitionNam...

2016-11-29 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/15998#discussion_r90164499 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala --- @@ -482,6 +483,19 @@ class InMemoryCatalog( }

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-11-29 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15998 I will work on additional unit test coverage tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark issue #15255: [SPARK-17680] [SQL] [TEST] Added a Testcase for Verifyin...

2016-11-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15255 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #15255: [SPARK-17680] [SQL] [TEST] Added a Testcase for Verifyin...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15255 **[Test build #69379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69379/consoleFull)** for PR 15255 at commit [`57817a1`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16066 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16074: [SPARK-18145] Update documentation for hive partition ma...

2016-11-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16074 LGTM merging in master/branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request #16066: [SPARK-18632][SQL] AggregateFunction should not i...

2016-11-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16066 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-29 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r90165717 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSinkSuite.scala --- @@ -86,7 +86,7 @@ class FileStreamSinkSuite extends StreamTest

[GitHub] spark pull request #16074: [SPARK-18145] Update documentation for hive parti...

2016-11-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16074 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operators allow...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16046 **[Test build #69369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69369/consoleFull)** for PR 16046 at commit [`1d32958`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operators allow...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16046 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69369/ Test PASSed. ---

[GitHub] spark issue #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operators allow...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16046 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16071: [SPARK-18635] [SQL] Partition name/values not escaped co...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16071 **[Test build #69370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69370/consoleFull)** for PR 16071 at commit [`4162229`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16071: [SPARK-18635] [SQL] Partition name/values not escaped co...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16071 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16071: [SPARK-18635] [SQL] Partition name/values not escaped co...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16071 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69370/ Test PASSed. ---

[GitHub] spark pull request #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint In...

2016-11-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16067#discussion_r90167277 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -58,13 +57,28 @@ abstract class QueryPlan[PlanType <: Query

[GitHub] spark issue #16072: [SPARK-18639] Build only a single pip package

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16072 **[Test build #69371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69371/consoleFull)** for PR 16072 at commit [`88b53c3`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16072: [SPARK-18639] Build only a single pip package

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16072 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16072: [SPARK-18639] Build only a single pip package

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16072 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69371/ Test FAILed. ---

[GitHub] spark pull request #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss ...

2016-11-29 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/15994#discussion_r90167716 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -437,4 +438,38 @@ final class DataFrameNaFunctions private[sql](df

[GitHub] spark issue #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KMeans of...

2016-11-29 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/13557 LGTM, merged into master and branch-2.1. Thank you all. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request #16061: [SPARK-18278] [Scheduler] Support native submissi...

2016-11-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16061#discussion_r90168566 --- Diff: kubernetes/README.md --- @@ -0,0 +1,21 @@ +# Pre-requisites +* maven, JDK and all other pre-requisites for building Spark. + +# Steps

[GitHub] spark pull request #16061: [SPARK-18278] [Scheduler] Support native submissi...

2016-11-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16061#discussion_r90168600 --- Diff: kubernetes/README.md --- @@ -0,0 +1,21 @@ +# Pre-requisites +* maven, JDK and all other pre-requisites for building Spark. + +# Steps

[GitHub] spark pull request #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KM...

2016-11-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13557 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint In...

2016-11-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16067#discussion_r90168753 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -58,13 +57,28 @@ abstract class QueryPlan[PlanType <: Q

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-11-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/15780 ping @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16052 **[Test build #69380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69380/consoleFull)** for PR 16052 at commit [`39b4867`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #16073: [SPARK-18640] Add synchronization to TaskScheduler.runni...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16073 **[Test build #69372 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69372/consoleFull)** for PR 16073 at commit [`c866a1a`](https://github.com/apache/spark/commit/

[GitHub] spark issue #16073: [SPARK-18640] Add synchronization to TaskScheduler.runni...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16073 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69372/ Test PASSed. ---

[GitHub] spark issue #16073: [SPARK-18640] Add synchronization to TaskScheduler.runni...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16073 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16009: [SPARK-18318][ML] ML, Graph 2.1 QA: API: New Scala APIs,...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16009 **[Test build #69381 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69381/consoleFull)** for PR 16009 at commit [`27b07ef`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint In...

2016-11-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16067#discussion_r90169798 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -58,13 +57,28 @@ abstract class QueryPlan[PlanType <: Query

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-29 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16014 Any more thought on this? Without this we don't really have a signed tarball in the official release to release to CRAN... --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #16009: [SPARK-18318][ML] ML, Graph 2.1 QA: API: New Scal...

2016-11-29 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16009#discussion_r90169879 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -49,15 +49,13 @@ private[feature] trait ChiSqSelectorParams extends

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-29 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16014 Sorry I was caught up with some other stuff today. Will take a final look tomm morning. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark issue #15982: [SPARK-18546][core] Fix merging shuffle spills when usin...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15982 **[Test build #69375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69375/consoleFull)** for PR 15982 at commit [`1025c6b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15982: [SPARK-18546][core] Fix merging shuffle spills when usin...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15982 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15982: [SPARK-18546][core] Fix merging shuffle spills when usin...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15982 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69375/ Test PASSed. ---

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15915 **[Test build #69382 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69382/consoleFull)** for PR 15915 at commit [`ad85da0`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #15987: [SPARK-17732][SPARK-18515][SQL] ALTER TABLE DROP PARTITI...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15987 **[Test build #69383 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69383/consoleFull)** for PR 15987 at commit [`9db07af`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-29 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r90172243 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -590,7 +591,11 @@ object ScalaReflection extends ScalaReflec

[GitHub] spark issue #16072: [SPARK-18639] Build only a single pip package

2016-11-29 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16072 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16072: [SPARK-18639] Build only a single pip package

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16072 **[Test build #69384 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69384/consoleFull)** for PR 16072 at commit [`88b53c3`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #16075: [SPARK-18516][Structured Streaming] Follow up PR to add ...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16075 **[Test build #69376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69376/consoleFull)** for PR 16075 at commit [`f936595`](https://github.com/apache/spark/commit/

<    1   2   3   4   5   6   7   >