[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84590664 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84590948 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84591160 --- Diff: docs/configuration.md --- @@ -1342,6 +1342,20 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84591185 --- Diff: docs/configuration.md --- @@ -1342,6 +1342,20 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84591178 --- Diff: docs/configuration.md --- @@ -1342,6 +1342,20 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84591169 --- Diff: docs/configuration.md --- @@ -1342,6 +1342,20 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-10-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14136 I think we can merge this one first, since it is no worse than the Hive one. Then we can think about how to make it more robust. cc @hvanhovell --- If your project is set up for it, you can

[GitHub] spark issue #15575: [SPARK-18038] [SQL] Move output partitioning definition ...

2016-10-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15575 LGTM too. Unforunately my internet sucks (on a plane) and I can't merge this right now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15354 @marmbrus Sure (I didn't mean I am not going to do this..), I just handled the case in this PR for `to_json`. BTW, I would like to note that there are the same problems in other JSON

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #67407 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67407/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67407/ Test FAILed. ---

[GitHub] spark pull request #15601: [SPARK-17123][SQL][BRANCH-2.0] Use type-widened e...

2016-10-23 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/15601 [SPARK-17123][SQL][BRANCH-2.0] Use type-widened encoder for DataFrame for set operations ## What changes were proposed in this pull request? This PR backports

[GitHub] spark issue #15601: [SPARK-17123][SQL][BRANCH-2.0] Use type-widened encoder ...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15601 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67413/ Test PASSed. ---

[GitHub] spark pull request #15602: [SPARK-18058][SQL] [BACKPORT-2.0]Comparing column...

2016-10-23 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/spark/pull/15602 [SPARK-18058][SQL] [BACKPORT-2.0]Comparing column types ignoring Nullability in Union and SetOperation ## What changes were proposed in this pull request? The PR tries to fix

[GitHub] spark pull request #15595: [SPARK-18058][SQL] Comparing column types ignorin...

2016-10-23 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/15595#discussion_r84600741 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala --- @@ -377,4 +377,23 @@ class AnalysisSuite extends

[GitHub] spark issue #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15603 **[Test build #67420 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67420/consoleFull)** for PR 15603 at commit

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-23 Thread jaceklaskowski
GitHub user jaceklaskowski opened a pull request: https://github.com/apache/spark/pull/15603 [WEBUI][MINOR] Return types in methods + cleanup ## What changes were proposed in this pull request? The main purpose of the change is to discuss the purpose of `SparkUITab` class

[GitHub] spark issue #15602: [SPARK-18058][SQL] [BRANCH-2.0]Comparing column types ig...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15602 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15602: [SPARK-18058][SQL] [BRANCH-2.0]Comparing column types ig...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67419/ Test PASSed. ---

[GitHub] spark issue #13252: [SPARK-15473][SQL] CSV data source writes header for emp...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13252 **[Test build #67421 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67421/consoleFull)** for PR 13252 at commit

[GitHub] spark issue #15484: [SPARK-17868][SQL] Do not use bitmasks during parsing an...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15484 **[Test build #67405 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67405/consoleFull)** for PR 15484 at commit

[GitHub] spark issue #15484: [SPARK-17868][SQL] Do not use bitmasks during parsing an...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67405/ Test PASSed. ---

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84590702 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #15484: [SPARK-17868][SQL] Do not use bitmasks during parsing an...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15484 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

2016-10-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15513#discussion_r84590685 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala --- @@ -125,7 +129,7 @@ case class DescribeFunctionCommand(

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84591073 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -250,24 +251,26 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #67407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67407/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #67408 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67408/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67408/ Test PASSed. ---

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15513 **[Test build #67411 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67411/consoleFull)** for PR 15513 at commit

[GitHub] spark pull request #15595: [SPARK-18058][SQL] Comparing column types ignorin...

2016-10-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15595 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-23 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/15595 sure, doing that now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15595 @CodingCat could you open a backport for branch-2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15595 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15589: [SPARKR][branch-2.0] R merge API doc and example ...

2016-10-23 Thread felixcheung
Github user felixcheung closed the pull request at: https://github.com/apache/spark/pull/15589 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #15589: [SPARKR][branch-2.0] R merge API doc and example fix

2016-10-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15589 merged to branch-2.0 and then cherry-picked to master for the error message fix not in master. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15602: [SPARK-18058][SQL] [BRANCH-2.0]Comparing column types ig...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15602 LGTM - pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-23 Thread yinxusen
Github user yinxusen commented on the issue: https://github.com/apache/spark/pull/9 How about the following: 1. Since the new generated model is derived from an estimator, the model should have the same params as its parent estimator. That's why there is no need to

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15541 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67418/ Test PASSed. ---

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15541 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15541 **[Test build #67418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67418/consoleFull)** for PR 15541 at commit

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14136 It is relatively straight forward to test `ImperativeAggregate`s. See: - org.apache.spark.sql.catalyst.expressions.aggregate.HyperLogLogPlusPlusSuite -

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-23 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/15595 Just saw that this got merged. I had a tiny nit but its ok without it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15595: [SPARK-18058][SQL] Comparing column types ignorin...

2016-10-23 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/15595#discussion_r84600424 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala --- @@ -377,4 +377,23 @@ class AnalysisSuite extends

[GitHub] spark issue #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15596 @JoshRosen could you shed some light on why we are not using the regular `EnsureRequirements` based code path for `CollectLimitExec`? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in the UI

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15441 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67417/ Test PASSed. ---

[GitHub] spark issue #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in the UI

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15441 **[Test build #67417 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67417/consoleFull)** for PR 15441 at commit

[GitHub] spark issue #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in the UI

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15441 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in...

2016-10-23 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/15441#discussion_r84599733 --- Diff: core/src/test/scala/org/apache/spark/ui/UISeleniumSuite.scala --- @@ -651,6 +671,15 @@ class UISeleniumSuite extends SparkFunSuite with

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15595 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in the UI

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15441 **[Test build #67417 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67417/consoleFull)** for PR 15441 at commit

[GitHub] spark issue #15541: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15541 **[Test build #67418 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67418/consoleFull)** for PR 15541 at commit

[GitHub] spark issue #15602: [SPARK-18058][SQL] [BRANCH-2.0]Comparing column types ig...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15602 **[Test build #67419 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67419/consoleFull)** for PR 15602 at commit

[GitHub] spark issue #15602: [SPARK-18058][SQL] [BRANCH-2.0]Comparing column types ig...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15602 **[Test build #67419 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67419/consoleFull)** for PR 15602 at commit

[GitHub] spark issue #15602: [SPARK-18058][SQL] [BRANCH-2.0]Comparing column types ig...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15602 Merging to branch-2.0. Thanks! Can you close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #15602: [SPARK-18058][SQL] [BRANCH-2.0]Comparing column t...

2016-10-23 Thread CodingCat
Github user CodingCat closed the pull request at: https://github.com/apache/spark/pull/15602 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-23 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15553 @srowen This is explicitly supports maven.test.skip=true - the rationale for using it can be project specific (generating clean builds for various profile combinations, internal release builds from

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84591137 --- Diff: docs/configuration.md --- @@ -1342,6 +1342,20 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #14957: [SPARK-4502][SQL]Support parquet nested struct pruning a...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14957 **[Test build #67410 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67410/consoleFull)** for PR 14957 at commit

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-23 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84592821 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +136,52 @@ object

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-23 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84592818 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +136,52 @@ object

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-23 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84592816 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +136,52 @@ object

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84590773 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-23 Thread jodersky
Github user jodersky commented on the issue: https://github.com/apache/spark/pull/15513 @HyukjinKwon (posting here to sum up the inline discussion) I looked through the way the final description string is created before printing, and it looks like the formatting will actually play

[GitHub] spark issue #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in the UI

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15441 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in the UI

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15441 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67406/ Test PASSed. ---

[GitHub] spark issue #15582: [SPARK-18045][SQL][TESTS] Move `HiveDataFrameAnalyticsSu...

2016-10-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15582 (Actually I'm on an airplane and the internet is not good enough to merge patches). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15582: [SPARK-18045][SQL][TESTS] Move `HiveDataFrameAnalyticsSu...

2016-10-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15582 cc @cloud-fan can you merge this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #67408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67408/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #67409 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67409/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15599: [SPARK-18022][SQL] java.lang.NullPointerException instea...

2016-10-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15599 I think we don't know exactly what the real exception is. The NPE occurs while handling the exception. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #15413: [SPARK-17847][ML][WIP] Copy GaussianMixture implementati...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15413 **[Test build #67414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67414/consoleFull)** for PR 15413 at commit

[GitHub] spark issue #15413: [SPARK-17847][ML][WIP] Copy GaussianMixture implementati...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15413 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67414/ Test PASSed. ---

[GitHub] spark issue #15413: [SPARK-17847][ML][WIP] Copy GaussianMixture implementati...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15413 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15601: [SPARK-17123][SQL][BRANCH-2.0] Use type-widened encoder ...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15601 **[Test build #67413 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67413/consoleFull)** for PR 15601 at commit

[GitHub] spark issue #15601: [SPARK-17123][SQL][BRANCH-2.0] Use type-widened encoder ...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15601 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15575: [SPARK-18038] [SQL] Move output partitioning definition ...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15575 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15575: [SPARK-18038] [SQL] Move output partitioning defi...

2016-10-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15575 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #67409 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67409/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15582: [SPARK-18045][SQL][TESTS] Move `HiveDataFrameAnalyticsSu...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15582 Merging to master. Thanks! Ping me for the follow-up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14957: [SPARK-4502][SQL]Support parquet nested struct pruning a...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14957 **[Test build #67410 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67410/consoleFull)** for PR 14957 at commit

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15553 For development time, use an IDE or maybe SBT to do incremental compilation, which is even faster. I don't otherwise see a strong use case for not compiling tests. Yes it's slower to compile tests,

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67409/ Test PASSed. ---

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14957: [SPARK-4502][SQL]Support parquet nested struct pruning a...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14957 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67410/ Test PASSed. ---

[GitHub] spark issue #14957: [SPARK-4502][SQL]Support parquet nested struct pruning a...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14957 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15582: [SPARK-18045][SQL][TESTS] Move `HiveDataFrameAnal...

2016-10-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15582 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15601: [SPARK-17123][SQL][BRANCH-2.0] Use type-widened encoder ...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15601 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84590747 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15541: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15541#discussion_r84591093 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -305,12 +309,8 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

2016-10-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15513#discussion_r84591475 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1455,50 +1455,59 @@ class DDLSuite extends QueryTest

[GitHub] spark pull request #15513: [SPARK-17963][SQL][Documentation] Add examples (e...

2016-10-23 Thread jodersky
Github user jodersky commented on a diff in the pull request: https://github.com/apache/spark/pull/15513#discussion_r84591482 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala --- @@ -125,7 +129,7 @@ case class DescribeFunctionCommand(

[GitHub] spark issue #15575: [SPARK-18038] [SQL] Move output partitioning definition ...

2016-10-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15575 yeah, LGTM, it doesn't change current outputPartitioning of operators. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-10-23 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r84591459 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala --- @@ -136,7 +136,7 @@ class HiveUDFSuite extends QueryTest with

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-10-23 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r84591451 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/AggregationQuerySuite.scala --- @@ -851,6 +851,42 @@ abstract class

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-10-23 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r84591458 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/AggregationQuerySuite.scala --- @@ -851,6 +851,42 @@ abstract class

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15601: [SPARK-17123][SQL][BRANCH-2.0] Use type-widened encoder ...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15601 **[Test build #67412 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67412/consoleFull)** for PR 15601 at commit

[GitHub] spark issue #15601: [SPARK-17123][SQL][BRANCH-2.0] Use type-widened encoder ...

2016-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15601 **[Test build #67413 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67413/consoleFull)** for PR 15601 at commit

  1   2   3   >