[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3093/consoleFull)** for PR 13498 at commit

[GitHub] spark issue #13492: [SPARK-15749][SQL]make the error message more meaningful

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13492 Looks good. retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13546: [SPARK-15808] [SQL] File Format Checking When Appending ...

2016-06-13 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/13546 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #13546: [SPARK-15808] [SQL] File Format Checking When Appending ...

2016-06-13 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/13546 LGTM pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #13558: [SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13558 **[Test build #60446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60446/consoleFull)** for PR 13558 at commit

[GitHub] spark pull request #13637: [SPARK-15914][SQL] Add deprecated method back to ...

2016-06-13 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/13637#discussion_r66886314 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -736,6 +736,290 @@ class SQLContext private[sql](val sparkSession:

[GitHub] spark issue #13637: [SPARK-15914][SQL] Add deprecated method back to SQLCont...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13637 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60433/ Test PASSed. ---

[GitHub] spark issue #13637: [SPARK-15914][SQL] Add deprecated method back to SQLCont...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13637 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13115: [SPARK-12492] Using spark-sql commond to run quer...

2016-06-13 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/13115#discussion_r66886230 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala --- @@ -110,24 +110,29 @@ class QueryExecution(val sparkSession:

[GitHub] spark issue #13648: [SQL][DOC][minor] document the contract of encoder seria...

2016-06-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13648 JIRA is down, will create a ticket later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13637: [SPARK-15914][SQL] Add deprecated method back to SQLCont...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13637 **[Test build #60433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60433/consoleFull)** for PR 13637 at commit

[GitHub] spark pull request #13592: [SPARK-15863][SQL][DOC] Initial SQL programming g...

2016-06-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13592#discussion_r66886012 --- Diff: docs/sql-programming-guide.md --- @@ -587,7 +590,7 @@ for the JavaBean. {% highlight java %} // sc is an existing

[GitHub] spark issue #13558: [SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13558 Looks good. Jenks retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13646: [SPARK-15927] Eliminate redundant DAGScheduler code.

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13646 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13646: [SPARK-15927] Eliminate redundant DAGScheduler code.

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13646 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60432/ Test PASSed. ---

[GitHub] spark pull request #13637: [SPARK-15914][SQL] Add deprecated method back to ...

2016-06-13 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/13637#discussion_r66885663 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -736,6 +736,290 @@ class SQLContext private[sql](val sparkSession:

[GitHub] spark issue #13646: [SPARK-15927] Eliminate redundant DAGScheduler code.

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13646 **[Test build #60432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60432/consoleFull)** for PR 13646 at commit

[GitHub] spark pull request #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13620#discussion_r66885556 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -369,3 +361,246 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark pull request #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13620#discussion_r66885396 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -369,3 +361,246 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark issue #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs tab

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13620 @nblintao this looks pretty good. There were a few methods that don't seem to be used anywhere. Am I missing something? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #13651: [SPARK-15776][SQL] Divide Expression inside Aggre...

2016-06-13 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/13651#discussion_r66885516 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -213,7 +213,7 @@ case class Multiply(left:

[GitHub] spark pull request #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13620#discussion_r66885448 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -369,3 +361,246 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark pull request #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13620#discussion_r66885512 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -369,3 +361,246 @@ private[ui] class AllJobsPage(parent: JobsTab) extends

[GitHub] spark issue #13561: [SPARK-15824][SQL] Run 'with ... insert ... select' fail...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13561 **[Test build #60445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60445/consoleFull)** for PR 13561 at commit

[GitHub] spark issue #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs tab

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13620 **[Test build #60444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60444/consoleFull)** for PR 13620 at commit

[GitHub] spark issue #13620: [SPARK-15590] [WEBUI] Paginate Job Table in Jobs tab

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13620 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #13561: [SPARK-15824][SQL] Run 'with ... insert ... select' fail...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13561 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13632: [SPARK-15910][SQL] Check schema consistency when using K...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13632 **[Test build #60443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60443/consoleFull)** for PR 13632 at commit

[GitHub] spark pull request #13638: [SPARK-15915][SQL] CacheManager should use canoni...

2016-06-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/13638#discussion_r66884468 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -155,8 +156,9 @@ private[sql] class CacheManager extends Logging {

[GitHub] spark issue #13539: [SPARK-15795] [SQL] Enable more optimizations in whole s...

2016-06-13 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/13539 As @cloud-fan said, the implementation is hacky, the improvements is not not obvious (I believe JIT compile do these very well, correct me if I'm wrong), I'd like not do this. There are millions

[GitHub] spark issue #13637: [SPARK-15914][SQL] Add deprecated method back to SQLCont...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13637 This looks good, just minor commnts --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13623: [SPARK-15895][SQL] Filters out metadata files whi...

2016-06-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13623#discussion_r66884268 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala --- @@ -890,4 +892,26 @@ class

[GitHub] spark pull request #13623: [SPARK-15895][SQL] Filters out metadata files whi...

2016-06-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13623#discussion_r66884199 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala --- @@ -890,4 +892,26 @@ class

[GitHub] spark pull request #13592: [SPARK-15863][SQL][DOC] Initial SQL programming g...

2016-06-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13592#discussion_r66884213 --- Diff: docs/sql-programming-guide.md --- @@ -12,130 +12,130 @@ title: Spark SQL and DataFrames Spark SQL is a Spark module for structured data

[GitHub] spark pull request #13637: [SPARK-15914][SQL] Add deprecated method back to ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13637#discussion_r66884192 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -736,6 +736,290 @@ class SQLContext private[sql](val sparkSession:

[GitHub] spark pull request #13592: [SPARK-15863][SQL][DOC] Initial SQL programming g...

2016-06-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13592#discussion_r66884198 --- Diff: docs/sql-programming-guide.md --- @@ -12,130 +12,130 @@ title: Spark SQL and DataFrames Spark SQL is a Spark module for structured data

[GitHub] spark pull request #13623: [SPARK-15895][SQL] Filters out metadata files whi...

2016-06-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13623#discussion_r66884135 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileCatalog.scala --- @@ -197,4 +201,9 @@ abstract class

[GitHub] spark pull request #13623: [SPARK-15895][SQL] Filters out metadata files whi...

2016-06-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13623#discussion_r66884097 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileCatalog.scala --- @@ -96,7 +96,11 @@ abstract class

[GitHub] spark pull request #13638: [SPARK-15915][SQL] CacheManager should use canoni...

2016-06-13 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/13638#discussion_r66884087 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -155,8 +156,9 @@ private[sql] class CacheManager extends Logging

[GitHub] spark pull request #13592: [SPARK-15863][SQL][DOC] Initial SQL programming g...

2016-06-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13592#discussion_r66884076 --- Diff: docs/sql-programming-guide.md --- @@ -517,24 +517,26 @@ types such as Sequences or Arrays. This RDD can be implicitly converted to a Dat

[GitHub] spark pull request #13637: [SPARK-15914][SQL] Add deprecated method back to ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13637#discussion_r66884075 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -736,6 +736,290 @@ class SQLContext private[sql](val sparkSession:

[GitHub] spark pull request #13652: [SPARK-] Fix incorrect days to millis conversion

2016-06-13 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13652#discussion_r66883860 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala --- @@ -503,5 +510,25 @@ class DateTimeUtilsSuite extends

[GitHub] spark pull request #13637: [SPARK-15914][SQL] Add deprecated method back to ...

2016-06-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/13637#discussion_r66884061 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -736,6 +736,290 @@ class SQLContext private[sql](val sparkSession:

[GitHub] spark pull request #13623: [SPARK-15895][SQL] Filters out metadata files whi...

2016-06-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13623#discussion_r66883639 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala --- @@ -83,8 +83,9 @@ class ListingFileCatalog(

[GitHub] spark issue #13636: [SPARK-15637][SPARKR] Remove R version check since maske...

2016-06-13 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13636 @liancheng we can use this PR to also address https://issues.apache.org/jira/browse/SPARK-15931 --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #12938: [SPARK-15162][SPARK-15164][PySpark][DOCS][ML] update som...

2016-06-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12938 @jkbradley - I don't think that is the issue but it does seem like the only "real" change - strangely enough I get the same failure locally on vanilla master at

[GitHub] spark issue #13648: [SQL][DOC][minor] document the contract of encoder seria...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60431/ Test PASSed. ---

[GitHub] spark issue #13648: [SQL][DOC][minor] document the contract of encoder seria...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13648 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13651: [SPARK-15776][SQL] Divide Expression inside Aggre...

2016-06-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13651#discussion_r66883329 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -525,7 +525,7 @@ object TypeCoercion {

[GitHub] spark issue #13648: [SQL][DOC][minor] document the contract of encoder seria...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13648 **[Test build #60431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60431/consoleFull)** for PR 13648 at commit

[GitHub] spark pull request #13638: [SPARK-15915][SQL] CacheManager should use canoni...

2016-06-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/13638#discussion_r66883158 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -155,8 +156,9 @@ private[sql] class CacheManager extends Logging {

[GitHub] spark pull request #13652: [SPARK-] Fix incorrect days to millis conversion

2016-06-13 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13652#discussion_r66883008 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DateType.scala --- @@ -30,7 +30,7 @@ import

[GitHub] spark pull request #13644: [SPARK-15925][SQL][SPARKR] Replaces registerTempT...

2016-06-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13644 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #13632: [SPARK-15910][SQL] Check schema consistency when ...

2016-06-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13632#discussion_r66882860 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/Encoders.scala --- @@ -209,7 +209,12 @@ object Encoders { BoundReference(0,

[GitHub] spark pull request #13652: [SPARK-] Fix incorrect days to millis conversion

2016-06-13 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13652#discussion_r66882931 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala --- @@ -503,5 +510,25 @@ class DateTimeUtilsSuite

[GitHub] spark issue #13652: [SPARK-] Fix incorrect days to millis conversion

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13652 **[Test build #60441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60441/consoleFull)** for PR 13652 at commit

[GitHub] spark issue #13632: [SPARK-15910][SQL] Check schema consistency when using K...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13632 **[Test build #60442 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60442/consoleFull)** for PR 13632 at commit

[GitHub] spark pull request #13527: [SPARK-15782] [CORE] Set spark.jars system proper...

2016-06-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13527#discussion_r66882709 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2305,6 +2305,23 @@ private[spark] object Utils extends Logging {

[GitHub] spark pull request #13651: [SPARK-15776][SQL] Divide Expression inside Aggre...

2016-06-13 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/13651#discussion_r66882617 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2847,4 +2847,15 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark issue #13649: [SPARK-15929] Fix portability of DataFrameSuite path glo...

2016-06-13 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13649 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #12951: [SPARK-15176][Core] Add maxShares setting to Pools

2016-06-13 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/12951 Also, once naming is settled on, this PR should include a documentation update to this page: https://spark.apache.org/docs/latest/job-scheduling.html to describe this. --- If your project

[GitHub] spark issue #13632: [SPARK-15910][SQL] Check schema consistency when using K...

2016-06-13 Thread clockfly
Github user clockfly commented on the issue: https://github.com/apache/spark/pull/13632 @cloud-fan Updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13652: [SPARK-] Fix incorrect days to millis conversion

2016-06-13 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/13652 cc @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13546: [SPARK-15808] [SQL] File Format Checking When Appending ...

2016-06-13 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/13546 I think we cannot ban the format when the append is used in `DataFrameWriter`. For example, when I use `createDF(10, 19).write.mode(SaveMode.Append).format("text").saveAsTable("appendJsonToText")`,

[GitHub] spark pull request #13652: [SPARK-] Fix incorrect days to millis conversion

2016-06-13 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/13652 [SPARK-] Fix incorrect days to millis conversion ## What changes were proposed in this pull request? Internally, we use Int to represent a date (the days since 1970-01-01), when we

[GitHub] spark pull request #13651: [SPARK-15776][SQL] Divide Expression inside Aggre...

2016-06-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13651#discussion_r66882340 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -213,7 +213,7 @@ case class Multiply(left:

[GitHub] spark pull request #13527: [SPARK-15782] [CORE] Set spark.jars system proper...

2016-06-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13527#discussion_r66882303 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2305,6 +2305,23 @@ private[spark] object Utils extends Logging {

[GitHub] spark issue #13644: [SPARK-15925][SQL][SPARKR] Replaces registerTempTable wi...

2016-06-13 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13644 Merging this to master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13650 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60436/ Test PASSed. ---

[GitHub] spark pull request #13527: [SPARK-15782] [CORE] Set spark.jars system proper...

2016-06-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13527#discussion_r66882183 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -484,6 +484,13 @@ object SparkSubmit {

[GitHub] spark pull request #13527: [SPARK-15782] [CORE] Set spark.jars system proper...

2016-06-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13527#discussion_r66882129 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -391,7 +391,9 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13650 **[Test build #60436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60436/consoleFull)** for PR 13650 at commit

[GitHub] spark pull request #13638: [SPARK-15915][SQL] CacheManager should use canoni...

2016-06-13 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/13638#discussion_r66882083 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -155,8 +156,9 @@ private[sql] class CacheManager extends Logging

[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60430/ Test PASSed. ---

[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13623 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13650 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13338: [SPARK-13723] [YARN] Change behavior of --num-executors ...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13338 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13638: [SPARK-15915][SQL] CacheManager should use canoni...

2016-06-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/13638#discussion_r66881822 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -155,8 +156,9 @@ private[sql] class CacheManager extends Logging {

[GitHub] spark issue #12951: [SPARK-15176][Core] Add maxShares setting to Pools

2016-06-13 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/12951 The naming on this PR is somewhat confusing, because it looks like maxShares is supposed to return the maximum number of *remaining* tasks that can be run, rather than the maximum number of

[GitHub] spark issue #13338: [SPARK-13723] [YARN] Change behavior of --num-executors ...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13338 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60426/ Test PASSed. ---

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60435/ Test PASSed. ---

[GitHub] spark pull request #13651: [SPARK-15776][SQL] Divide Expression inside Aggre...

2016-06-13 Thread clockfly
Github user clockfly commented on a diff in the pull request: https://github.com/apache/spark/pull/13651#discussion_r66881813 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -213,7 +213,7 @@ case class Multiply(left:

[GitHub] spark issue #13623: [SPARK-15895][SQL] Filters out metadata files while doin...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13623 **[Test build #60430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60430/consoleFull)** for PR 13623 at commit

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13650 **[Test build #60435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60435/consoleFull)** for PR 13650 at commit

[GitHub] spark pull request #12951: [SPARK-15176][Core] Add maxShares setting to Pool...

2016-06-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/12951#discussion_r66881687 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala --- @@ -59,12 +59,15 @@ private[spark] class

[GitHub] spark issue #13338: [SPARK-13723] [YARN] Change behavior of --num-executors ...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13338 **[Test build #60426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60426/consoleFull)** for PR 13338 at commit

[GitHub] spark pull request #13651: [SPARK-15776][SQL] Divide Expression inside Aggre...

2016-06-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13651#discussion_r66881454 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -213,7 +213,7 @@ case class Multiply(left:

[GitHub] spark issue #13644: [SPARK-15925][SQL][SPARKR] Replaces registerTempTable wi...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13644 **[Test build #60428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60428/consoleFull)** for PR 13644 at commit

[GitHub] spark pull request #10953: [SPARK-12177] [STREAMING] Update KafkaDStreams to...

2016-06-13 Thread markgrover
Github user markgrover closed the pull request at: https://github.com/apache/spark/pull/10953 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #13644: [SPARK-15925][SQL][SPARKR] Replaces registerTempTable wi...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13644 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60428/ Test PASSed. ---

[GitHub] spark pull request #12951: [SPARK-15176][Core] Add maxShares setting to Pool...

2016-06-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/12951#discussion_r66881361 --- Diff: core/src/main/scala/org/apache/spark/scheduler/Pool.scala --- @@ -47,6 +48,15 @@ private[spark] class Pool( var name = poolName

[GitHub] spark pull request #13651: [SPARK-15776][SQL] Divide Expression inside Aggre...

2016-06-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13651#discussion_r66881195 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -213,7 +213,7 @@ case class Multiply(left:

[GitHub] spark issue #13140: [SPARK-15230] [SQL] distinct() does not handle column na...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13140 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60429/ Test PASSed. ---

[GitHub] spark issue #13140: [SPARK-15230] [SQL] distinct() does not handle column na...

2016-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13140 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13563: [SPARK-15826] [CORE] PipedRDD to allow configurable char...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13563 **[Test build #60440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60440/consoleFull)** for PR 13563 at commit

[GitHub] spark pull request #13651: [SPARK-15776][SQL] Divide Expression inside Aggre...

2016-06-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13651#discussion_r66880904 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -213,7 +213,7 @@ case class Multiply(left:

[GitHub] spark issue #13140: [SPARK-15230] [SQL] distinct() does not handle column na...

2016-06-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13140 **[Test build #60429 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60429/consoleFull)** for PR 13140 at commit

[GitHub] spark issue #13563: [SPARK-15826] [CORE] PipedRDD to allow configurable char...

2016-06-13 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/13563 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #13638: [SPARK-15915][SQL] CacheManager should use canoni...

2016-06-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13638#discussion_r66880856 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -155,8 +156,9 @@ private[sql] class CacheManager extends

<    1   2   3   4   5   6   7   8   >