[GitHub] spark issue #22276: [SPARK-25242][SQL] make sql config setting fluent

2018-08-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22276 We can think about this API change in 3.0 release. Now, Spark 2.4 release is not allowed to break the binary compatibility. ---

[GitHub] spark issue #22278: emphasize deprecation [documentation tweak]

2018-08-29 Thread MichaelChirico
Github user MichaelChirico commented on the issue: https://github.com/apache/spark/pull/22278 @HyukjinKwon this is about usability/user-friendliness. As mentioned, despite being a seasoned R user & increasingly familiar with `SparkR` documentation, I spent several minutes googling

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22138 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95450/ Test PASSed. ---

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22138 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22138 **[Test build #95450 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95450/testReport)** for PR 22138 at commit

[GitHub] spark issue #22204: [SPARK-25196][SQL] Analyze column statistics in cached q...

2018-08-29 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22204 @maropu . Is this PR suggesting that the user need to use `CacheManager.analyzeColumnCacheQuery`? In general `CacheManager` is internal to Spark SQL, isn't it? ---

[GitHub] spark pull request #22197: [SPARK-25207][SQL] Case-insensitve field resoluti...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22197#discussion_r213906227 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -1021,6 +1022,113 @@ class

[GitHub] spark issue #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metastore sche...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22267 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95453/ Test PASSed. ---

[GitHub] spark issue #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metastore sche...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22267 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metastore sche...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22267 **[Test build #95453 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95453/testReport)** for PR 22267 at commit

[GitHub] spark pull request #22263: [SPARK-25269][SQL] SQL interface support specify ...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22263#discussion_r213904343 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -162,7 +162,8 @@ statement tableIdentifier

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95455 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95455/testReport)** for PR 22197 at commit

[GitHub] spark issue #22149: [SPARK-25158][SQL]Executor accidentally exit because Scr...

2018-08-29 Thread LuciferYang
Github user LuciferYang commented on the issue: https://github.com/apache/spark/pull/22149 > Is that possible to add a test case? Thanks for your reply Xiao, I tried to add a test case to reproduce bad case, but it only reproduce in small probability,we need to ensure that Task

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22197 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22197 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22197 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95449/ Test PASSed. ---

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95449/testReport)** for PR 22197 at commit

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22198 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95448/ Test PASSed. ---

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22198 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22278: emphasize deprecation [documentation tweak]

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22278 I think there are many same instances. We should fix them everywhere consistently. But I don't think it's worth enough fixing everywhere. It's marked as deprecated and so it's deprecated, isn't

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22198 **[Test build #95448 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95448/testReport)** for PR 22198 at commit

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22197 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95454/ Test FAILed. ---

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22197 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95454/testReport)** for PR 22197 at commit

[GitHub] spark issue #22278: emphasize deprecation [documentation tweak]

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22278 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22278: emphasize deprecation [documentation tweak]

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22278 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22278: emphasize deprecation [documentation tweak]

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22278 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22278: emphasize deprecation

2018-08-29 Thread MichaelChirico
GitHub user MichaelChirico opened a pull request: https://github.com/apache/spark/pull/22278 emphasize deprecation Was poking around documentation for `formals` of this method as I'd seen it recommended and completely missed the `Deprecated` tag; even after seeing elsewhere it's

[GitHub] spark issue #22277: [SPARK-25276] Redundant constrains when using alias

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22277 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22277: [SPARK-25276] Redundant constrains when using alias

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22277 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22277: [SPARK-25276] Redundant constrains when using alias

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22277 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22277: [SPARK-25276] Redundant constrains when using ali...

2018-08-29 Thread ajithme
GitHub user ajithme opened a pull request: https://github.com/apache/spark/pull/22277 [SPARK-25276] Redundant constrains when using alias Attaching a test to reproduce the issue. The test fails with following message test("redundant constrains") { val tr =

[GitHub] spark pull request #22264: [SPARK-25256][SQL] Plan mismatch errors in Hive t...

2018-08-29 Thread sadhen
Github user sadhen closed the pull request at: https://github.com/apache/spark/pull/22264 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22264: [SPARK-25256][SQL] Plan mismatch errors in Hive t...

2018-08-29 Thread sadhen
GitHub user sadhen reopened a pull request: https://github.com/apache/spark/pull/22264 [SPARK-25256][SQL] Plan mismatch errors in Hive tests in 2.12 ## What changes were proposed in this pull request? ### For `SPARK-5775 read array from

[GitHub] spark issue #22264: [SPARK-25256][SQL] Plan mismatch errors in Hive tests in...

2018-08-29 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22264 @srowen please review, and this PR should be rebased on #22260 and then tested. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22209: [SPARK-24415][Core] Fixed the aggregated stage metrics b...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22209 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22209: [SPARK-24415][Core] Fixed the aggregated stage metrics b...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22209 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95444/ Test PASSed. ---

[GitHub] spark issue #22209: [SPARK-24415][Core] Fixed the aggregated stage metrics b...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22209 **[Test build #95444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95444/testReport)** for PR 22209 at commit

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95451/ Test FAILed. ---

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #95451 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95451/testReport)** for PR 19691 at commit

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-29 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/22146 I've been looking to mount additional volumes to the executor containers and just tried out the PR. It doesn't seem possible since if you add the container in the pod template,

[GitHub] spark issue #22276: [SPARK-25242][SQL] make sql config setting fluent

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22276 oh, yes, this is the issue of binary compatibility... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/2 Got it, I'll revert the changes in file source in this commit, thanks for your reply. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22197 @cloud-fan I reverted to the previous version. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22197 @dongjoon-hyun Sorry for the late response, description is changed to: > Although filter "ID < 100L" is generated by Spark, it fails to pushdown into parquet actually, Spark still does the

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21721 Since I'm continuously working on data source v2 API, this gets my attention. Do we have a story for the metrics in data source v2 streaming API? It's weird to me that we add public APIs that

[GitHub] spark issue #22274: [SPARK-25167][SPARKR][TEST][MINOR] Minor fixes for R sql...

2018-08-29 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22274 @HyukjinKwon Sure.. here is the info. Please let me know if you need anything else. ``` > sessionInfo() R version 3.5.1 (2018-07-02) Platform: x86_64-apple-darwin15.6.0

[GitHub] spark issue #22265: [SPARK-25253][PYSPARK][FOLLOWUP] Undefined name: from py...

2018-08-29 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22265 ah, of course, `_exception_message` is only used in the exception handling, so we never get an error about an undefined name in any of the tests. Ok, thanks for the explanations, I appreciate it!

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-29 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22273 I'm kinda 'meh' / -0 on this change. My issue wasn't so much not seeing something printed out, it was more (a) python output isn't integrated into jenkins test reports and (b) I'm still learning my

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95454/testReport)** for PR 22197 at commit

[GitHub] spark issue #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metastore sche...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22267 **[Test build #95453 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95453/testReport)** for PR 22267 at commit

[GitHub] spark issue #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metastore sche...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22267 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metastore sche...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22267 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22260: [SQL][MINOR] Fix compiling for scala 2.12

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22260 LGTM, too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-29 Thread xuejianbest
Github user xuejianbest commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213888028 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -1184,6 +1184,25 @@ class UtilsSuite extends SparkFunSuite with

[GitHub] spark issue #22260: [SQL][MINOR] Fix compiling for scala 2.12

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22260 **[Test build #95452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95452/testReport)** for PR 22260 at commit

[GitHub] spark issue #22260: [SQL][MINOR] Fix compiling for scala 2.12

2018-08-29 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22260 @maropu @srowen please review --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r213886899 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/FailureSafeParser.scala --- @@ -15,50 +15,51 @@ * limitations under the

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r213885777 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,28 @@ case class JsonToStructs(

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r213885328 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,28 @@ case class JsonToStructs(

[GitHub] spark issue #22184: [SPARK-25132][SQL][DOC] Add migration doc for case-insen...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22184 see `ParquetOptions`. Option can be specified per-query while SQL conf is per-session. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22267: [SPARK-24716][TESTS][FOLLOW-UP] Test Hive metasto...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22267#discussion_r213885090 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -1021,6 +1021,18 @@ class

[GitHub] spark pull request #22260: [SQL][MINOR] Fix compiling for scala 2.12

2018-08-29 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/22260#discussion_r213884979 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ProjectionOverSchema.scala --- @@ -38,7 +38,7 @@ private[execution] case class

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r213884870 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +35,24 @@ class

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213884655 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -301,16 +301,16 @@ class Dataset[T] private[sql]( // Compute the width

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/2 can we do it for data source v2 first? It seems hard to fix the file source, as its reader function may lie about the return type. Let's see what's the simplest fix to remove the hack for

[GitHub] spark pull request #22260: [SQL][MINOR] Fix compiling for scala 2.12

2018-08-29 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/22260#discussion_r213884559 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala --- @@ -19,7 +19,7 @@ package org.apache.spark.sql.hive

[GitHub] spark issue #22276: [SPARK-25242][SQL] make sql config setting fluent

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22276 ?? This doesn't break source compatibility but breaks binary compatibility. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22275: [SPARK-25274][PYTHON][SQL] In toPandas with Arrow send o...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22275 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95441/ Test PASSed. ---

[GitHub] spark issue #22275: [SPARK-25274][PYTHON][SQL] In toPandas with Arrow send o...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22275 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19691 @DazhuangSu Are u there? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22275: [SPARK-25274][PYTHON][SQL] In toPandas with Arrow send o...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22275 **[Test build #95441 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95441/testReport)** for PR 22275 at commit

[GitHub] spark pull request #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createU...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21912#discussion_r213884198 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -385,107 +385,124 @@ case class

[GitHub] spark pull request #22260: [SQL][MINOR] Fix compiling for scala 2.12

2018-08-29 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/22260#discussion_r213884075 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala --- @@ -19,7 +19,7 @@ package org.apache.spark.sql.hive

[GitHub] spark pull request #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createU...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21912#discussion_r213883747 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -452,6 +452,16 @@ public UnsafeArrayData

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-29 Thread xuejianbest
Github user xuejianbest commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213883324 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2794,6 +2794,27 @@ private[spark] object Utils extends Logging { }

[GitHub] spark issue #22274: [SPARK-25167][SPARKR][TEST][MINOR] Minor fixes for R sql...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22274 > The "date function on DataFrame" test fails consistently on my laptop. @dilipbiswal, would you mind if I ask to list up R version (and additional env information if possible)? ---

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22197 > Is it acceptable? apparently not... OK let's just check duplicated filed names twice: one in filter pushdown, one in column pruning. And clean it up in followup PRs. ---

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-29 Thread xuejianbest
Github user xuejianbest commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213882803 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -301,16 +301,16 @@ class Dataset[T] private[sql]( // Compute the

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #95451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95451/testReport)** for PR 19691 at commit

[GitHub] spark pull request #21977: [SPARK-25004][CORE] Add spark.executor.pyspark.me...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r213882447 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -62,14 +63,20 @@ private[spark] object PythonEvalType { */

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19691 Could anyone take over this then? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19691 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22253: [SPARK-24411] [SQL] Adding native Java tests for ...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22253#discussion_r213881717 --- Diff: sql/core/src/test/java/test/org/apache/spark/sql/JavaColumnExpressionSuite.java --- @@ -0,0 +1,80 @@ +package test.org.apache.spark.sql;

[GitHub] spark pull request #22253: [SPARK-24411] [SQL] Adding native Java tests for ...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22253#discussion_r213881608 --- Diff: sql/core/src/test/java/test/org/apache/spark/sql/JavaColumnExpressionSuite.java --- @@ -0,0 +1,80 @@ +package test.org.apache.spark.sql;

[GitHub] spark issue #22255: [SPARK-25102][Spark Core] Write Spark version informatio...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22255 I would also rather write the justification for this change, for instance, linking the usage of this name in Parquet side, potential usage, etc. ---

[GitHub] spark pull request #22255: [SPARK-25102][Spark Core] Write Spark version inf...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22255#discussion_r213881049 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetWriteSupport.scala --- @@ -29,6 +29,7 @@ import

[GitHub] spark pull request #22266: [SPARK-25270] lint-python: Add flake8 to find syn...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22266#discussion_r213880767 --- Diff: dev/lint-python --- @@ -82,6 +82,26 @@ else rm "$PYCODESTYLE_REPORT_PATH" fi +python -m pip install flake8 --- End

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213880735 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -1184,6 +1184,25 @@ class UtilsSuite extends SparkFunSuite with

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22273 Hm .. what I was thinking was, we assume they are installed by default, and skipped tests are logged. --- - To

[GitHub] spark issue #22272: [SPARK-25273][DOC] How to install testthat 1.0.2

2018-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22272 @MaxGekk, mind checking https://github.com/apache/spark/blob/master/docs/README.md#prerequisites too and fix or remove it too? ---

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-29 Thread xuejianbest
Github user xuejianbest commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213879659 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -1184,6 +1184,25 @@ class UtilsSuite extends SparkFunSuite with

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-29 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/22138 Thanks @attilapiros I fixed the missing spots you left comments. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22138 **[Test build #95450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95450/testReport)** for PR 22138 at commit

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22198 Aha, I see. It is simple to match identifiers literally. So, let me wait for other developers comments. cc: @gatorsmile --- -

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22197 **[Test build #95449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95449/testReport)** for PR 22197 at commit

[GitHub] spark issue #22204: [SPARK-25196][SQL] Analyze column statistics in cached q...

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22204 ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22198 **[Test build #95448 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95448/testReport)** for PR 22198 at commit

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22198 Test PASSed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   4   5   6   7   >