[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-08-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16677 @hvanhovell Shall we consider to include this into 2.4? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For add

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-03 Thread ajacques
Github user ajacques commented on the issue: https://github.com/apache/spark/pull/21889 @mallman: [This one](https://github.com/apache/spark/pull/21889/files#diff-0c6c7481232e9637b91c179f1005426aR120)? I just enabled it on my branch and the test passed. Was it fixed by your latest ch

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1789/

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16677 **[Test build #94195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94195/testReport)** for PR 16677 at commit [`69513d1`](https://github.com/apache/spark/commit/69

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-08-03 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20838 oh god another random single test failure. Jenkins retest this please. Maybe time to set aside some cycles for test stability :( --- ---

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20838 **[Test build #94196 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94196/testReport)** for PR 20838 at commit [`e41a8cc`](https://github.com/apache/spark/commit/e4

[GitHub] spark issue #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `Signed-o...

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21991 +1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark issue #21994: [SPARK-24529][Build][test-maven][follow-up] Add spotbugs...

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21994 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21994: [SPARK-24529][Build][test-maven][follow-up] Add spotbugs...

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21994 cc @yhuai too --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #21994: [SPARK-24529][Build][test-maven][follow-up] Add spotbugs...

2018-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21994 **[Test build #94197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94197/testReport)** for PR 21994 at commit [`3501209`](https://github.com/apache/spark/commit/35

[GitHub] spark issue #21993: [SPARK-24983][Catalyst] Add configuration for maximum nu...

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21993 Usually @gatorsmile and @cloud-fan. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional command

[GitHub] spark issue #21994: [SPARK-24529][Build][test-maven][follow-up] Add spotbugs...

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21994 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21994: [SPARK-24529][Build][test-maven][follow-up] Add spotbugs...

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21994 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1790/

[GitHub] spark issue #21956: [MINOR][DOCS] Fix grammatical error in SortShuffleManage...

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21956 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #21956: [MINOR][DOCS] Fix grammatical error in SortShuffleManage...

2018-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21956 **[Test build #94198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94198/testReport)** for PR 21956 at commit [`460e388`](https://github.com/apache/spark/commit/46

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21986 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark pull request #21984: [SPARK-24772][SQL] Avro: support logical date typ...

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21984#discussion_r207700882 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala --- @@ -92,7 +92,7 @@ class AvroSerializer(rootCatalystType: DataT

[GitHub] spark issue #21993: [SPARK-24983][Catalyst] Add configuration for maximum nu...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21993 Let us blacklist CASE WHEN in CollapseProject, instead of introducing this new conf. --- - To unsubscribe, e-mail: reviews-

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21986 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21986 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1791/

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21986 **[Test build #94199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94199/testReport)** for PR 21986 at commit [`3f88e2a`](https://github.com/apache/spark/commit/3f

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21721 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21993: [SPARK-24983][Catalyst] Add configuration for maximum nu...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21993 @dvogelbacher Currently, in the master branch (2.4 release), you have a workaround. Add CollapseProject to `spark.sql.optimizer.excludedRules` before such queries. ---

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21721 **[Test build #94200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94200/testReport)** for PR 21721 at commit [`1775c2a`](https://github.com/apache/spark/commit/17

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21699 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint ...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21911#discussion_r207701114 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala --- @@ -102,6 +104,32 @@ object ResolveHints {

[GitHub] spark pull request #21699: [SPARK-24722][SQL] pivot() with Column type argum...

2018-08-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21699 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21975: [SPARK-25001][BUILD] Fix miscellaneous build warnings

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21975 Ooops, i didn't know you were looking into this @srowen. Thanks for reviewing this. Will make some changes per the review comments. ---

[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21911 LGTM Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comm

[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21911 You can address the comment in the follow-up PR. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org Fo

[GitHub] spark pull request #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint ...

2018-08-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21911 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21889: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21889#discussion_r207701260 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -0,0 +1,205 @@ +/* +

[GitHub] spark pull request #21975: [SPARK-25001][BUILD] Fix miscellaneous build warn...

2018-08-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21975#discussion_r207701268 --- Diff: sql/hive/src/test/java/org/apache/spark/sql/hive/test/Complex.java --- @@ -676,7 +676,7 @@ public int compareTo(Complex other) { }

[GitHub] spark issue #21956: [MINOR][DOCS] Fix grammatical error in SortShuffleManage...

2018-08-03 Thread deshanxiao
Github user deshanxiao commented on the issue: https://github.com/apache/spark/pull/21956 Thanks a lot, I am sorry that I'm not able to find more clear errors. As a beginner, I am very glad to write more docs to help others understanding the spark core easier. --- -

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r207701302 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataType.scala --- @@ -336,4 +337,97 @@ object DataType { case (fromDataType,

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21948 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark pull request #21909: [SPARK-24959][SQL] Speed up count() for JSON and ...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21909#discussion_r207701331 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1476,6 +1476,14 @@ object SQLConf { "are performed

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21948 **[Test build #94201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94201/testReport)** for PR 21948 at commit [`86817c7`](https://github.com/apache/spark/commit/86

[GitHub] spark issue #21909: [SPARK-24959][SQL] Speed up count() for JSON and CSV

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21909 Please document it in the migration guide. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21403 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21948 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21948 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1792/

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1793/

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21403 **[Test build #94202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94202/testReport)** for PR 21403 at commit [`a6114a6`](https://github.com/apache/spark/commit/a6

[GitHub] spark pull request #21403: [SPARK-24341][SQL] Support only IN subqueries wit...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21403#discussion_r207701506 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -505,6 +505,7 @@ object NullPropagation extends R

[GitHub] spark issue #21975: [SPARK-25001][BUILD] Fix miscellaneous build warnings

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21975 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1794/

[GitHub] spark issue #21975: [SPARK-25001][BUILD] Fix miscellaneous build warnings

2018-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21975 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21975: [SPARK-25001][BUILD] Fix miscellaneous build warnings

2018-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21975 **[Test build #94203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94203/testReport)** for PR 21975 at commit [`6e85fc6`](https://github.com/apache/spark/commit/6e

[GitHub] spark pull request #21403: [SPARK-24341][SQL] Support only IN subqueries wit...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21403#discussion_r207701622 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala --- @@ -154,7 +154,7 @@ class ExpressionParserSui

[GitHub] spark pull request #21403: [SPARK-24341][SQL] Support only IN subqueries wit...

2018-08-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21403#discussion_r207701674 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -505,6 +505,7 @@ object NullPropagation extends R

[GitHub] spark issue #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `Signed-o...

2018-08-03 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21991 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

<    3   4   5   6   7   8