[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread SongYadong
Github user SongYadong commented on the issue: https://github.com/apache/spark/pull/21836 I'm sorry for the scala style checks failure, i will read scala style guide doc again. --- - To unsubscribe, e-mail: reviews-

[GitHub] spark issue #21830: [SPARK-24878][SQL] Fix reverse function for array type o...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21830 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21830: [SPARK-24878][SQL] Fix reverse function for array type o...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21830 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93425/ Test PASSed. ---

[GitHub] spark issue #21830: [SPARK-24878][SQL] Fix reverse function for array type o...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21830 **[Test build #93425 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93425/testReport)** for PR 21830 at commit [`30d08ca`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21836 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21836 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93432/ Test FAILed. ---

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21836 **[Test build #93432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93432/testReport)** for PR 21836 at commit [`085ffdb`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21836 **[Test build #93432 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93432/testReport)** for PR 21836 at commit [`085ffdb`](https://github.com/apache/spark/commit/08

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21836 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93428/ Test FAILed. ---

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93428/testReport)** for PR 21764 at commit [`39b6ce9`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #21836: [SQL][HIVE] Correct an assert message in function...

2018-07-22 Thread SongYadong
Github user SongYadong commented on a diff in the pull request: https://github.com/apache/spark/pull/21836#discussion_r204290930 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala --- @@ -111,7 +111,8 @@ class HadoopTableReader( filterOpt: Opti

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21474 retest this please -- --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional c

[GitHub] spark pull request #21836: [SQL][HIVE] Correct an assert message in function...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21836#discussion_r204290224 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala --- @@ -111,7 +111,8 @@ class HadoopTableReader( filterOpt: Opt

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21474 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-m

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21822 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93420/ Test FAILed. ---

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r204289193 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -0,0 +1,153 @@ +/* + * L

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21822 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21822 **[Test build #93420 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93420/testReport)** for PR 21822 at commit [`38980ad`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21474 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21474 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93423/ Test FAILed. ---

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21474 **[Test build #93423 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93423/testReport)** for PR 21474 at commit [`61e1991`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r204288547 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -0,0 +1,156 @@ +/*

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r204288381 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/planning/SelectedFieldSuite.scala --- @@ -0,0 +1,388 @@ +/* + * Licensed to

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r204288233 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruning.scala --- @@ -0,0 +1,153 @@ +/* + * L

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21775 **[Test build #93431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93431/testReport)** for PR 21775 at commit [`76a34c6`](https://github.com/apache/spark/commit/76

[GitHub] spark pull request #21775: [SPARK-24812][SQL] Last Access Time in the table ...

2018-07-22 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/21775#discussion_r204287713 --- Diff: docs/sql-programming-guide.md --- @@ -1843,6 +1843,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see #

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21839 @xuanyuanking Could you add an end-to-end test into ScriptTransformationSuite to verify the results? --- - To unsubscribe, e-

[GitHub] spark pull request #21839: [SPARK-24339][SQL] Prunes the unused columns from...

2018-07-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21839#discussion_r204286954 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -450,13 +450,16 @@ object ColumnPruning extends Rul

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21845 **[Test build #93430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93430/testReport)** for PR 21845 at commit [`7afc5c5`](https://github.com/apache/spark/commit/7a

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21845 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1230/

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21845 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93422/ Test PASSed. ---

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21123 **[Test build #93422 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93422/testReport)** for PR 21123 at commit [`5ed7854`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #21775: [SPARK-24812][SQL] Last Access Time in the table ...

2018-07-22 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/21775#discussion_r204286227 --- Diff: docs/sql-programming-guide.md --- @@ -1843,6 +1843,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see #

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21845 **[Test build #93429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93429/testReport)** for PR 21845 at commit [`c57b745`](https://github.com/apache/spark/commit/c5

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21845 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1229/

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21845 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21775: [SPARK-24812][SQL] Last Access Time in the table ...

2018-07-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21775#discussion_r204285948 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -114,7 +114,10 @@ case class CatalogTablePartition(

[GitHub] spark pull request #21775: [SPARK-24812][SQL] Last Access Time in the table ...

2018-07-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21775#discussion_r204285853 --- Diff: docs/sql-programming-guide.md --- @@ -1843,6 +1843,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see ##

[GitHub] spark pull request #21845: [SPARK-24886][INFRA] Fix the testing script to in...

2018-07-22 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/21845 [SPARK-24886][INFRA] Fix the testing script to increase timeout for Jenkins build (from 300m to 350m) ## What changes were proposed in this pull request? Currently, looks we hit the ti

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21845 cc @rxin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21775 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93421/ Test PASSed. ---

[GitHub] spark issue #21775: [SPARK-24812][SQL] Last Access Time in the table descrip...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21775 **[Test build #93421 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93421/testReport)** for PR 21775 at commit [`140c4ce`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 LGTM, too --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.a

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93419/ Test PASSed. ---

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93419 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93419/testReport)** for PR 21764 at commit [`87afe4f`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21764 LGTM pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21764 **[Test build #93428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93428/testReport)** for PR 21764 at commit [`39b6ce9`](https://github.com/apache/spark/commit/39

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-22 Thread hthuynh2
Github user hthuynh2 commented on the issue: https://github.com/apache/spark/pull/21653 @tgravescs Can you please run the test again, thank you. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1228/

[GitHub] spark issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rul...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21764 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21764: [SPARK-24802][SQL] Add a new config for Optimizat...

2018-07-22 Thread maryannxue
Github user maryannxue commented on a diff in the pull request: https://github.com/apache/spark/pull/21764#discussion_r204279843 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -175,6 +191,41 @@ abstract class Optimizer(sessionC

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21653 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21653 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93418/ Test FAILed. ---

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21653 **[Test build #93418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93418/testReport)** for PR 21653 at commit [`b6585da`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #21764: [SPARK-24802][SQL] Add a new config for Optimizat...

2018-07-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21764#discussion_r204278968 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -175,6 +191,41 @@ abstract class Optimizer(sessionC

[GitHub] spark issue #21826: [SPARK-24872] Remove the symbol “||” of the “OR”...

2018-07-22 Thread httfighter
Github user httfighter commented on the issue: https://github.com/apache/spark/pull/21826 I have a suggestion that I don't know if it is reasonable. In our spark, since we already support “||” as a string concatenation function, I don't know if we can make such an improve

[GitHub] spark pull request #21830: [SPARK-24878][SQL] Fix reverse function for array...

2018-07-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21830#discussion_r204277225 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1244,46 +1244,50 @@ case class Reverse(ch

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21772 @liutang123 Thanks for this work. I'm curious that if this is an actual problem you hit in real application, or you just think it is problematic? --- ---

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-22 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r204276973 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala --- @@ -278,6 +278,39 @@ class HashedRelationSuite extends

[GitHub] spark issue #21826: [SPARK-24872] Remove the symbol “||” of the “OR”...

2018-07-22 Thread httfighter
Github user httfighter commented on the issue: https://github.com/apache/spark/pull/21826 I did the following tests in mysql. mysql> select "abc" || "def"; ++ | "abc" || "def" | ++ | 0

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-22 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r204276886 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -770,8 +771,10 @@ private[execution] final class LongToUn

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-22 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r204276788 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -726,9 +726,10 @@ private[execution] final class LongToUn

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21836 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21802: [SPARK-23928][SQL] Add shuffle collection functio...

2018-07-22 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21802#discussion_r204276502 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -1419,4 +1421,71 @@ class Collection

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21836 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93427/ Test FAILed. ---

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21836 **[Test build #93427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93427/testReport)** for PR 21836 at commit [`6a0f729`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21836 **[Test build #93427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93427/testReport)** for PR 21836 at commit [`6a0f729`](https://github.com/apache/spark/commit/6a

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21836 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21836: [SQL][HIVE] Correct an assert message in function makeRD...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21836 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #21440: [SPARK-24307][CORE] Support reading remote cached partit...

2018-07-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21440 cc @squito @mridulm @markhamstra @jerryshao @vanzin @JoshRosen @rxin @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #21440: [SPARK-24307][CORE] Support reading remote cached partit...

2018-07-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21440 Although the code quality is pretty good, I am still afraid it could introduce some unexpected issues. Is that possible we can introduce a conf to fall back to the previous implementation? We can

[GitHub] spark pull request #21830: [SPARK-24878][SQL] Fix reverse function for array...

2018-07-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21830#discussion_r204275800 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1244,46 +1244,50 @@ case class Reverse(ch

[GitHub] spark pull request #21802: [SPARK-23928][SQL] Add shuffle collection functio...

2018-07-22 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21802#discussion_r204275498 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -1419,4 +1421,71 @@ class Collection

[GitHub] spark issue #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21764 Also, can you update the title? You need to at least add `[SQL]` in the title: https://github.com/apache/spark/pull/21764#issuecomment-405832763 ---

[GitHub] spark pull request #21764: [SPARK-24802] Optimization Rule Exclusion

2018-07-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21764#discussion_r204275071 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -175,6 +191,41 @@ abstract class Optimizer(sessionCatal

[GitHub] spark issue #21784: [SPARK-24873][YARN] Turn off spark-shell noisy log outpu...

2018-07-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21784 Logging is already configurable; a switch is overkill. This seems fine. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.ap

[GitHub] spark pull request #21439: [SPARK-24391][SQL] Support arrays of any types by...

2018-07-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21439#discussion_r204274790 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -101,6 +102,17 @@ class JacksonParser( }

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-22 Thread liutang123
Github user liutang123 commented on the issue: https://github.com/apache/spark/pull/21772 @viirya Hi, Could you have more time to review this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21826: [SPARK-24872] Remove the symbol “||” of the �...

2018-07-22 Thread httfighter
Github user httfighter commented on a diff in the pull request: https://github.com/apache/spark/pull/21826#discussion_r204274497 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -442,8 +442,6 @@ case class Or(left: Expression,

[GitHub] spark pull request #21826: [SPARK-24872] Remove the symbol “||” of the �...

2018-07-22 Thread httfighter
Github user httfighter commented on a diff in the pull request: https://github.com/apache/spark/pull/21826#discussion_r204274481 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -442,8 +442,6 @@ case class Or(left: Expression,

[GitHub] spark issue #21784: [SPARK-24873][YARN] Turn off spark-shell noisy log outpu...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21784 Adding a configuration to control some logs sounds an overkill. I wouldn't go for this way. --- - To unsubscribe, e-mail: re

[GitHub] spark issue #20699: [SPARK-23544][SQL]Remove redundancy ShuffleExchange in t...

2018-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20699 **[Test build #93426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93426/testReport)** for PR 20699 at commit [`be96e39`](https://github.com/apache/spark/commit/be

[GitHub] spark issue #21784: [SPARK-24873][YARN] Turn off spark-shell noisy log outpu...

2018-07-22 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/21784 We need to listen to @vanzin opinion. Because the relevant code is what he wrote. --- - To unsubscribe, e-mail: review

[GitHub] spark issue #21828: Update regression.py

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21828 @woodthom2, if you have some plans to update this PR quite soon, please see https://spark.apache.org/contributing.html and proceed. Otherwise, I would suggest to leave this closed so that active

[GitHub] spark issue #21844: Spark 24873

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21844 I don't think we should as discussed in https://github.com/apache/spark/pull/21784. I think this PR would better be left closed and go discuss there. --- -

[GitHub] spark issue #21844: Spark 24873

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21844 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21844: Spark 24873

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21844 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21844: Spark 24873

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21844 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21844: Spark 24873

2018-07-22 Thread hejiefang
GitHub user hejiefang opened a pull request: https://github.com/apache/spark/pull/21844 Spark 24873 [https://issues.apache.org/jira/browse/SPARK-24873](url) I think it is better to add a switch to increase a switch to shielding frequent interaction reports with yarn. You can m

[GitHub] spark issue #21784: [SPARK-24873][YARN] Turn off spark-shell noisy log outpu...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21784 It has never been printed before, right? I think we can consider to turn it on specifically for spark-submit applications for some spark-submit applications although I am not fully sure if it's

[GitHub] spark issue #21784: [SPARK-24873][YARN] Turn off spark-shell noisy log outpu...

2018-07-22 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/21784 But for some spark-submit applications, I want these Application report for information. What should I do? --- - To

[GitHub] spark pull request #21843: for test

2018-07-22 Thread cuijunyao
Github user cuijunyao closed the pull request at: https://github.com/apache/spark/pull/21843 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21843: for test

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21843 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21843: for test

2018-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21843 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   3   >