[GitHub] spark issue #23268: [Hive][Minor] Refactor on HiveShim and Add Unit Tests

2018-12-09 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/23268 OK, I will modify the PR several hours later. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240110126 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This

[GitHub] spark issue #23270: [SPARK-26317][BUILD] Upgrade SBT to 0.13.18

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23270 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23269: [SPARK-26316] Currently the wrong implementation in the ...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23269 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23270: [SPARK-26317][BUILD] Upgrade SBT to 0.13.18

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23270 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23269: [SPARK-26316] Currently the wrong implementation in the ...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23269 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99896/ Test FAILed. ---

[GitHub] spark issue #23269: [SPARK-26316] Currently the wrong implementation in the ...

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23269 **[Test build #99896 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99896/testReport)** for PR 23269 at commit

[GitHub] spark issue #23270: [SPARK-26317][BUILD] Upgrade SBT to 0.13.18

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23270 **[Test build #99897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99897/testReport)** for PR 23270 at commit

[GitHub] spark pull request #23270: [SPARK-26317][BUILD] Upgrade SBT to 0.13.18

2018-12-09 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/23270 [SPARK-26317][BUILD] Upgrade SBT to 0.13.18 ## What changes were proposed in this pull request? SBT 0.13.14 ~ 1.1.1 has a bug on accessing `java.util.Base64.getDecoder` on JDK9+.

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240107064 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread maomaoChibei
Github user maomaoChibei commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240107003 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This

[GitHub] spark issue #23132: [SPARK-26163][SQL] Parsing decimals from JSON using loca...

2018-12-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/23132 `spark.sql.legacy.decimalParsing.enabled` is still shown in the PR description and commit messages. --- - To unsubscribe,

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240105070 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This

[GitHub] spark pull request #23124: [SPARK-25829][SQL] remove duplicated map keys wit...

2018-12-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/23124#discussion_r240104815 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -27,6 +27,8 @@ displayTitle: Spark SQL Upgrading Guide - In Spark version 2.4 and

[GitHub] spark pull request #23204: Revert "[SPARK-21052][SQL] Add hash map metrics t...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23204#discussion_r240104812 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashJoin.scala --- @@ -213,10 +213,6 @@ trait HashJoin {

[GitHub] spark pull request #23225: [SPARK-26287][CORE]Don't need to create an empty ...

2018-12-09 Thread wangjiaochun
Github user wangjiaochun commented on a diff in the pull request: https://github.com/apache/spark/pull/23225#discussion_r239990587 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -161,6 +161,10 @@ private void writeSortedFile(boolean

[GitHub] spark pull request #23225: [SPARK-26287][CORE]Don't need to create an empty ...

2018-12-09 Thread wangjiaochun
Github user wangjiaochun commented on a diff in the pull request: https://github.com/apache/spark/pull/23225#discussion_r239990083 --- Diff: core/src/test/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriterSuite.java --- @@ -235,6 +235,7 @@ public void writeEmptyIterator()

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240103941 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This function

[GitHub] spark issue #18784: [SPARK-21559][Mesos] remove mesos fine-grained mode

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18784 Looks inactive. @srowen and @felixcheung, do you know anyone who might be interested in this? --- - To unsubscribe,

[GitHub] spark issue #23269: [SPARK-26316] partial revert 21052 because of the perfor...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23269 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23269: [SPARK-26316] partial revert 21052 because of the perfor...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23269 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23269: [SPARK-26316] partial revert 21052 because of the perfor...

2018-12-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23269 @JkSelf . Since PR description already mentioned the background and the partital revert stuff, could you update the PR title to describe your solution more directly? `partial revert` doesn't

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240101927 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This

[GitHub] spark issue #23268: [Hive][Minor] Refactor on HiveShim and Add Unit Tests

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23268 Yup, the test looks okay in that way. Let's file a JIRA and only leave the test case. --- - To unsubscribe, e-mail:

[GitHub] spark issue #23269: [SPARK-26316] partial revert 21052 because of the perfor...

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23269 **[Test build #99896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99896/testReport)** for PR 23269 at commit

[GitHub] spark issue #23269: [SPARK-26316] partial revert 21052 because of the perfor...

2018-12-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23269 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23208: [SPARK-25530][SQL] data source v2 API refactor (b...

2018-12-09 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/23208#discussion_r240101369 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,52 +17,49 @@

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread JkSelf
Github user JkSelf commented on the issue: https://github.com/apache/spark/pull/23204 @cloud-fan the new ticket is in [here](https://github.com/apache/spark/pull/23269 ). I will close this ticket. --- - To

[GitHub] spark issue #23268: [Hive][Minor] Refactor on HiveShim and Add Unit Tests

2018-12-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/23268 The existing way is too JAVA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23204 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23204 **[Test build #99894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99894/testReport)** for PR 23204 at commit

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99894/ Test FAILed. ---

[GitHub] spark issue #23268: [Hive][Minor] Refactor on HiveShim and Add Unit Tests

2018-12-09 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/23268 Nevermind, just do not like the coding style personally. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #23269: partial revert 21052 because of the performance degradat...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23269 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23269: partial revert 21052 because of the performance degradat...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23269 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23269: partial revert 21052 because of the performance degradat...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23269 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23269: partial revert 21052 because of the performance d...

2018-12-09 Thread JkSelf
GitHub user JkSelf opened a pull request: https://github.com/apache/spark/pull/23269 partial revert 21052 because of the performance degradation in TPC-DS ## What changes were proposed in this pull request? We tested TPC-DS in spark2.3 with and without

[GitHub] spark pull request #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partit...

2018-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23255 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23255 thanks, merging to master/2.4/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240098806 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This function

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240098620 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This function

[GitHub] spark issue #23268: [Hive][Minor] Refactor on HiveShim and Add Unit Tests

2018-12-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/23268 @sadhen What is the motivation of this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23268: [Hive][Minor] Refactor on HiveShim and Add Unit Tests

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23268 Let's close this one. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240097931 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This

[GitHub] spark issue #23211: [SPARK-19712][SQL] Move PullupCorrelatedPredicates and R...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23211 to make the PR smaller, can we add an individual rule `PushdownLeftSemiOrAntiJoin` first? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #23225: [SPARK-26287][CORE]Don't need to create an empty ...

2018-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23225 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23211: [SPARK-19712][SQL] Move PullupCorrelatedPredicate...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23211#discussion_r240097479 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -984,6 +1002,28 @@ object PushDownPredicate extends

[GitHub] spark issue #23268: [Hive][Minor] Refactor on HiveShim and Add Unit Tests

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23268 **[Test build #99895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99895/testReport)** for PR 23268 at commit

[GitHub] spark pull request #23211: [SPARK-19712][SQL] Move PullupCorrelatedPredicate...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23211#discussion_r240097255 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -649,13 +664,16 @@ object CollapseProject extends

[GitHub] spark issue #23225: [SPARK-26287][CORE]Don't need to create an empty spill f...

2018-12-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23225 Thank you, @wangjiaochun . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #23268: [Hive][Minor] Refactor on HiveShim and Add Unit Tests

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23268 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23268: [Hive][Minor] Refactor on HiveShim and Add Unit Tests

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23268 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23268#discussion_r240097105 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala --- @@ -53,19 +53,12 @@ private[hive] object HiveShim { * This

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23255 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23255 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99891/ Test PASSed. ---

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23255 **[Test build #99891 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99891/testReport)** for PR 23255 at commit

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23204 can we follow https://github.com/apache/spark/pull/23204#issuecomment-445510026 and create a new ticket? --- - To

[GitHub] spark pull request #23268: [Hive][Minor] Refactor on HiveShim and Add Unit T...

2018-12-09 Thread sadhen
GitHub user sadhen opened a pull request: https://github.com/apache/spark/pull/23268 [Hive][Minor] Refactor on HiveShim and Add Unit Tests ## What changes were proposed in this pull request? Refactor on HiveShim, and add Unit Tests. ## How was this patch tested?

[GitHub] spark issue #22290: [SPARK-25285][CORE] Add executor task metrics, successfu...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22290 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread LuciferYang
Github user LuciferYang commented on the issue: https://github.com/apache/spark/pull/23204 ok~ already close #23214 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23214: [SPARK-26155] Optimizing the performance of LongT...

2018-12-09 Thread LuciferYang
Github user LuciferYang closed the pull request at: https://github.com/apache/spark/pull/23214 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23214: [SPARK-26155] Optimizing the performance of LongToUnsafe...

2018-12-09 Thread LuciferYang
Github user LuciferYang commented on the issue: https://github.com/apache/spark/pull/23214 As @cloud-fan said `the hash join metrics is wrongly implemented`, we will partial revert # SPARK-21052, no longer need this patch, close it ~ ---

[GitHub] spark issue #23225: [SPARK-26287][CORE]Don't need to create an empty spill f...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23225 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23225: [SPARK-26287][CORE]Don't need to create an empty spill f...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23225 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99890/ Test PASSed. ---

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread JkSelf
Github user JkSelf commented on the issue: https://github.com/apache/spark/pull/23204 @cloud-fan @dongjoon-hyun update the patch, please help review if you have time. Thanks. --- - To unsubscribe, e-mail:

[GitHub] spark issue #23225: [SPARK-26287][CORE]Don't need to create an empty spill f...

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23225 **[Test build #99890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99890/testReport)** for PR 23225 at commit

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23204 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23204 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23204 **[Test build #99894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99894/testReport)** for PR 23204 at commit

[GitHub] spark pull request #23211: [SPARK-19712][SQL] Move PullupCorrelatedPredicate...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23211#discussion_r240092936 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -267,6 +267,17 @@ object ScalarSubquery {

[GitHub] spark pull request #22305: [SPARK-24561][SQL][Python] User-defined window ag...

2018-12-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22305#discussion_r240092271 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/WindowInPandasExec.scala --- @@ -144,24 +282,107 @@ case class WindowInPandasExec(

[GitHub] spark issue #21877: [SPARK-24923][SQL][WIP] Add unpartitioned CTAS and RTAS ...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21877 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99893/ Test FAILed. ---

[GitHub] spark issue #21877: [SPARK-24923][SQL][WIP] Add unpartitioned CTAS and RTAS ...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21877 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21877: [SPARK-24923][SQL][WIP] Add unpartitioned CTAS and RTAS ...

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21877 **[Test build #99893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99893/testReport)** for PR 21877 at commit

[GitHub] spark issue #23248: [SPARK-26293][SQL] Cast exception when having python udf...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23248 If it's fine for 2.4, I think it's also fine for master as a temporary fix? We can create another ticket to clean up the subquery optimization hack. IIUC

[GitHub] spark issue #23228: [MINOR][DOC] Update the condition description of seriali...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23228 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21877: [SPARK-24923][SQL][WIP] Add unpartitioned CTAS and RTAS ...

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21877 **[Test build #99893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99893/testReport)** for PR 21877 at commit

[GitHub] spark issue #23228: [MINOR][DOC] Update the condition description of seriali...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23228 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23258: [SPARK-23375][SQL][FOLLOWUP][TEST] Test Sort metr...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23258#discussion_r240090371 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -182,10 +182,13 @@ class SQLMetricsSuite extends

[GitHub] spark pull request #23201: [SPARK-26246][SQL] Infer date and timestamp types...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23201#discussion_r240090192 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonInferSchema.scala --- @@ -121,7 +122,26 @@ private[sql] class

[GitHub] spark issue #23228: [MINOR][DOC] Update the condition description of seriali...

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23228 **[Test build #99892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99892/testReport)** for PR 23228 at commit

[GitHub] spark pull request #23265: [2.4][SPARK-26021][SQL][FOLLOWUP] only deal with ...

2018-12-09 Thread cloud-fan
Github user cloud-fan closed the pull request at: https://github.com/apache/spark/pull/23265 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23228: [MINOR][DOC] Update the condition description of seriali...

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23228 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23204 If we can quickly finish #23214 (within several days), let's go for it. But if we can't, I'd suggest we do the partial revert first to fix the perf regression, and add back the metrics later.

[GitHub] spark issue #23204: Revert "[SPARK-21052][SQL] Add hash map metrics to join"

2018-12-09 Thread LuciferYang
Github user LuciferYang commented on the issue: https://github.com/apache/spark/pull/23204 @cloud-fan If we decide to partial revert SPARK-21052 and no need for #23214, I will close it. --- - To unsubscribe,

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23255 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23255 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23255 **[Test build #99891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99891/testReport)** for PR 23255 at commit

[GitHub] spark issue #23223: [SPARK-26269][YARN]Yarnallocator should have same blackl...

2018-12-09 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/23223 Hi @tgravescs , I tried it, but found it's difficult to produce KILLED_BY_RESOURCEMANAGER exit status. I followed [YARN-73](https://issues.apache.org/jira/browse/YARN-73)

[GitHub] spark issue #23248: [SPARK-26293][SQL] Cast exception when having python udf...

2018-12-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/23248 LGTM to the surgical fix for backporting. We need to fix this rule with the other rules for avoiding making such a strong and hidden assumption. ---

[GitHub] spark pull request #23248: [SPARK-26293][SQL] Cast exception when having pyt...

2018-12-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/23248#discussion_r240079120 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -131,8 +131,20 @@ object ExtractPythonUDFs

[GitHub] spark pull request #22707: [SPARK-25717][SQL] Insert overwrite a recreated e...

2018-12-09 Thread fjh100456
Github user fjh100456 commented on a diff in the pull request: https://github.com/apache/spark/pull/22707#discussion_r240077883 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -774,4 +774,23 @@ class InsertSuite extends QueryTest with

[GitHub] spark pull request #23266: [SPARK-26313][SQL] move read related methods from...

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23266#discussion_r240074711 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/SupportsBatchRead.java --- @@ -20,14 +20,27 @@ import

[GitHub] spark pull request #23266: [SPARK-26313][SQL] move read related methods from...

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23266#discussion_r240073831 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/SupportsBatchRead.java --- @@ -20,14 +20,27 @@ import

[GitHub] spark pull request #22707: [SPARK-25717][SQL] Insert overwrite a recreated e...

2018-12-09 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22707#discussion_r240073151 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala --- @@ -774,4 +774,23 @@ class InsertSuite extends QueryTest with

[GitHub] spark issue #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23072 It looks enough to me, @srowen . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16812: [SPARK-19465][SQL] Added options for custom boolean valu...

2018-12-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16812 I still don't think need this since the workaround is easy. If other committers find it worth, I won't object. If there are no interests fro this PR afterwards, I would just close this.

[GitHub] spark pull request #22707: [SPARK-25717][SQL] Insert overwrite a recreated e...

2018-12-09 Thread fjh100456
Github user fjh100456 commented on a diff in the pull request: https://github.com/apache/spark/pull/22707#discussion_r240070378 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -227,18 +227,22 @@ case class

[GitHub] spark issue #23228: [MINOR][DOC] Update the condition description of seriali...

2018-12-09 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/23228 I have updated, thanks all. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #23267: [SPARK-25401] [SQL] Reorder join predicates to match chi...

2018-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23267 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   >