Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117600957
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,69 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117600110
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,69 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117600909
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,69 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117600707
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,69 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117600885
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,69 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117601249
--- Diff:
common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java
---
@@ -730,4 +726,62 @@ public void testToLong() throws
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117601293
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
---
@@ -375,24 +374,61 @@ class
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117600921
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,69 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117601306
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -523,6 +524,14 @@ class SQLQuerySuite extends QueryTest
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117600718
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,69 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117600840
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,69 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117601121
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,69 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117601355
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
---
@@ -2015,4 +2015,121 @@ class SQLQuerySuite extends QueryTest
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/12646
Jenkins is about to shut down, we can retest this later
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/12646
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/18031#discussion_r117491182
--- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ---
@@ -193,8 +219,27 @@ private[spark] object HighlyCompressedMapStatus
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/18031#discussion_r117489904
--- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ---
@@ -193,8 +219,27 @@ private[spark] object HighlyCompressedMapStatus
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/18031#discussion_r117443085
--- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ---
@@ -193,8 +219,27 @@ private[spark] object HighlyCompressedMapStatus
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/18031#discussion_r117440204
--- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ---
@@ -121,48 +126,69 @@ private[spark] class CompressedMapStatus
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/18031#discussion_r117440029
--- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ---
@@ -193,8 +219,27 @@ private[spark] object HighlyCompressedMapStatus
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17985
@BartekH Yes, I think we can add that to exception message. Please also add
a test case for checking supported join types.
---
If your project is set up for it, you can reply to this email and have
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14971#discussion_r117175951
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala ---
@@ -175,7 +178,7 @@ class StatisticsSuite extends
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14971#discussion_r117176108
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala ---
@@ -175,7 +178,7 @@ class StatisticsSuite extends
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14971#discussion_r117174966
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
---
@@ -414,6 +415,50 @@ private[hive] class HiveClientImpl
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14971#discussion_r117169136
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveComparisonTest.scala
---
@@ -192,13 +192,7 @@ abstract class HiveComparisonTest
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14971#discussion_r117168812
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala ---
@@ -215,6 +218,217 @@ class StatisticsSuite extends
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14971#discussion_r117156770
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
---
@@ -414,6 +415,50 @@ private[hive] class HiveClientImpl
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14971#discussion_r117172781
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala ---
@@ -215,6 +218,217 @@ class StatisticsSuite extends
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14971#discussion_r117169168
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala ---
@@ -232,7 +446,8 @@ class StatisticsSuite extends
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r117148664
--- Diff:
common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java
---
@@ -730,4 +726,49 @@ public void testToLong() throws
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116947912
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,67 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116947565
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,67 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116925628
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java ---
@@ -510,6 +510,67 @@ public UTF8String trim
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116924800
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -1069,6 +1069,8 @@ class AstBuilder(conf: SQLConf
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116924583
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,246 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116924283
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,246 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116924073
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116903142
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116659903
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116656846
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116656190
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116655649
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116655664
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116655632
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116646341
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2306,7 +2306,15 @@ object functions {
* @group string_funcs
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116656332
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116646257
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2382,7 +2390,15 @@ object functions {
* @group string_funcs
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116646329
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2382,7 +2390,15 @@ object functions {
* @group string_funcs
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116656230
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116645110
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -1086,10 +1106,22 @@ class AstBuilder extends
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116656069
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116655023
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2447,7 +2463,15 @@ object functions {
* @group string_funcs
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116644643
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -1106,10 +1126,22 @@ class AstBuilder(conf: SQLConf
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116655691
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116646246
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2306,7 +2306,15 @@ object functions {
* @group string_funcs
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116644377
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -1091,6 +1094,23 @@ class AstBuilder(conf: SQLConf
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116655997
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r116642218
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -461,68 +462,249 @@ case class FindInSet(left
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17935#discussion_r116482273
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala
---
@@ -868,6 +868,29 @@ class SubquerySuite extends QueryTest
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17977
LGTM. I always set this true when running tpcds.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17985
"full_outer" will be replaced by "fullouter" after `.replace("_", "")`,
right?
---
If your project is set up for it, you can reply to this email and have your
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17962#discussion_r116174540
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
---
@@ -442,4 +442,14 @@ object QueryPlan
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17962#discussion_r116171180
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
---
@@ -442,4 +442,14 @@ object QueryPlan
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17962
cc @gatorsmile @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user wzhfy opened a pull request:
https://github.com/apache/spark/pull/17962
[SPARK-20718][SQL][followup] Fix canonicalization for HiveTableScanExec
## What changes were proposed in this pull request?
Fix canonicalization for different filter orders
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17959
@gatorsmile Right, thanks for pointing this out!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17959#discussion_r116162790
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
---
@@ -519,8 +519,18 @@ case class FileSourceScanExec
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17959
also cc @gatorsmile
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17959
cc @cloud-fan @hvanhovell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user wzhfy opened a pull request:
https://github.com/apache/spark/pull/17959
[SPARK-20718][SQL] FileSourceScanExec with different filter orders should
be the same after canonicalization
## What changes were proposed in this pull request?
Since `constraints
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/16677
@hvanhovell @cloud-fan We have seen value of this PR in our customer
scenarios, and that's why we started a discussion in dev list before. And thank
@viirya to discuss with us and implement
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16677#discussion_r115680727
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ---
@@ -90,25 +95,102 @@ trait BaseLimitExec extends UnaryExecNode
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17930
Not introduced by this pr, but I just saw a style nit
[here](https://github.com/apache/spark/pull/17930/files#diff-1d14ac233eac6f233c027dba0bdf871dR140)
```
if (conditions.isEmpty
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17918#discussion_r115672685
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala
---
@@ -421,7 +420,7 @@ class
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16677#discussion_r115642947
--- Diff: core/src/main/scala/org/apache/spark/MapOutputStatistics.scala ---
@@ -23,5 +23,9 @@ package org.apache.spark
* @param shuffleId ID
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16677#discussion_r115642967
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -156,12 +156,14 @@ private[spark] abstract class MapOutputTracker(conf
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16677#discussion_r115644428
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchange.scala
---
@@ -221,6 +221,12 @@ object ShuffleExchange
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16677#discussion_r115667172
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ---
@@ -90,25 +95,102 @@ trait BaseLimitExec extends UnaryExecNode
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16677#discussion_r115645927
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ---
@@ -90,25 +95,102 @@ trait BaseLimitExec extends UnaryExecNode
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16677#discussion_r115657718
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala ---
@@ -90,25 +95,102 @@ trait BaseLimitExec extends UnaryExecNode
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/16677#discussion_r115643011
--- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala ---
@@ -39,16 +40,18 @@ private[spark] sealed trait MapStatus {
* necessary
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17918#discussion_r115664182
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala
---
@@ -217,32 +217,18 @@ case class
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17918#discussion_r115664061
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala
---
@@ -217,32 +217,18 @@ case class
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17918#discussion_r115489131
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala
---
@@ -217,32 +217,18 @@ case class
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17918
@ron8hu You can look at tpcds q18.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17918
@ron8hu Yes, but I forgot the query numbers. The case is, first we filter a
large part of table A, then join table B on another column. Since ndv of that
column is not scaled down, the estimated
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17918
cc @cloud-fan @gatorsmile @ron8hu
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user wzhfy opened a pull request:
https://github.com/apache/spark/pull/17918
[SPARK-20678][SQL] Ndv for columns not in filter condition should also be
updated
## What changes were proposed in this pull request?
In filter estimation, we update column stats for those
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17897
@cloud-fan OK, let me close this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user wzhfy closed the pull request at:
https://github.com/apache/spark/pull/17897
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17897
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17897#discussion_r115238974
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -1025,3 +1025,5 @@ case class ShowCreateTableCommand(table
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17897#discussion_r115189581
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ---
@@ -114,7 +114,7 @@ class QueryExecution(val sparkSession
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17897#discussion_r115189148
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
---
@@ -2561,6 +2561,14 @@ class SQLQuerySuite extends QueryTest
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17897
@cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user wzhfy opened a pull request:
https://github.com/apache/spark/pull/17897
[SPARK-20635][SQL] No SQL tab in Spark UI
## What changes were proposed in this pull request?
There's no SQL tab in Spark UI since `SQLExecution.withNewExecutionId` is
removed in `def
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17649
The changes look good to me if we don't care about the case sensitivity
issue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17649
@gatorsmile Hive treats comment simply as a key in the string-string
parameter map, while spark extracts comment from the map as a field in
`CatalogTable`. So the question is, should spark consider
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17649#discussion_r112635039
--- Diff:
sql/core/src/test/resources/sql-tests/inputs/describe-table-after-alter-table.sql
---
@@ -0,0 +1,29 @@
+CREATE TABLE table_with_comment
Github user wzhfy commented on a diff in the pull request:
https://github.com/apache/spark/pull/17649#discussion_r112634259
--- Diff:
sql/core/src/test/resources/sql-tests/inputs/describe-table-after-alter-table.sql
---
@@ -0,0 +1,29 @@
+CREATE TABLE table_with_comment
601 - 700 of 1296 matches
Mail list logo
