Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23132
`spark.sql.legacy.decimalParsing.enabled` is still shown in the PR
description and commit messages.
---
-
To unsubscribe, e
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r240104815
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -27,6 +27,8 @@ displayTitle: Spark SQL Upgrading Guide
- In Spark version 2.4
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23268
The existing way is too JAVA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23268
@sadhen What is the motivation of this PR?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23248
LGTM to the surgical fix for backporting.
We need to fix this rule with the other rules for avoiding making such a
strong and hidden assumption
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23248#discussion_r240079120
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala
---
@@ -131,8 +131,20 @@ object ExtractPythonUDFs
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23255
For the reviewers, we need to merge this to 2.3, 2.4 and master. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r239895028
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
---
@@ -95,9 +77,116 @@ case class
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/23255
[SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned table using Hive
serde
## What changes were proposed in this pull request?
This was a Spark 2.3 regression introduced in
https
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r239887300
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
---
@@ -95,9 +77,116 @@ case class
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r239725957
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
---
@@ -95,9 +77,116 @@ case class
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22104#discussion_r239722680
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala ---
@@ -31,7 +31,8 @@ class SparkOptimizer(
override
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23245#discussion_r239574982
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1612,7 +1612,7 @@ object SQLConf {
.createWithDefault
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r239539992
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
---
@@ -95,9 +77,116 @@ case class
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23243
cc @jiangxb1987
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r239319889
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
---
@@ -95,9 +77,116 @@ case class
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r239264673
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ---
@@ -181,62 +180,39 @@ case class RelationConversions(
conf
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r239264207
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala
---
@@ -95,9 +77,116 @@ case class
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23222
We can compare the plans and see whether the rule takes an effect.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r238933039
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ---
@@ -181,62 +180,39 @@ case class RelationConversions(
conf
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23222
cc @ptkool @jiangxb1987 @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/23222
[SPARK-20636] Add the rule TransposeWindow to the optimization batch
## What changes were proposed in this pull request?
This PR is a follow-up of the PR
https://github.com/apache
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r238871523
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ---
@@ -181,62 +180,39 @@ case class RelationConversions(
conf
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22721#discussion_r238864486
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala
---
@@ -45,6 +45,8 @@ object CommandUtils extends Logging
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22899
LGTM
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22899
To be honest, we might still need to revisit it since it is still very
confusing to the developer which one they should use, top-down? or bottom-up?
The current use case for top-down is majorly
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23037
@vanzin Could you create a JIRA for this flaky test?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22899#discussion_r238483571
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -880,21 +880,38 @@ class Analyzer
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22899
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22857#discussion_r238450750
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -31,14 +31,14 @@ import org.apache.spark.scheduler.{SparkListener
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23163
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23197
Thank you for your contributions. Could you please close the PR?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23197#discussion_r238110847
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -119,14 +121,26 @@ object TypeCoercion
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23010
cc @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23196#discussion_r238110375
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -33,6 +33,8 @@ displayTitle: Spark SQL Upgrading Guide
- Spark applications which
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23196#discussion_r238110416
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -33,6 +33,8 @@ displayTitle: Spark SQL Upgrading Guide
- Spark applications which
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23199#discussion_r238110252
--- Diff:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -1045,6 +1046,11 @@ DOUBLE_LITERAL
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23199#discussion_r238109668
--- Diff: sql/core/src/test/resources/sql-tests/inputs/literals.sql ---
@@ -53,6 +53,10 @@ select .e3;
-- very large decimals (overflowing double
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238028577
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
---
@@ -223,14 +223,35 @@ abstract class Expression
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238027941
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala ---
@@ -780,6 +780,23 @@ class PlannerSuite extends SharedSQLContext
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23183
LGTM
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23183#discussion_r238014562
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/QueryPlanningTracker.scala
---
@@ -51,6 +58,18 @@ object QueryPlanningTracker
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23153#discussion_r237944306
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
---
@@ -155,19 +155,20 @@ object EliminateOuterJoin extends
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23152#discussion_r237776897
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
---
@@ -2276,4 +2276,16 @@ class SQLQuerySuite extends
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23086
I still do not think we should mix the catalog support with the data source
APIs. Catalog is a well-defined concept. The so-called "table catalog" is not a
catalog to me.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23176#discussion_r237766198
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/PredicateSuite.scala
---
@@ -293,6 +293,54 @@ class PredicateSuite
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23152
CC @liancheng @juliuszsompolski @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23152#discussion_r237717671
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -879,13 +879,13 @@ case
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21306
A general question. How to use this catalog API to implement the Hive
metastore? Is it doable?
---
-
To unsubscribe, e-mail
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21306#discussion_r237699163
--- Diff:
sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/Table.java ---
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23068
Could you post the UI if this is a job without using Spark SQL?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23068
This is a pretty useful enhancement.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23160
cc @gengliangwang
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23163
@c21 Any perf number?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23163
add to whitelist
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23166
cc @HyukjinKwon @ueshin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23168
cc @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23169
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23170
@wangyum Could you check whether the example works before? Is it a
regression?
---
-
To unsubscribe, e-mail: reviews
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23171
Also cc @ueshin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23131#discussion_r236803943
--- Diff: R/pkg/R/DataFrame.R ---
@@ -2732,6 +2732,20 @@ setMethod("union",
dataFrame(unioned)
})
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23139#discussion_r236468395
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala
---
@@ -0,0 +1,110
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23139#discussion_r236468109
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala
---
@@ -0,0 +1,110
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23139#discussion_r236456694
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala
---
@@ -79,29 +80,31 @@ object
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23139#discussion_r236450239
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala
---
@@ -79,29 +80,31 @@ object
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23139#discussion_r236377208
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala
---
@@ -0,0 +1,107
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23130#discussion_r236361995
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
---
@@ -388,7 +388,7 @@ case class FileSourceScanExec
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23139#discussion_r236157802
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala
---
@@ -0,0 +1,107
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23139
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23139#discussion_r236157275
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -736,69 +736,3 @@ object CombineConcats extends
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23104#discussion_r236137768
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -459,6 +459,7 @@ object LimitPushDown extends Rule
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23104
The title has a typo.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23104#discussion_r236137426
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -459,6 +459,7 @@ object LimitPushDown extends Rule
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23138#discussion_r236136273
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/SetOperationSuite.scala
---
@@ -196,4 +196,31 @@ class SetOperationSuite
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23138#discussion_r236136228
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/SetOperationSuite.scala
---
@@ -196,4 +196,31 @@ class SetOperationSuite
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23138#discussion_r236136178
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/SetOperationSuite.scala
---
@@ -196,4 +196,31 @@ class SetOperationSuite
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23130#discussion_r236135787
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
---
@@ -388,7 +388,7 @@ case class FileSourceScanExec
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23130#discussion_r236135647
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
---
@@ -388,7 +388,7 @@ case class FileSourceScanExec
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23131
Thanks! Merged to master.
Yes. Adding Distinct over Union is super expensive especially when the
underlying data set is huge
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23139#discussion_r236120731
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala
---
@@ -0,0 +1,107
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23135#discussion_r236117773
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
---
@@ -43,9 +43,24 @@ import
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23139
cc @dbtsai @aokolnychyi @rednaxelafx @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/23139
[SPARK-25860][SPARK-26107] [FOLLOW-UP] Rule ReplaceNullWithFalseInPredicate
## What changes were proposed in this pull request?
Based on https://github.com/apache/spark/pull/22857
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22857#discussion_r236098905
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -736,3 +736,60 @@ object CombineConcats extends
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22857#discussion_r236098841
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala
---
@@ -736,3 +736,60 @@ object CombineConcats extends
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23135#discussion_r236097033
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala
---
@@ -575,6 +575,19 @@ case class Range
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23137
CC @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/23137
[SPARK-26169] Create DataFrameSetOperationsSuite
## What changes were proposed in this pull request?
Create a new suite DataFrameSetOperationsSuite for the test cases of
DataFrame
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/23135
[SPARK-26168] Update the code comments in Expression and Aggregate
## What changes were proposed in this pull request?
This PR is to improve the code comments to document some common traits
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23135
cc @rxin @rednaxelafx @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23131
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23131#discussion_r236054278
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -1852,6 +1852,19 @@ class Dataset[T] private[sql](
CombineUnions
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23131
cc @rxin @srowen @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/23131
[SPARK-25908][SQL][FOLLOW-UP] Add back unionAll
## What changes were proposed in this pull request?
This PR is to add back `unionAll`, which is widely used. The name is also
consistent
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23128
@xuanyuanking Could you address the conflicts? Thanks for you fast work!
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22466
Regarding the potentially high cost of file listing, `CREATE DATABASE` is
not a frequent operation. The cost is high only if the target directory is
non-empty with many many files. We
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22466#discussion_r236005686
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
---
@@ -2370,4 +2370,17 @@ class HiveDDLSuite
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/23096
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22237#discussion_r234789275
--- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
@@ -1694,7 +1694,7 @@ test_that("column functions", {
df <- as.DataFrame(l
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22721
cc @jiangxb1987 Could you take a look at this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
1 - 100 of 14035 matches
Mail list logo
