Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16928#discussion_r102645010
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -45,24 +45,41 @@ private[csv] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16928#discussion_r102646058
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -190,8 +208,9 @@ private[csv] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16928#discussion_r102646336
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -202,21 +221,25 @@ private[csv] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16928#discussion_r102646572
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -202,21 +221,25 @@ private[csv] class
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16938
@tejasapatil Spark doesn't need to be exactly same with Hive, we follow
hive behavior if it's reasonable, or use our own logic if hive's behavior
doesn't make sense.
---
If
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16594#discussion_r102647596
--- Diff:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -794,6 +795,7 @@ EXPLAIN: 'EXPLAIN';
FORMA
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16594
LGTM except one comment
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17001#discussion_r102648988
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
---
@@ -339,10 +340,17 @@ private[hive] class HiveClientImpl
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r102651748
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -233,3 +236,28 @@ private[csv] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r102649654
--- Diff: python/test_support/sql/ages_newlines.csv ---
@@ -0,0 +1,6 @@
+Joe,20,"Hi,
+I am Jeo"
+Tom,30,"My name is Tom"
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r102650051
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala
---
@@ -0,0 +1,256 @@
+/*
+ * Licensed to
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r102649816
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala
---
@@ -0,0 +1,256 @@
+/*
+ * Licensed to
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17009#discussion_r102652094
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -147,6 +147,10 @@ class SQLListenerSuite extends
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17009
The change looks reasonable
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/17036
[SPARK-19706][pyspark] add Column.contains in pyspark
## What changes were proposed in this pull request?
to be consistent with the scala API, we should also add `contains` to
`Column
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17036
cc @davies
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16928#discussion_r102656121
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -193,8 +193,9 @@ def json(self, path, schema=None,
primitivesAsString=None, prefersDecimal=None
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16928#discussion_r102657214
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -202,21 +212,41 @@ private[csv] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16928#discussion_r102657839
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -202,21 +212,41 @@ private[csv] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16928#discussion_r102658185
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -193,8 +193,9 @@ def json(self, path, schema=None,
primitivesAsString=None, prefersDecimal=None
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16928#discussion_r102658465
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -193,8 +193,9 @@ def json(self, path, schema=None,
primitivesAsString=None, prefersDecimal=None
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r102660255
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -233,3 +236,31 @@ private[csv] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r102661723
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -958,4 +975,46 @@ class CSVSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r102664433
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala
---
@@ -0,0 +1,256 @@
+/*
+ * Licensed to
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r102664832
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala
---
@@ -43,23 +37,26 @@ class CSVFileFormat extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17001#discussion_r102781147
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -408,7 +408,13 @@ private[spark] class HiveExternalCatalog
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17030
thanks, merging to master!
it conflicts with branch 2.1, can you submit a new PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16928
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16330
yea that's a good point, if we use temp dir by default, then Spark may lose
data without notice. So I'm not sure if we really want to do this in Spark R,
maybe we can ask users to all
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16996#discussion_r102808290
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala ---
@@ -61,6 +62,9 @@ private[sql] class SharedState(val sparkContext
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17009
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17036
thanks for the review, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17009
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16499#discussion_r102851869
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -1018,7 +1025,9 @@ private[spark] class BlockManager(
try
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16395#discussion_r102859839
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -0,0 +1,531
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16395#discussion_r102864512
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -0,0 +1,511
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16395#discussion_r102860017
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -0,0 +1,511
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16395#discussion_r102862837
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/Range.scala
---
@@ -57,6 +58,20 @@ object Range
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16395#discussion_r102865689
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala
---
@@ -0,0 +1,403
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16395#discussion_r102863871
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -0,0 +1,511
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16395#discussion_r102862889
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -0,0 +1,511
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16395#discussion_r102862209
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -0,0 +1,511
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16395#discussion_r102860256
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -0,0 +1,511
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16395
LGTM except some minor comments, you can address them in follow-up
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17001
In `SharedState`, we should create the default database without location.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16996
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17001
can we use a special static path? It's weird that the underlying value of
default database location is decided by the first Spark application who connect
to hive.
---
If your project is s
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17001
> the default database is not created in ShareState, it is created when
HiveClientImpl init.
what do you mean? are you saying it's useless to create default database in
`Sha
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16395
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17051#discussion_r102887105
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -1,511 +1,509
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17051#discussion_r102887733
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala
---
@@ -398,6 +398,27 @@ class
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16594
LGTM, pending test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16996
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17051
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16594
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r103012984
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/CodecStreams.scala
---
@@ -86,4 +88,11 @@ object CodecStreams
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r103015384
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -961,56 +978,135 @@ class CSVSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r103012905
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/CodecStreams.scala
---
@@ -86,4 +88,11 @@ object CodecStreams
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r103016252
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -961,56 +978,135 @@ class CSVSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r103015905
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -958,4 +975,77 @@ class CSVSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r103014118
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala
---
@@ -0,0 +1,238 @@
+/*
+ * Licensed to
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r103015057
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -269,3 +273,89 @@ private[csv] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16976#discussion_r103014004
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala
---
@@ -0,0 +1,238 @@
+/*
+ * Licensed to
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16938#discussion_r103017698
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala
---
@@ -140,8 +140,8 @@ case class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16938#discussion_r103018002
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala
---
@@ -140,8 +140,8 @@ case class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17001#discussion_r103045973
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -408,7 +408,15 @@ private[spark] class HiveExternalCatalog
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17001#discussion_r103046202
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -408,7 +408,15 @@ private[spark] class HiveExternalCatalog
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16944#discussion_r103046982
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -296,6 +296,25 @@ object SQLConf {
.longConf
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16944#discussion_r103047268
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala ---
@@ -21,6 +21,7 @@ import org.apache.hadoop.fs.Path
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16944#discussion_r103048200
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -510,8 +510,13 @@ private[spark] class HiveExternalCatalog
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16944#discussion_r103049318
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -296,6 +296,25 @@ object SQLConf {
.longConf
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16944#discussion_r103049597
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -161,22 +164,51 @@ private[hive] class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16944#discussion_r103049915
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -226,6 +258,41 @@ private[hive] class HiveMetastoreCatalog
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16944#discussion_r103050028
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -226,6 +258,41 @@ private[hive] class HiveMetastoreCatalog
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17027#discussion_r103050183
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -175,19 +175,14 @@ class Dataset[T] private[sql
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17027#discussion_r103050359
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ---
@@ -125,8 +125,6 @@ class QueryExecution(val sparkSession
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17027
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16944#discussion_r103055530
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -510,8 +510,13 @@ private[spark] class HiveExternalCatalog
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/14412
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/17065
[SPARK-17075][SQL][followup] fix some minor issues and clean up the code
## What changes were proposed in this pull request?
This fixes some code style issues, naming issues, some
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17065#discussion_r103073076
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -95,15 +84,16 @@ case
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17065#discussion_r103073081
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -140,56 +129,56 @@ case
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17065#discussion_r103073122
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -258,27 +246,20 @@ case
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17065#discussion_r103073163
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -297,6 +278,8 @@ case
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17065#discussion_r103073176
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -361,57 +343,52 @@ case
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17065
CC @ron8hu @wzhfy
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17063
LGTM, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17027#discussion_r103073282
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -175,19 +175,14 @@ class Dataset[T] private[sql
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17027
merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17065#discussion_r103097107
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala
---
@@ -95,15 +84,16 @@ case
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17065
thanks for the review, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17065
@ron8hu can you fix
https://github.com/apache/spark/pull/17065#discussion_r103087483 and
https://github.com/apache/spark/pull/17065#discussion_r103073122 if you have
time? thanks!
---
If your
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17071#discussion_r103097154
--- Diff:
examples/src/main/scala/org/apache/spark/examples/sql/SQLDataSourceExample.scala
---
@@ -135,10 +138,10 @@ object SQLDataSourceExample
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17071#discussion_r103097192
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
---
@@ -590,7 +590,7 @@ class JsonSuite extends
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17071#discussion_r103097195
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
---
@@ -828,7 +828,7 @@ class JsonSuite extends
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/17075
I think we should fix `changePrecison` to return a new instance instead of
updating itself.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/17064#discussion_r103277557
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala ---
@@ -168,15 +168,16 @@ class CacheManager extends Logging
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16119#discussion_r93575937
--- Diff: python/pyspark/sql/tests.py ---
@@ -1851,6 +1851,71 @@ def test_hivecontext(self):
self.assertIn("default", out.dec
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16119#discussion_r93576154
--- Diff: python/pyspark/sql/context.py ---
@@ -72,8 +72,13 @@ def __init__(self, sparkContext, sparkSession=None,
jsqlContext=None):
self
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16378
LGTM, merging to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
401 - 500 of 17657 matches
Mail list logo
