Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r137031435
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -503,69 +504,319 @@ case class FindInSet
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r137366326
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -503,69 +504,319 @@ case class FindInSet
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/12646#discussion_r137366302
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
---
@@ -503,69 +504,319 @@ case class FindInSet
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20611
@sujith71955 Can you run this test case with your code?
create a file on your local machine (for example, text1.txt) with some data
then run these two LOAD DATA commands in your spark-sql
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/21285
@HyukjinKwon thanks for reviewing this pr. I didn't notice that pr until
you point out. If we plan to support wildcard in the LOAD DATA command, then we
can close this PR.
But with his
GitHub user kevinyu98 opened a pull request:
https://github.com/apache/spark/pull/21285
[SPARK-24176][SQL] LOAD DATA can't identify wildcard in the hdfs file path
## What changes were proposed in this pull request?
When the wildcard characters (like "?") were i
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r189688743
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/LookupFunctionsSuite.scala
---
@@ -0,0 +1,63 @@
+/*
+ * Licensed
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20795
@gatorsmile Hello Sean, I have updated the code, can you review the code
and see anything I need to make changes when you have time? thanks
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20611
@sujith71955 Sorry for the delay. I just ran your test case with my fix
only, and it run successfully. Can you verify it? If it is true, then my fix is
much simple, what do you think? Thanks
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20795
sorry for the delay. I was working on some other projects. I am back and
focus on addressing the comments now
GitHub user kevinyu98 opened a pull request:
https://github.com/apache/spark/pull/22100
[PySpark] [SQL]include PandasUDFType in the import all of
pyspark.sql.functions
## What changes were proposed in this pull request?
include pyspark.sql.functions in the import all
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22100#discussion_r210091625
--- Diff: python/pyspark/sql/functions.py ---
@@ -2927,7 +2927,7 @@ def pandas_udf(f=None, returnType=None,
functionType=None):
__all__ = [k for k
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/22100
@HyukjinKwon @dilipbiswal Thanks for your comments, I have changed the
code. I removed the code changes in tests.py, because it got warning message
during style checking, and I did manually
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/22100
@holdenk Hello Holden: Can you check the code changes? Is this what you
expected behavior? Thanks.
---
-
To unsubscribe, e
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/21285
close this pr, pr#20611 has combined this fix into his.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user kevinyu98 closed the pull request at:
https://github.com/apache/spark/pull/21285
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20795
@gatorsmile Hi Sean, I am so sorry for the long delay. I will address the
comments today and submit the code for reviewing.
Thanks very much !
Kevin
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r202138683
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1204,16 +1207,32 @@ class Analyzer(
* only
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r202207142
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/LookupFunctionsSuite.scala
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r201921920
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/LookupFunctionsSuite.scala
---
@@ -0,0 +1,63 @@
+/*
+ * Licensed
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r201921878
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/LookupFunctionsSuite.scala
---
@@ -0,0 +1,70 @@
+/*
+ * Licensed
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20795
@viirya Thanks a lot. I will create a new test file LookupFunctionsSuite
under sql/catalyst/analysis.
---
-
To unsubscribe
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r173662663
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1192,10 +1193,18 @@ class Analyzer(
* @see
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r173692751
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1192,10 +1193,18 @@ class Analyzer(
* @see
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r174019434
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1192,11 +1195,23 @@ class Analyzer(
* @see
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r174017514
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1192,11 +1195,23 @@ class Analyzer(
* @see
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r174998505
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1192,11 +1195,23 @@ class Analyzer(
* @see
GitHub user kevinyu98 opened a pull request:
https://github.com/apache/spark/pull/20842
[SPARK-23162][PySpark][ML] Add r2adj into Python API in
LinearRegressionSummary
## What changes were proposed in this pull request?
Adding r2adj in LinearRegressionSummary for Python
GitHub user kevinyu98 opened a pull request:
https://github.com/apache/spark/pull/20795
[SPARK-23486]cache the function name from the catalog for lookupFunctions
## What changes were proposed in this pull request?
This PR will cache the function name from spark and external
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r179238708
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -1072,8 +1072,17 @@ class SessionCatalog
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r177637334
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -1072,8 +1072,17 @@ class SessionCatalog
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r177487877
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -1072,8 +1072,17 @@ class SessionCatalog
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20842#discussion_r177187425
--- Diff: python/pyspark/ml/regression.py ---
@@ -336,17 +336,31 @@ def rootMeanSquaredError(self):
@since("2.0.0")
d
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20842
@tengpeng Thanks, are you using ./dev/lint-python to run the python style
test locally?
---
-
To unsubscribe, e-mail
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r175159061
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1192,11 +1195,24 @@ class Analyzer(
* @see
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r175158413
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1192,11 +1195,24 @@ class Analyzer(
* @see
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r176249308
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala ---
@@ -175,6 +175,8 @@ private[sql] class HiveSessionCatalog
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r176249337
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
---
@@ -1076,6 +1076,16 @@ class SessionCatalog
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20795
@viirya yes, my latest submitted code only caching the external functions,
skip the built-in functions.
@WeichenXu123 I will change this comment only
[comment](https://github.com/apache/spark
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r175248943
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -1192,11 +1195,23 @@ class Analyzer(
* @see
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20795
@WeichenXu123 I didn't split until this disussion [discussion]
(https://github.com/apache/spark/pull/20795#discussion_r175156300). The
original jira report is about lookup HiveSessionCatalog, so
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r176276263
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala ---
@@ -175,6 +175,8 @@ private[sql] class HiveSessionCatalog
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r176308524
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala ---
@@ -175,6 +175,8 @@ private[sql] class HiveSessionCatalog
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/20795
the reason I was thinking to split is for the below scenario:
In order to avoid cache twice for the external function name in the cache
as the scenario described by Dilip, we decide to use
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20842#discussion_r176899545
--- Diff: python/pyspark/ml/regression.py ---
@@ -347,6 +347,20 @@ def r2(self):
"""
return self
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20842#discussion_r176899541
--- Diff: python/pyspark/ml/regression.py ---
@@ -347,6 +347,20 @@ def r2(self):
"""
return self
GitHub user kevinyu98 opened a pull request:
https://github.com/apache/spark/pull/22918
[SPARK-25902][SQL]Change AttributeReference.withMetadata's return type to
AttributeReference
## What changes were proposed in this pull request?
Currently
GitHub user kevinyu98 opened a pull request:
https://github.com/apache/spark/pull/22862
Spark 24207 2.4.0
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
(Please explain how
Github user kevinyu98 closed the pull request at:
https://github.com/apache/spark/pull/22862
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/22918
@xuanyuanking thanks, I fixed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r237691564
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
---
@@ -186,6 +186,54 @@ abstract class OrcSuite
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239266908
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
---
@@ -186,6 +186,82 @@ abstract class OrcSuite
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239266747
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala
---
@@ -190,4 +190,12 @@ class HiveOrcSourceSuite extends
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239266791
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
---
@@ -2370,4 +2370,51 @@ class HiveDDLSuite
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239266720
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
---
@@ -2370,4 +2370,51 @@ class HiveDDLSuite
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239266816
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
---
@@ -2370,4 +2370,51 @@ class HiveDDLSuite
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239266775
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
---
@@ -2370,4 +2370,51 @@ class HiveDDLSuite
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239266673
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
---
@@ -186,6 +186,54 @@ abstract class OrcSuite
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r238367919
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
---
@@ -186,6 +186,54 @@ abstract class OrcSuite
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/23108
Retest please, I didn't change Pyspark code, and tested
`--modules=pyspark-sql` in local machine, it is fine. Thanks
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239580991
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSourceSuite.scala
---
@@ -32,6 +32,7 @@ import org.apache.spark.util.Utils
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239577623
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala
---
@@ -73,9 +73,11 @@ class HiveOrcSourceSuite extends
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239635890
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala
---
@@ -190,4 +192,103 @@ class HiveOrcSourceSuite extends
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/23108
retest please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r239663711
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSourceSuite.scala
---
@@ -222,4 +223,66 @@ class HiveParquetSourceSuite extends
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r238469695
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
---
@@ -186,6 +186,54 @@ abstract class OrcSuite
GitHub user kevinyu98 opened a pull request:
https://github.com/apache/spark/pull/23108
[Spark-25993][SQL][TEST]Add test cases for resolution of ORC table location
## What changes were proposed in this pull request?
Add these test cases for resolution of ORC table location
Github user kevinyu98 commented on the issue:
https://github.com/apache/spark/pull/23108
I fixed a typo in the testcase, retest please. Thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r235790938
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala
---
@@ -597,6 +597,38 @@ abstract class
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r235790826
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -111,6 +111,8 @@ displayTitle: Spark SQL Upgrading Guide
- Since Spark 2.0, Spark
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r237272454
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
---
@@ -186,6 +186,54 @@ abstract class OrcSuite
Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23108#discussion_r237272654
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -111,6 +111,8 @@ displayTitle: Spark SQL Upgrading Guide
- Since Spark 2.0, Spark
301 - 372 of 372 matches
Mail list logo
