This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
from 5432cef483c4 [SPARK-50152][SS] Support handleInitialState with state
data source reader
add 0a66c371cd4c [SPARK-50130][SQL][PYTHON] Add DataFrame APIs for scalar
and exists subqueries
No new revisions were added by this update.
Summary of changes:
.../src/main/resources/error/error-conditions.json | 6 +
.../main/scala/org/apache/spark/sql/Dataset.scala | 13 +
dev/sparktestsupport/modules.py | 2 +
python/pyspark/sql/classic/column.py | 4 +
python/pyspark/sql/classic/dataframe.py | 9 +
python/pyspark/sql/column.py | 14 +
python/pyspark/sql/connect/column.py | 14 +-
python/pyspark/sql/connect/dataframe.py | 18 +
python/pyspark/sql/dataframe.py | 161 +++++++
...{test_parity_tvf.py => test_parity_subquery.py} | 8 +-
python/pyspark/sql/tests/test_subquery.py | 487 +++++++++++++++++++++
.../main/scala/org/apache/spark/sql/Column.scala | 17 +-
.../scala/org/apache/spark/sql/api/Dataset.scala | 27 ++
.../apache/spark/sql/internal/columnNodes.scala | 18 +
.../spark/sql/catalyst/analysis/Analyzer.scala | 24 +-
.../sql/catalyst/analysis/CheckAnalysis.scala | 21 +-
.../catalyst/analysis/ColumnResolutionHelper.scala | 41 +-
.../spark/sql/catalyst/analysis/unresolved.scala | 40 ++
.../sql/catalyst/expressions/Expression.scala | 16 +-
.../spark/sql/catalyst/expressions/subquery.scala | 15 +-
.../spark/sql/catalyst/optimizer/Optimizer.scala | 2 +-
.../spark/sql/catalyst/optimizer/expressions.scala | 2 +-
.../spark/sql/catalyst/optimizer/subquery.scala | 15 +-
.../spark/sql/catalyst/trees/TreePatterns.scala | 5 +
.../main/scala/org/apache/spark/sql/Dataset.scala | 67 ++-
.../spark/sql/execution/QueryExecution.scala | 3 +
.../adaptive/PlanAdaptiveSubqueries.scala | 2 +-
.../spark/sql/internal/columnNodeSupport.scala | 13 +
.../apache/spark/sql/DataFrameSubquerySuite.scala | 367 ++++++++++++++++
29 files changed, 1370 insertions(+), 61 deletions(-)
copy python/pyspark/sql/tests/connect/{test_parity_tvf.py =>
test_parity_subquery.py} (80%)
create mode 100644 python/pyspark/sql/tests/test_subquery.py
create mode 100644
sql/core/src/test/scala/org/apache/spark/sql/DataFrameSubquerySuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
