This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
from a3aa08697686 [SPARK-47358][SQL][COLLATION] Improve repeat expression
support to return correct datatype
add a127df47f7a2 [SPARK-47367][PYTHON][CONNECT] Support Python data
sources with Spark Connect
No new revisions were added by this update.
Summary of changes:
.../src/main/protobuf/spark/connect/commands.proto | 1 +
.../main/protobuf/spark/connect/relations.proto | 19 ++
.../sql/connect/planner/SparkConnectPlanner.scala | 31 +++
dev/sparktestsupport/modules.py | 1 +
python/pyspark/sql/connect/client/core.py | 21 ++
.../__init__.py => sql/connect/datasource.py} | 53 ++--
python/pyspark/sql/connect/plan.py | 39 +++
python/pyspark/sql/connect/proto/commands_pb2.py | 180 ++++++-------
python/pyspark/sql/connect/proto/commands_pb2.pyi | 12 +
python/pyspark/sql/connect/proto/relations_pb2.py | 292 +++++++++++----------
python/pyspark/sql/connect/proto/relations_pb2.pyi | 72 +++++
python/pyspark/sql/connect/session.py | 9 +
...foreach.py => test_parity_python_datasource.py} | 9 +-
python/pyspark/sql/tests/test_python_datasource.py | 2 -
14 files changed, 474 insertions(+), 267 deletions(-)
copy python/pyspark/{ml/connect/__init__.py => sql/connect/datasource.py} (55%)
copy python/pyspark/sql/tests/connect/{streaming/test_parity_foreach.py =>
test_parity_python_datasource.py} (82%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
