This is an automated email from the ASF dual-hosted git repository.
Yicong-Huang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 035d510b029b [SPARK-58188][PYTHON] Remove unused
ArrowStreamUDTFSerializer
035d510b029b is described below
commit 035d510b029be1f08219f5e94585952a655073fd
Author: Yicong Huang <[email protected]>
AuthorDate: Fri Jul 17 16:42:41 2026 +0000
[SPARK-58188][PYTHON] Remove unused ArrowStreamUDTFSerializer
### What changes were proposed in this pull request?
Delete `ArrowStreamUDTFSerializer` from
`python/pyspark/sql/pandas/serializers.py`.
### Why are the changes needed?
`ArrowStreamUDTFSerializer` is no longer used after SPARK-57394 refactored
`SQL_ARROW_TABLE_UDF` to use `ArrowStreamSerializer` in `worker.py`. This class
can be safely deleted.
Part of SPARK-55384.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests: `pyspark.sql.tests.arrow.test_arrow_udtf`.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57330 from Yicong-Huang/SPARK-58188.
Authored-by: Yicong Huang <[email protected]>
Signed-off-by: Yicong-Huang <[email protected]>
---
python/pyspark/sql/pandas/serializers.py | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/python/pyspark/sql/pandas/serializers.py
b/python/pyspark/sql/pandas/serializers.py
index 13de009fb77c..890ce0d773e8 100644
--- a/python/pyspark/sql/pandas/serializers.py
+++ b/python/pyspark/sql/pandas/serializers.py
@@ -246,15 +246,6 @@ class ArrowStreamUDFSerializer(ArrowStreamSerializer):
return super().dump_stream(batches, stream)
-class ArrowStreamUDTFSerializer(ArrowStreamUDFSerializer):
- """
- Same as :class:`ArrowStreamUDFSerializer` but it does not flatten when
loading batches.
- """
-
- def load_stream(self, stream):
- return ArrowStreamSerializer.load_stream(self, stream)
-
-
class ArrowStreamPandasSerializer(ArrowStreamSerializer):
"""
Serializes pandas.Series as Arrow data with Arrow streaming format.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
