This is an automated email from the ASF dual-hosted git repository.

HeartSaVioR pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.1 by this push:
     new 517c5e932381 [SPARK-60609][SS][PYTHON][FOLLOWUP] Fix simple stream 
reader test on branch-4.1
517c5e932381 is described below

commit 517c5e93238165265f02dc074505b5167835595c
Author: Jungtaek Lim <[email protected]>
AuthorDate: Fri Oct 30 11:38:37 2026 +0900

    [SPARK-60609][SS][PYTHON][FOLLOWUP] Fix simple stream reader test on 
branch-4.1
    
    ### What changes were proposed in this pull request?
    
    Use `latestOffset()` in the list-reader test on branch-4.1.
    
    ### Why are the changes needed?
    
    The #59855 backport copied calls to a newer API, including undefined 
`ReadAllAvailable`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Five Python-only datasource tests passed. Full JVM suite not run.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Yes. Generated-by: OpenAI Codex 0.161.0.
    
    Closes #59948 from 
HeartSaVioR/SPARK-60609-fix-branch-4.1-stream-reader-test.
    
    Authored-by: Jungtaek Lim <[email protected]>
    Signed-off-by: Jungtaek Lim <[email protected]>
---
 python/pyspark/sql/tests/test_python_streaming_datasource.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/python/pyspark/sql/tests/test_python_streaming_datasource.py 
b/python/pyspark/sql/tests/test_python_streaming_datasource.py
index 9298f95245c8..80a4aeb22630 100644
--- a/python/pyspark/sql/tests/test_python_streaming_datasource.py
+++ b/python/pyspark/sql/tests/test_python_streaming_datasource.py
@@ -333,20 +333,20 @@ class BasePythonStreamingDataSourceTestsMixin:
 
         # Non-empty list with an advanced end offset is cached and readable.
         wrapper = _SimpleStreamReaderWrapper(ListReader([(0,), (1,)], 
advance=True))
-        end = wrapper.latestOffset(start, ReadAllAvailable())
+        end = wrapper.latestOffset()
         self.assertEqual(end, {"offset": 2})
         self.assertEqual(list(wrapper.getCache(start, end)), [(0,), (1,)])
 
         # Empty list with end == start is allowed and not cached.
         wrapper = _SimpleStreamReaderWrapper(ListReader([], advance=False))
-        end = wrapper.latestOffset(start, ReadAllAvailable())
+        end = wrapper.latestOffset()
         self.assertEqual(end, start)
         self.assertEqual(len(wrapper.cache), 0)
 
         # Non-empty list with end == start raises the offset validation error.
         wrapper = _SimpleStreamReaderWrapper(ListReader([(0,)], advance=False))
         with self.assertRaises(PySparkException) as cm:
-            wrapper.latestOffset(start, ReadAllAvailable())
+            wrapper.latestOffset()
         self.assertEqual(
             cm.exception.getCondition(),
             "SIMPLE_STREAM_READER_OFFSET_DID_NOT_ADVANCE",


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to