This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new c98ade32386 [SPARK-44833][CONNECT][PYTHON][FOLLOW-UP] Fix the type
annotation for iterator at reattach.py
c98ade32386 is described below
commit c98ade32386ff396c706cab7ece4d287e4b58453
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Wed Sep 6 16:48:54 2023 +0900
[SPARK-44833][CONNECT][PYTHON][FOLLOW-UP] Fix the type annotation for
iterator at reattach.py
### What changes were proposed in this pull request?
This PR fixes the type hint of `self._iterator` at `reattach.py` to be
`Optional` because `None` is assigned here.
### Why are the changes needed?
To fix the CI:
```
starting mypy annotations test...
annotations failed mypy checks:
python/pyspark/sql/connect/client/reattach.py:149: error: Incompatible
types in assignment (expression has type "None", variable has type
"Iterator[ExecutePlanResponse]") [assignment]
python/pyspark/sql/connect/client/reattach.py:254: error: Incompatible
types in assignment (expression has type "None", variable has type
"Iterator[ExecutePlanResponse]") [assignment]
python/pyspark/sql/connect/client/reattach.py:[25](https://github.com/apache/spark/actions/runs/6093065151/job/16532169212#step:19:26)8:
error: Incompatible types in assignment (expression has type "None", variable
has type "Iterator[ExecutePlanResponse]") [assignment]
Found 3 errors in 1 file (checked 703 source files)
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually ran the script of `./dev/lint-python`
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #42830 from HyukjinKwon/SPARK-44833-followup.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 48872d4b3938fd82c4d753549787f6e7c62cce7e)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/sql/connect/client/reattach.py | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/python/pyspark/sql/connect/client/reattach.py
b/python/pyspark/sql/connect/client/reattach.py
index d3765fb6696..7e1e722d5fd 100644
--- a/python/pyspark/sql/connect/client/reattach.py
+++ b/python/pyspark/sql/connect/client/reattach.py
@@ -94,7 +94,7 @@ class ExecutePlanResponseReattachableIterator(Generator):
# Note: This is not retried, because no error would ever be thrown
here, and GRPC will only
# throw error on first self._has_next().
self._metadata = metadata
- self._iterator: Iterator[pb2.ExecutePlanResponse] = iter(
+ self._iterator: Optional[Iterator[pb2.ExecutePlanResponse]] = iter(
self._stub.ExecutePlan(self._initial_request, metadata=metadata)
)
@@ -133,7 +133,9 @@ class ExecutePlanResponseReattachableIterator(Generator):
with attempt:
if self._current is None:
try:
- self._current = self._call_iter(lambda:
next(self._iterator))
+ self._current = self._call_iter(
+ lambda: next(self._iterator) # type:
ignore[arg-type]
+ )
except StopIteration:
pass
@@ -150,7 +152,9 @@ class ExecutePlanResponseReattachableIterator(Generator):
# shouldn't change
assert not self._result_complete
try:
- self._current = self._call_iter(lambda:
next(self._iterator))
+ self._current = self._call_iter(
+ lambda: next(self._iterator) # type:
ignore[arg-type]
+ )
except StopIteration:
pass
has_next = self._current is not None
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
