This is an automated email from the ASF dual-hosted git repository.
gaogaotiantian pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new f3f61218af92 [SPARK-60144][PYTHON] Add type annotations to
LooseVersion comparisons
f3f61218af92 is described below
commit f3f61218af9254a80f67f727c882f32373999d36
Author: Tim Hsiung <[email protected]>
AuthorDate: Mon Nov 2 16:16:53 2026 -0800
[SPARK-60144][PYTHON] Add type annotations to LooseVersion comparisons
### What changes were proposed in this pull request?
Annotate `LooseVersion` comparison methods and `_cmp`, removing six
`no-untyped-def` suppressions. Keep the existing `Optional` style and use
`Union` to minimize the diff. Only `python/pyspark/loose_version.py` changes.
### Why are the changes needed?
Comparisons currently infer as `Any` rather than `bool`. The annotations
also let mypy reject unsupported ordering operands.
Two implementation adjustments are needed for the annotations to pass mypy:
- **`return NotImplemented` instead of `return c`:** mypy 1.19.1 does not
narrow `c` after `c is NotImplemented`. It still sees `Union[int,
NotImplementedType]`, which cannot be returned as `bool`. Returning the
sentinel directly avoids this limitation; typeshed permits it for
boolean-returning comparison methods.
- **Unconditional final `return 1`:** mypy cannot prove the `==`, `<`, and
`>` checks exhaustive. Keeping the final `if` produces a missing-return error
because the function could implicitly return `None`. After equality and
less-than are ruled out, greater-than is the remaining case for comparable
parsed components.
An upstream refresh does not solve this: [CPython removed distutils in
3.12](https://docs.python.org/3/library/distutils.html), and both the [3.11
source](https://github.com/python/cpython/blob/3.11/Lib/distutils/version.py)
and [setuptools
copy](https://github.com/pypa/setuptools/blob/main/setuptools/_distutils/version.py)
still leave these methods untyped.
### Does this PR introduce _any_ user-facing change?
Static comparisons infer as `bool`, and unsupported ordering operands are
rejected. No runtime behavior change is intended for parsed version strings.
### How was this patch tested?
- All 6 existing core typing cases in `python/pyspark/tests/typing` passed.
- Targeted mypy 1.19.1 check passed:
`python -m mypy --config-file python/mypy.ini --follow-imports=silent
python/pyspark/loose_version.py`
- Ruff lint/format and `git diff --check` passed.
No test files are changed. No full Spark build or JVM-backed tests were run.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: pi 1.1.0
Closes #59345 from bearomorphism/fix-loose-version-typing.
Authored-by: Tim Hsiung <[email protected]>
Signed-off-by: Tian Gao <[email protected]>
(cherry picked from commit 939a102b14de95abbee6661f3391249b1a9c5ff8)
Signed-off-by: Tian Gao <[email protected]>
---
python/pyspark/loose_version.py | 28 ++++++++++++++--------------
1 file changed, 14 insertions(+), 14 deletions(-)
diff --git a/python/pyspark/loose_version.py b/python/pyspark/loose_version.py
index 11c27f4ead8f..10db5228a480 100644
--- a/python/pyspark/loose_version.py
+++ b/python/pyspark/loose_version.py
@@ -4,7 +4,8 @@
# https://github.com/python/cpython/blob/3.11/Lib/distutils/version.py
import re
-from typing import Optional
+from types import NotImplementedType
+from typing import Optional, Union
class LooseVersion:
@@ -31,37 +32,37 @@ class LooseVersion:
def __repr__(self) -> str:
return "LooseVersion ('%s')" % str(self)
- def __eq__(self, other): # type: ignore[no-untyped-def]
+ def __eq__(self, other: object) -> bool:
c = self._cmp(other)
if c is NotImplemented:
- return c
+ return NotImplemented
return c == 0
- def __lt__(self, other): # type: ignore[no-untyped-def]
+ def __lt__(self, other: Union["LooseVersion", str]) -> bool:
c = self._cmp(other)
if c is NotImplemented:
- return c
+ return NotImplemented
return c < 0
- def __le__(self, other): # type: ignore[no-untyped-def]
+ def __le__(self, other: Union["LooseVersion", str]) -> bool:
c = self._cmp(other)
if c is NotImplemented:
- return c
+ return NotImplemented
return c <= 0
- def __gt__(self, other): # type: ignore[no-untyped-def]
+ def __gt__(self, other: Union["LooseVersion", str]) -> bool:
c = self._cmp(other)
if c is NotImplemented:
- return c
+ return NotImplemented
return c > 0
- def __ge__(self, other): # type: ignore[no-untyped-def]
+ def __ge__(self, other: Union["LooseVersion", str]) -> bool:
c = self._cmp(other)
if c is NotImplemented:
- return c
+ return NotImplemented
return c >= 0
- def _cmp(self, other): # type: ignore[no-untyped-def]
+ def _cmp(self, other: object) -> Union[int, NotImplementedType]:
if isinstance(other, str):
other = LooseVersion(other)
elif not isinstance(other, LooseVersion):
@@ -71,5 +72,4 @@ class LooseVersion:
return 0
if self.version < other.version:
return -1
- if self.version > other.version:
- return 1
+ return 1
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]