This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.1 by this push:
new e397c77 [SPARK-37288][PYTHON][3.2] Backport since annotation update
e397c77 is described below
commit e397c77a08ab27960ff9594bed88b9f16cc3f628
Author: zero323 <[email protected]>
AuthorDate: Sat Nov 13 23:53:24 2021 +0900
[SPARK-37288][PYTHON][3.2] Backport since annotation update
### What changes were proposed in this pull request?
This PR changes since annotation to support `float` arguments:
```python
def since(version: Union[str, float]) -> Callable[[T], T]: ...
```
### Why are the changes needed?
`since` is used with both `str` and `float` both in Spark and related
libraries and this change has been already done for Spark >= 3.3 (SPARK-36906),
Note that this technically fixes a bug in the downstream projects that run
mypy checks against `pyspark.since`. When they use it, for example, with
`pyspark.since(3.2)`, mypy checks fails; however, this case is legitimate.
After this change, the mypy check can pass in thier CIs.
### Does this PR introduce _any_ user-facing change?
```python
since(3.2)
def f():
...
```
is going to type check if downstream projects run mypy to validate the
types.
Otherwise, it does not affect anything invasive or user-facing behavior
change.
### How was this patch tested?
Existing tests and manual testing.
Closes #34555 from zero323/SPARK-37288.
Authored-by: zero323 <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit bdde69363f75f4f37370e38cddb47cd3631168b0)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/__init__.pyi | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/python/pyspark/__init__.pyi b/python/pyspark/__init__.pyi
index 852de35..8f5058c 100644
--- a/python/pyspark/__init__.pyi
+++ b/python/pyspark/__init__.pyi
@@ -16,7 +16,7 @@
# specific language governing permissions and limitations
# under the License.
-from typing import Callable, Optional, TypeVar
+from typing import Callable, Optional, TypeVar, Union
from pyspark.accumulators import ( # noqa: F401
Accumulator as Accumulator,
@@ -64,7 +64,7 @@ from pyspark.sql import ( # noqa: F401
T = TypeVar("T")
F = TypeVar("F", bound=Callable)
-def since(version: str) -> Callable[[T], T]: ...
+def since(version: Union[str, float]) -> Callable[[T], T]: ...
def copy_func(
f: F,
name: Optional[str] = ...,
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
