This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 977e445865c [SPARK-41864][INFRA][PYTHON] Fix mypy linter errors
977e445865c is described below
commit 977e445865c0b835b19db44a129502c135b3348a
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Jan 3 15:00:50 2023 -0800
[SPARK-41864][INFRA][PYTHON] Fix mypy linter errors
Currently, the GitHub Action Python linter job is broken. This PR will
recover Python linter failure.
There are two kind of failures.
1. https://github.com/apache/spark/actions/runs/3829330032/jobs/6524170799
```
python/pyspark/pandas/sql_processor.py:221: error: unused "type: ignore"
comment
Found 1 error in 1 file (checked 380 source files)
```
2. After fixing (1), we hit the following.
```
ModuleNotFoundError: No module named 'py._path'; 'py' is not a package
```
No.
Pass the GitHub CI on this PR. Or, manually run the following.
```
$ dev/lint-python
starting python compilation test...
python compilation succeeded.
starting black test...
black checks passed.
starting flake8 test...
flake8 checks passed.
starting mypy annotations test...
annotations passed mypy checks.
starting mypy examples test...
examples passed mypy checks.
starting mypy data test...
annotations passed data checks.
all lint-python tests passed!
```
Closes #39373 from dongjoon-hyun/SPARK-41864.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 13b2856e6e77392a417d2bb2ce804f873ee72b28)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/requirements.txt | 1 +
python/pyspark/pandas/sql_processor.py | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/dev/requirements.txt b/dev/requirements.txt
index e7e0a4b4274..79a70624312 100644
--- a/dev/requirements.txt
+++ b/dev/requirements.txt
@@ -43,3 +43,4 @@ PyGithub
# pandas API on Spark Code formatter.
black
+py
diff --git a/python/pyspark/pandas/sql_processor.py
b/python/pyspark/pandas/sql_processor.py
index d8ae6888b68..7cf2f7461ba 100644
--- a/python/pyspark/pandas/sql_processor.py
+++ b/python/pyspark/pandas/sql_processor.py
@@ -218,7 +218,7 @@ def _get_ipython_scope() -> Dict[str, Any]:
in an IPython notebook environment.
"""
try:
- from IPython import get_ipython # type: ignore[import]
+ from IPython import get_ipython
shell = get_ipython()
return shell.user_ns
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
