This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 13b2856e6e7 [SPARK-41864][INFRA][PYTHON] Fix mypy linter errors
13b2856e6e7 is described below
commit 13b2856e6e77392a417d2bb2ce804f873ee72b28
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Jan 3 15:00:50 2023 -0800
[SPARK-41864][INFRA][PYTHON] Fix mypy linter errors
### What changes were proposed in this pull request?
Currently, the GitHub Action Python linter job is broken. This PR will
recover Python linter failure.
### Why are the changes needed?
There are two kind of failures.
1. https://github.com/apache/spark/actions/runs/3829330032/jobs/6524170799
```
python/pyspark/pandas/sql_processor.py:221: error: unused "type: ignore"
comment
Found 1 error in 1 file (checked 380 source files)
```
2. After fixing (1), we hit the following.
```
ModuleNotFoundError: No module named 'py._path'; 'py' is not a package
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the GitHub CI on this PR. Or, manually run the following.
```
$ dev/lint-python
starting python compilation test...
python compilation succeeded.
starting black test...
black checks passed.
starting flake8 test...
flake8 checks passed.
starting mypy annotations test...
annotations passed mypy checks.
starting mypy examples test...
examples passed mypy checks.
starting mypy data test...
annotations passed data checks.
all lint-python tests passed!
```
Closes #39373 from dongjoon-hyun/SPARK-41864.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/requirements.txt | 1 +
python/pyspark/pandas/sql_processor.py | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/dev/requirements.txt b/dev/requirements.txt
index c3911b57eb9..1d978c4602c 100644
--- a/dev/requirements.txt
+++ b/dev/requirements.txt
@@ -47,6 +47,7 @@ PyGithub
# pandas API on Spark Code formatter.
black==22.6.0
+py
# Spark Connect (required)
grpcio==1.48.1
diff --git a/python/pyspark/pandas/sql_processor.py
b/python/pyspark/pandas/sql_processor.py
index ec6b0498511..28e2329b8f9 100644
--- a/python/pyspark/pandas/sql_processor.py
+++ b/python/pyspark/pandas/sql_processor.py
@@ -218,7 +218,7 @@ def _get_ipython_scope() -> Dict[str, Any]:
in an IPython notebook environment.
"""
try:
- from IPython import get_ipython # type: ignore[import]
+ from IPython import get_ipython
shell = get_ipython()
return shell.user_ns
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
