This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new c1434b1 [SPARK-36279][INFRA][PYTHON] Fix lint-python to work with
Python 3.9
c1434b1 is described below
commit c1434b1928a654c53a12060f680171a9a641ddc6
Author: Takuya UESHIN <[email protected]>
AuthorDate: Sat Jul 24 16:49:11 2021 +0900
[SPARK-36279][INFRA][PYTHON] Fix lint-python to work with Python 3.9
### What changes were proposed in this pull request?
Fix `lint-python` to pick `PYTHON_EXECUTABLE` from the environment variable
first to switch the Python and explicitly specify `PYTHON_EXECUTABLE` to use
`python3.9` in CI.
### Why are the changes needed?
Currently `lint-python` uses `python3`, but it's not the one we expect in
CI.
As a result, `black` check is not working.
```
The python3 -m black command was not found. Skipping black checks for now.
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The `black` check in `lint-python` should work.
Closes #33507 from ueshin/issues/SPARK-36279/lint-python.
Authored-by: Takuya UESHIN <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 663cbdfbe5da6fa4af969344a6281c90b711207e)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.github/workflows/build_and_test.yml | 2 +-
dev/lint-python | 2 +-
python/pyspark/pandas/tests/data_type_ops/test_num_ops.py | 8 ++------
3 files changed, 4 insertions(+), 8 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index 7a15172..73a45df 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -386,7 +386,7 @@ jobs:
- name: Java linter
run: ./dev/lint-java
- name: Python linter
- run: ./dev/lint-python
+ run: PYTHON_EXECUTABLE=python3.9 ./dev/lint-python
- name: R linter
run: ./dev/lint-r
- name: JS linter
diff --git a/dev/lint-python b/dev/lint-python
index 8650058e..e54e391 100755
--- a/dev/lint-python
+++ b/dev/lint-python
@@ -24,7 +24,7 @@ MYPY_BUILD="mypy"
PYCODESTYLE_BUILD="pycodestyle"
MINIMUM_PYCODESTYLE="2.7.0"
-PYTHON_EXECUTABLE="python3"
+PYTHON_EXECUTABLE="${PYTHON_EXECUTABLE:-python3}"
BLACK_BUILD="$PYTHON_EXECUTABLE -m black"
diff --git a/python/pyspark/pandas/tests/data_type_ops/test_num_ops.py
b/python/pyspark/pandas/tests/data_type_ops/test_num_ops.py
index b6f965a..fb2d2be 100644
--- a/python/pyspark/pandas/tests/data_type_ops/test_num_ops.py
+++ b/python/pyspark/pandas/tests/data_type_ops/test_num_ops.py
@@ -358,9 +358,7 @@ class NumOpsTest(PandasOnSparkTestCase, TestCasesUtils):
for col in self.numeric_df_cols:
pser, psser = pdf[col], psdf[col]
if isinstance(psser.spark.data_type, DecimalType):
- self.assertRaisesRegex(
- TypeError, "< can not be applied to", lambda: psser < psser
- )
+ self.assertRaisesRegex(TypeError, "< can not be applied to",
lambda: psser < psser)
else:
self.assert_eq(pser < pser, psser < psser)
@@ -380,9 +378,7 @@ class NumOpsTest(PandasOnSparkTestCase, TestCasesUtils):
for col in self.numeric_df_cols:
pser, psser = pdf[col], psdf[col]
if isinstance(psser.spark.data_type, DecimalType):
- self.assertRaisesRegex(
- TypeError, "> can not be applied to", lambda: psser > psser
- )
+ self.assertRaisesRegex(TypeError, "> can not be applied to",
lambda: psser > psser)
else:
self.assert_eq(pser > pser, psser > psser)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
