This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 91910d3af53 [SPARK-42171][PYSPARK][TESTS] Fix `pyspark-errors` module
and enable it in GitHub Action
91910d3af53 is described below
commit 91910d3af537e8104f1d3bf956bb30f18a7c5fe1
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Jan 24 13:08:14 2023 -0800
[SPARK-42171][PYSPARK][TESTS] Fix `pyspark-errors` module and enable it in
GitHub Action
### What changes were proposed in this pull request?
This PR aims to enable `pyspark-errors` module test coverall in GitHub
Action and fix it.
### Why are the changes needed?
#39387 added `pyspark_errors` module, but didn't enable it in GitHub Action.
The bug was not exposed in #39387 initially because `ERROR_CLASSES_JSON`
has only one item in this PR.
https://github.com/apache/spark/blob/8b8d92334faee889d4106a665ce769403e098240/python/pyspark/errors/tests/test_errors.py#L31
The bug is exposed in the subsequent PRs. So, currently, it's broken.
```
$ python/run-tests.py --modules pyspark-errors
...
test_error_classes
(pyspark.errors.tests.test_errors.ErrorsTest.test_error_classes) ... ERROR
======================================================================
ERROR: test_error_classes
(pyspark.errors.tests.test_errors.ErrorsTest.test_error_classes)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Users/dongjoon/APACHE/spark-merge/python/pyspark/errors/tests/test_errors.py",
line 31, in test_error_classes
error_class_names[i] < error_class_names[i + 1],
~~~~~~~~~~~~~~~~~^^^
KeyError: 0
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
This PR shows the failure at the first commit by adding `pyspark-errors`
test coverage.
-
https://github.com/dongjoon-hyun/spark/actions/runs/3999180114/jobs/6862791939
The second commit passed `pyspark-errors` in the CIs.
-
https://github.com/dongjoon-hyun/spark/actions/runs/3999260298/jobs/6862968301
Closes #39721 from dongjoon-hyun/SPARK-42171.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/build_and_test.yml | 2 +-
python/pyspark/errors/tests/test_errors.py | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index 07c5ddf5c73..e4397554303 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -337,7 +337,7 @@ jobs:
- >-
pyspark-pandas-slow
- >-
- pyspark-connect
+ pyspark-connect, pyspark-errors
env:
MODULES_TO_TEST: ${{ matrix.modules }}
HADOOP_PROFILE: ${{ inputs.hadoop }}
diff --git a/python/pyspark/errors/tests/test_errors.py
b/python/pyspark/errors/tests/test_errors.py
index cd2a8a4a22c..833edcf9f42 100644
--- a/python/pyspark/errors/tests/test_errors.py
+++ b/python/pyspark/errors/tests/test_errors.py
@@ -25,7 +25,7 @@ class ErrorsTest(unittest.TestCase):
def test_error_classes(self):
# Test error classes is sorted alphabetically
error_reader = ErrorClassesReader()
- error_class_names = error_reader.error_info_map
+ error_class_names = list(error_reader.error_info_map.keys())
for i in range(len(error_class_names) - 1):
self.assertTrue(
error_class_names[i] < error_class_names[i + 1],
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
