This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new a6869b25fb9a [SPARK-46801][PYTHON][TESTS] Do not treat exit code 5 as
a test failure in Python testing script
a6869b25fb9a is described below
commit a6869b25fb9a7ac0e7e5015d342435e5c1b5f044
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Mon Jan 22 17:06:59 2024 -0800
[SPARK-46801][PYTHON][TESTS] Do not treat exit code 5 as a test failure in
Python testing script
### What changes were proposed in this pull request?
This PR proposes to avoid treating the exit code 5 as a test failure in
Python testing script.
### Why are the changes needed?
```
...
========================================================================
Running PySpark tests
========================================================================
Running PySpark tests. Output is in /__w/spark/spark/python/unit-tests.log
Will test against the following Python executables: ['python3.12']
Will test the following Python modules: ['pyspark-core',
'pyspark-streaming', 'pyspark-errors']
python3.12 python_implementation is CPython
python3.12 version is: Python 3.12.1
Starting test(python3.12): pyspark.streaming.tests.test_context (temp
output:
/__w/spark/spark/python/target/8674ed86-36bd-47d1-863b-abb0405557f6/python3.12__pyspark.streaming.tests.test_context__umu69c3v.log)
Finished test(python3.12): pyspark.streaming.tests.test_context (12s)
Starting test(python3.12): pyspark.streaming.tests.test_dstream (temp
output:
/__w/spark/spark/python/target/847eb56b-3c5f-49ab-8a83-3326bb96bc5d/python3.12__pyspark.streaming.tests.test_dstream__rorhk0lc.log)
Finished test(python3.12): pyspark.streaming.tests.test_dstream (102s)
Starting test(python3.12): pyspark.streaming.tests.test_kinesis (temp
output:
/__w/spark/spark/python/target/78f23c83-c24d-4fa1-abbd-edb90f48dff1/python3.12__pyspark.streaming.tests.test_kinesis__q5l1pv0h.log)
test_kinesis_stream
(pyspark.streaming.tests.test_kinesis.KinesisStreamTests.test_kinesis_stream)
... skipped "Skipping all Kinesis Python tests as environmental variable
'ENABLE_KINESIS_TESTS' was not set."
test_kinesis_stream_api
(pyspark.streaming.tests.test_kinesis.KinesisStreamTests.test_kinesis_stream_api)
... skipped "Skipping all Kinesis Python tests as environmental variable
'ENABLE_KINESIS_TESTS' was not set."
----------------------------------------------------------------------
Ran 0 tests in 0.000s
NO TESTS RAN (skipped=2)
Had test failures in pyspark.streaming.tests.test_kinesis with python3.12;
see logs.
Error: running /__w/spark/spark/python/run-tests
--modules=pyspark-core,pyspark-streaming,pyspark-errors --parallelism=1
--python-executables=python3.12 ; received return code 255
Error: Process completed with exit code 19.
```
Scheduled job fails because of exit 5, see
https://github.com/pytest-dev/pytest/issues/2393. This isn't a test failure.
### Does this PR introduce _any_ user-facing change?
No, test-only.
### How was this patch tested?
Manually tested.
### Was this patch authored or co-authored using generative AI tooling?
No,
Closes #44841 from HyukjinKwon/SPARK-46801.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 52b62921cadb05da5b1183f979edf7d608256f2e)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
python/run-tests.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/python/run-tests.py b/python/run-tests.py
index 19e39c822cbb..b9031765d943 100755
--- a/python/run-tests.py
+++ b/python/run-tests.py
@@ -147,8 +147,8 @@ def run_individual_python_test(target_dir, test_name,
pyspark_python, keep_test_
# this code is invoked from a thread other than the main thread.
os._exit(1)
duration = time.time() - start_time
- # Exit on the first failure.
- if retcode != 0:
+ # Exit on the first failure but exclude the code 5 for no test ran, see
SPARK-46801.
+ if retcode != 0 and retcode != 5:
try:
with FAILURE_REPORTING_LOCK:
with open(LOG_FILE, 'ab') as log_file:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
