This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 15dc67e [SPARK-33123][INFRA] Ignore GitHub only changes in Amplab
Jenkins build
15dc67e is described below
commit 15dc67eb10339792bbf03a4717eefe0d95acb39b
Author: William Hyun <[email protected]>
AuthorDate: Mon Oct 19 14:13:37 2020 +0900
[SPARK-33123][INFRA] Ignore GitHub only changes in Amplab Jenkins build
### What changes were proposed in this pull request?
This PR aims to ignore GitHub only changes in Amplab Jenkins build.
### Why are the changes needed?
This will save server resources.
### Does this PR introduce _any_ user-facing change?
No, this is a dev-only change.
### How was this patch tested?
Manually. I used the following doctest during testing and removed it at the
clean-up.
E2E tests:
```
cd dev
cat test.py
```
```python
import importlib
runtests = importlib.import_module("run-tests")
print([x.name for x in
runtests.determine_modules_for_files([".github/workflows/build_and_test.yml"])])
```
```python
$ GITHUB_ACTIONS=1 python test.py
['root']
$ python test.py
[]
```
Unittests:
```bash
$ GITHUN_ACTIONS=1 python3 -m doctest dev/run-tests.py
$ python3 -m doctest dev/run-tests.py
```
Closes #30020 from williamhyun/SPARK-33123.
Lead-authored-by: William Hyun <[email protected]>
Co-authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
(cherry picked from commit e6c53c2c1b538d6272df4d1ca294d04f8b49bd6c)
Signed-off-by: HyukjinKwon <[email protected]>
---
dev/run-tests.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/dev/run-tests.py b/dev/run-tests.py
index 58acffa..b726b0c 100755
--- a/dev/run-tests.py
+++ b/dev/run-tests.py
@@ -42,7 +42,8 @@ def determine_modules_for_files(filenames):
"""
Given a list of filenames, return the set of modules that contain those
files.
If a file is not associated with a more specific submodule, then this
method will consider that
- file to belong to the 'root' module. GitHub Action and Appveyor files are
ignored.
+ file to belong to the 'root' module. `.github` directory is counted only
in GitHub Actions,
+ and `appveyor.yml` is always ignored because this file is dedicated only
to AppVeyor builds.
>>> sorted(x.name for x in
determine_modules_for_files(["python/pyspark/a.py", "sql/core/foo"]))
['pyspark-core', 'sql']
@@ -55,6 +56,8 @@ def determine_modules_for_files(filenames):
for filename in filenames:
if filename in ("appveyor.yml",):
continue
+ if ("GITHUB_ACTIONS" not in os.environ) and
filename.startswith(".github"):
+ continue
matched_at_least_one_module = False
for module in modules.all_modules:
if module.contains_file(filename):
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
