nchammas commented on code in PR #56312:
URL: https://github.com/apache/spark/pull/56312#discussion_r3815260271
##########
dev/sparktestsupport/utils.py:
##########
@@ -33,30 +33,15 @@ def determine_modules_for_files(filenames):
"""
Given a list of filenames, return the set of modules that contain those
files.
If a file is not associated with a more specific submodule, then this
method will consider that
- file to belong to the 'root' module. `.github` directory is counted only
in GitHub Actions,
- and `README.md`, `AGENTS.md`, `CONTRIBUTING.md` are always ignored.
+ file to belong to the 'root' module. `.github` directory is counted only
in GitHub Actions.
>>> sorted(x.name for x in
determine_modules_for_files(["python/pyspark/a.py", "sql/core/foo"]))
['pyspark-core', 'pyspark-errors', 'sql']
>>> [x.name for x in
determine_modules_for_files(["file_not_matched_by_any_subproject"])]
['root']
- >>> [x.name for x in determine_modules_for_files(["sql/README.md"])]
- []
- >>> [x.name for x in determine_modules_for_files(["AGENTS.md"])]
- []
- >>> [x.name for x in determine_modules_for_files(["CONTRIBUTING.md"])]
- []
"""
changed_modules = set()
for filename in filenames:
- if filename.endswith(("README.md", "AGENTS.md", "CONTRIBUTING.md")):
- continue
- if filename in (
- "scalastyle-config.xml",
- "dev/checkstyle.xml",
- "dev/checkstyle-suppressions.xml",
- ):
- continue
Review Comment:
I don't think this change had quite the intended effect. README-only changes
still trigger unnecessary CI.
https://github.com/apache/spark/commit/75ac74bafec1f489224765041a19bec6bad36a9d,
for example, built `pyspark-install` and `pyspark-core`. That's because this
method now returns `dev-tools` _in addition_ to those modules, not instead of
them.
I will submit a patch for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
