Aaron Fabbri created HADOOP-19895:
-------------------------------------
Summary: ci: shared, secure maven cache to speed up builds
Key: HADOOP-19895
URL: https://issues.apache.org/jira/browse/HADOOP-19895
Project: Hadoop Common
Issue Type: Sub-task
Components: ci
Reporter: Aaron Fabbri
In the quest for fast & efficient CI testing, we want to provide builds with
warm maven caches wherever we can.
The biggest concern here is security: Allowing arbitrary PRs to populate a
build cache that will be used by other workflows or release builds is
dangerous, since can poison the cache by injecting malicious code. Github
partially addresses this by not allowing different PR branches to see each
others' caches. Branches can access the cache of parent branches, though, so if
artifacts are cached for `trunk`, any branches off of it can read those
artifacts.
Some initial conversations on this topic are in the PR discussion here:
[https://github.com/apache/hadoop/pull/8467#discussion_r3228620488]
Initial approach:
1. On push to trunk, run a new workflow `maven_cache_refresh.yml` which
downloads all external (non-hadoop) dependencies, and then saves the cache.
2. Modify other CI workflows to take advantage of this shared trunk cache.
3. Validate the cache is working as expected. Test with fork and non-fork PRs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
