[
https://issues.apache.org/jira/browse/YUNIKORN-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788616#comment-17788616
]
Kuan-Po Tseng commented on YUNIKORN-2170:
-----------------------------------------
It seems like upgrading Spark to 3.3.3 is fast enough, I tried using github
actions to cache the tools directory to avoid downloading everything every time
when executing e2e. But the time spent on e2e is almost the same as the time
spent after upgrading Spark to 3.3.3.
> e2e test setup takes a lot of time
> ----------------------------------
>
> Key: YUNIKORN-2170
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2170
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: test - e2e
> Reporter: Peter Bacsko
> Assignee: Kuan-Po Tseng
> Priority: Major
> Labels: pull-request-available
>
> Setting up e2e test takes significant amount of time.
> For example:
> https://github.com/apache/yunikorn-k8shim/actions/runs/6927721705/job/18842345746?pr=730
> The whole run took 1h 18min.
> e2e tests: 34min.
> Downloading Spark: 40min.
> {noformat}
> Mon, 20 Nov 2023 09:06:35 GMT installing shellcheck v0.9.0
> Mon, 20 Nov 2023 09:06:35 GMT installing golangci-lint v1.54.2
> Mon, 20 Nov 2023 09:06:36 GMT installing kubectl v1.27.7
> Mon, 20 Nov 2023 09:06:36 GMT installing kind v0.20.0
> Mon, 20 Nov 2023 09:06:36 GMT installing helm v3.12.1
> Mon, 20 Nov 2023 09:06:37 GMT installing spark 3.3.1
> Mon, 20 Nov 2023 09:45:48 GMT Cloning into './build/yunikorn-release'...
> Mon, 20 Nov 2023 09:45:48 GMT step 3/7: building docker images from latest
> code
> {noformat}
> Options:
> * Use a different mirror if there's any or set up one
> * Take advantage of Github caching:
> https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
