[
https://issues.apache.org/jira/browse/SPARK-32244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-32244:
---------------------------------
Description:
Last week and onwards, the Jenkins machines became very unstable for some
reasons.
- Apparently, the machines became extremely slow. Almost all tests can't pass.
- One machine (worker 4) started to have the corrupt .m2 which fails the
build.
- Documentation build fails time to time for an unknown reason in Jenkins
machine specifically.
Almost all PRs are basically blocked by this instability currently.
This JIRA aims to run the tests in Github Actions.
- To avoid depending on few persons who can access to the cluster.
- To reduce the elapsed time in the build - we could split the tests (e.g.,
SQL, ML, CORE), and run them in parallel so the total build time will
significantly reduce.
- To control the environment more flexibly.
- Other contributors can test and propose to fix Github Actions
configurations so we can distribute this build management cost.
was:
Last week and onwards, the Jenkins machines became very unstable for some
reasons.
- Apparently, the machines became extremely slow. Almost all tests can't pass.
- One machine (worker 4) started to have the corrupt .m2 which fails the
build.
- Documentation build fails time to time for an unknown reason in Jenkins
machine specifically.
Almost all PRs are basically blocked by this instability currently.
This JIRA aims to run the tests in Github Actions.
- To avoid depending on few persons who can access to the cluster.
- To reduce the elapsed time in the build - we could split the tests (e.g.,
SQL, ML, CORE), and run them in parallel so the total build time will
significantly reduce.
- To control the environment more flexibly. Currently, we should keep asking
Shane to install or control something.
- Other contributors can test and propose to fix Github Actions
configurations so we can distribute this build management cost.
> Build and run the Spark with test cases in Github Actions
> ---------------------------------------------------------
>
> Key: SPARK-32244
> URL: https://issues.apache.org/jira/browse/SPARK-32244
> Project: Spark
> Issue Type: Umbrella
> Components: Project Infra
> Affects Versions: 2.4.6, 3.0.0, 3.1.0
> Reporter: Hyukjin Kwon
> Priority: Critical
>
> Last week and onwards, the Jenkins machines became very unstable for some
> reasons.
> - Apparently, the machines became extremely slow. Almost all tests can't
> pass.
> - One machine (worker 4) started to have the corrupt .m2 which fails the
> build.
> - Documentation build fails time to time for an unknown reason in Jenkins
> machine specifically.
> Almost all PRs are basically blocked by this instability currently.
> This JIRA aims to run the tests in Github Actions.
> - To avoid depending on few persons who can access to the cluster.
> - To reduce the elapsed time in the build - we could split the tests (e.g.,
> SQL, ML, CORE), and run them in parallel so the total build time will
> significantly reduce.
> - To control the environment more flexibly.
> - Other contributors can test and propose to fix Github Actions
> configurations so we can distribute this build management cost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
