[
https://ovirt-jira.atlassian.net/browse/OVIRT-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barak Korren updated OVIRT-1984:
--------------------------------
Epic Link: OVIRT-2178 (was: OVIRT-400)
> Create "out-of-band" slave cleanup and setup jobs
> -------------------------------------------------
>
> Key: OVIRT-1984
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1984
> Project: oVirt - virtualization made easy
> Issue Type: New Feature
> Components: Jenkins Slaves
> Reporter: Barak Korren
> Assignee: infra
>
> Right now, we run slave cleaup and setup steps as part or every single job we
> run. This has several shortcomings:
> # It takes a long time from the point a user submitted a patch to the point
> his actual test or build code runs
> # If slave setup or cleanup steps fail - they fail the whole job for the user
> # If slave setup or cleanup steps fail - they can keep failing for many jobs
> until the CI team intervenes manually
> # There is a "chicken and an egg" issue where some parts of the CI code have
> to run before the slave was properly cleaned up and configured. This makes
> if harder to add new slaves for the system.
> Here is a suggested scheme to fix all this:
> # Label all slaves that should be cleaned up automatically as 'cleanable'.
> This is mostly to prevent the jobs described here from operating on the
> master node.
> # Have a "cleanup scheduler" job that finds all slaves labelled as
> "cleanable" but not as "dirty" or "clean", labels them as "dirty" and runs a
> cleanup job on them.
> # Have a "cleanup" job that is triggered on particular slaves by the "cleanup
> scheduler" job, runs cleaup and setup steps on them and then labels them as
> "clean" and removes the "dirty" label.
> # Have all other CI jobs only use slaves with the "clean" label.
> Notes:
> # The "dirty" label is there to make the "cleanup scheduler" job not trigger
> twice on the same slave before the"cleanup" job started cleaning it up.
> # Since all slaves used by the real jobs will always be clean - there will no
> longer be a need to run cleanup steps in the real jobs, thus saving time.
> # If cleanup steps fail - the cleanup job will fail and the slave will not be
> marked as "clean" so real jobs will never try to use it.
> # To solve the "chicken and egg" issue, the cleanup job probably must be a
> FreeStyle jobs and all the cleanup and setup code must be embedded into it by
> JJB. This will probably require a newer version of JJB then what we have so
> setting OVIRT-1983 as a blocker.
> # There is an issue of how to make CI for this - if cleanup and setup steps
> are removed from the normal STDCI jobs, they they will not be checked by the
> "check-patch" job of the "jenkins repo". Here is a suggested scheme to solve
> this:
> ## Have a way to "loan" slaves from the production jenkins to other Jenkins
> instances - this could be done by having a job that starts up the Jenkins
> JNLP client and tells it to connect to another Jenkins master.
> ## As part of the "check-patch" job for the 'jenkins' repo - start a Jenkins
> master in a container - attach some production slaves to it and have it run
> cleanup and setup steps on them
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100087)
_______________________________________________
Infra mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/[email protected]/message/Y3OR4S2H3CKQT24SAWDOKCYJEP43S2DN/
