[
https://issues.jenkins-ci.org/browse/JENKINS-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=160404#comment-160404
]
Ed Randall commented on JENKINS-10520:
--------------------------------------
This is our build pipeline:
1) Compile - checks files out of Perforce, cleans, configures for the target
architecture and compiles. Archives the workspace for susequent steps in the
pipeline.
2) Zip - builds the .ear file, verifies it, builds a complete shippable .zip of
all project deliverables.
3) Quick test - runs a relatively short test suite.
2a) Zip is "promoted" if all tests in QuickTest pass. This delivers the Zip to
the System test team (outside of Jenkins).
Now we didn't want each step to sync the files down from Perforce and compile
afresh, the clone-workspace plugin seemed ideal for us. Unfortunately there are
a number of problems with it including:
a) Only the latest workspace from step(1) is maintained. So if another change
triggers a Compile whilst Zip is in progress, it could be that (3) Quick test
actually tests that subsequent build and not the one it is supposed to be
testing. This has regularly caused us considerable confusion as the jobs were
out-of-step.
b) It takes so long ... our project is somewhat monolithic, including all
binaries required within the workspace. The workspace.zip archive is about
1.5Gb and creating it takes around 50 minutes using clone-workspace-plugin vs.
only 10 minutes for the actual compile, that's unacceptable. Un-zipping at the
start of the steps (2) and (3) is not quite so bad but still 10-20 minutes. It
seems to be something to do with the way data is piped between master and slave.
The solution we now have working is to replace this plugin by a shell script.
This script takes 2 minutes to archive the workspace at the end of job (1) and
2 minutes at to un-archive it at the start of the other steps. It is attached
to this reportb for reference, to illustrate our use case which we would like
this plugin to support.
Use the script as follows:
At the end of job (1) invoke the script using "Execute Shell":
${JENKINS_BIN}/clone-workspace PACK "${BUILD_TAG}"
Optional INCLUDE= and EXCLUDE= filters can be used to limit the contents of the
archive. We actually create 2 archives, one is much smaller containing just
the information required for the "promote" step (2a) to work.
Also add a Post-Build action "Trigger parameterized build on other projects" to
start job (2) using the parameter:
UPSTREAM_BUILD_TAG=${BUILD_TAG}
Jobs (2) and (3) are parameterized, accepting UPSTREAM_BUILD_TAG. The first
task that each of these downstream jobs runs is:
Execute Shell
${JENKINS_BIN}/clone-workspace UNPACK "${UPSTREAM_BUILD_TAG}"
The archive dir must exist on the master node and be referenced by the
JOBS_ARCHIVE environment variable.
The last job in the pipeline removes the redundant archive by invoking:
${JENKINS_BIN}/clone-workspace CLEAN "${UPSTREAM_BUILD_TAG}"
We also purge all archives from it overnight using a "cron" job, just to be
sure.
Now the downstream jobs don't show the change history; that was solved by
re-instating the clone-workspace-plugin but configured to archive just one file.
> clone-workspace-scm performance is poor
> ---------------------------------------
>
> Key: JENKINS-10520
> URL: https://issues.jenkins-ci.org/browse/JENKINS-10520
> Project: Jenkins
> Issue Type: Improvement
> Components: clone-workspace
> Affects Versions: current
> Environment: Solaris/x86, cluster of 5 nodes, all machines in cluster
> are identical
> Reporter: Ed Randall
> Assignee: abayer
> Attachments: clone-workspace
>
>
> Clone workspace performance is poor. Whilst compilation takes around 4
> minutes, archiving the workspace afterwards takes the total job time out to
> 35 minutes! Similarly un-archiving it at the start of downstream jobs.
> We have 4 steps, Compile, quick test, package, full test.
> Compile uses Perforce SCM, we use clone workspace after that to ensure
> downstream builds use identical files but other compiles can continue in
> parallel. So we need almost the files in the entire workspace.
> The filesystem workspace size is about 1.6Gb
> The archived workspace.zip file size is about 1.4Gb
> An "exclude" filter may help a little but I think there is something slow
> going on.
> Note that we use slaves as well so the piped data may have an impact, but all
> the machines are very close together.
> When the compile runs on the master node it doesn't seem any quicker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
