Not sure how have the exact same "checkout" multiple times is useful (most of my jobs modify the workspace so I need to isolate them from each other -> separate workspace == separate "checkouts").
OTOH, I was also interested to save on disk space when it comes to large git repos cloned multiple times. Note that I mean "clone" not "checkout". So, an easy/safe enough solution to reduce waste of duplicate clones: 1) have 1 proper clone of the big repo on each slave - just by create a dumb "mirror" job with no build steps, just to keep the clone up to date 2) the actual jobs use the "Reference" repo option: although the main clone location can be still be the central server, you point to that workspace of the mirror job as a reference (see --reference https://www.kernel.org/pub/software/scm/git/docs/git-clone.html, and the Advanced git plugin option). My results using this is quite dramatic, the repo (.git) sizes under the worker jobs are just a few Mb, even the actually repo is 1.5G+ in my case. Also the "clone" for each worker job will just take a fraction of time. Now I'd like Jenkins (plugin?) to provide a setup like this out-of-the-box, eliminating the manual fiddling (I needed to add site-wide variables overridden for each slave, etc..). Anyway it's good enough for me now. Of course the actual "checkout" sizes are the same, but I see no safe&straightforward way around that, that's just the nature of building/testing code. So, not sure if this helps but good luck anyway. Greg On 16 December 2013 17:06, <[email protected]> wrote: > Has anyone considered a modification to the git plugin (or other SCM > plugins), to reduce the total number of checkouts physically required on a > slave machine? > > We run 6 mult-configuration projects (debug and release configurations) on > a handful of computers. Each configuration requires one code checkout (2.1 > G). That's 2.1*2*6 = 25.2 G on each slave machine, for the source code > only. The master build machine requires another 12.6 G, because it checks > out additional copies *outside* the scope of build directories (no one > knows why). Forget about leaving "Advanced Project Options" > "Restrict > where this project can be run" unchecked, because if the build is started > on a different computer each time, the code has to be checked out at the > top level (potentially adding the 12.6 G to each slave machine, if the load > is shared). > > Perhaps these issues have already been addressed; we run Jenkins version > 1.524. > > Suggestions: > a) Live with redundant code checkouts > b) Stop using the git plugin, and have my shell script handle code checkout > c) Rewrite the git plugin, or add an option, to allow one checkout per > project on a slave. Technically this is the minimum number of checkouts > allowed, because each project is allowed to access a unique sha1 hash. > d) Add an option to the Jenkins core to disregard the top-level code > checkouts (which I gather are done for some kind of consistency check...we > can live without that check I think). > e) I don't know Java and don't have much free time, otherwise I would > write a completely new git plugin. > > Thanks for any suggestions, especially if I am missing something obvious > here. > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
