Sami, Yes, I had been thinking about that too - but I actually already have a gitolite mirror repo (+git-svn clone) on the master... .. so I was already using "local remote urls"... however that mirror is on a different partition (LV). .. which breaks the hard links, I assume.
So thanks for the reminder - that will help on the master and linux slaves... BTW, you'd have one clone for each slave right? Also, is it a recommended practice to set up a Jenkins job solely to maintain this clone? If so, it would be nice to have plugin support (e.g a wizard to add "mirrored git clones" across master/slaves, and SCM type "Git local clone") - maybe I could start something like that, unless someone has already:) Still, a more general underlying mechanism would be great, e.g. one that even works on Windows slaves.. the object sharing would not work the same way (at least not with hard links - I assume). Thank you, Gergo On Mon, Feb 20, 2012 at 11:51 PM, Sami Tikka <[email protected]> wrote: > You can already achieve the same benefit by making a local clone of the > git repo (use --bare for this) and then configuring each job to have 2 > repos: the first should be /path/to/local/repo and the second can be the > location where you usually clone from. > > This way most git objects will be shared because a local git clone will > use hard links. > > My build slaves at work have small but fast ssd disks and we use this > trick (plus running git clean -fxd as a post-task step) to keep disk space > usage in control. > > -- Sami > > Gergely Nagy <[email protected]> kirjoitti 15.2.2012 kello 19.15: > > Thanks Mark, > that's great info - to me it sounds like the way to go. > Gergo > > On Wed, Feb 15, 2012 at 3:03 AM, Mark Waite <[email protected]> wrote: > >> The git plugin rework discussions mentioned the possibility of including >> the "---reference <existing-repository>" argument to git clone so the pack >> files for a single repository could be reused in multiple repositories on >> the same machine. Then you could clone to a single directory on the slave, >> and reference that clone rather than copying the pack files to each of the >> workspace copies. >> >> I don't think it has been implemented yet, but the plugin developers may >> be willing to share their ideas in case they have an even better idea than >> using the --reference argument to git clone. >> >> Mark Waite >> >> *From:* Gergely Nagy <[email protected]> >> *To:* [email protected] >> *Sent:* Tuesday, February 14, 2012 1:23 PM >> *Subject:* git: reduce clones' disk space >> >> Hi Jenkins gurus, >> >> I have a load of jobs (50+ I think) which clone the same repository, but >> different branches, to build/unit/test/functional test stages. >> >> Also, it's a special application of the "job splitting pattern" ( >> https://wiki.jenkins-ci.org/display/JENKINS/Splitting+a+big+job+into+smaller+jobs >> ): >> the tarball that downstream jobs receive is a much smaller than the >> entire workspace: it only contains unknown files(git ls-files -oz: the >> build artifacts), which is "just" 400m >> vs 1.8G. Downstream jobs unpack this on top of a pristine clone to get up >> to speed. This is quite fast (most files are there already) and also seems >> to do better change tracking. >> >> However it costs space - each of the workspace is ~ 4-5G - half of which >> is the git clone. >> While git has a good reason to clone everything with all the branches, I >> don't need that duplicated 50 times on the Jenkins box. >> So am wondering if there is a way to optimise this? >> I guess, i'd rather have one single full clone, and let jobs have the >> work directories (+index?).. >> >> Any enlightments/alternative ideas are appreciated. >> thanks, >> Gergo >> >> >> >> >> >> >
