Yep, was having a similar setup with CruiseControl before we moved to Jenkins. Really would love to avoid that. So while I like your ideas for how to work aorund the problem and get the Jenkins stuff to work I would still love to know why git is behaving this way (being a geek I can't sleep at night when I can't figure out ;) ).
As for the problem... I don't see sparse checkouts helping here. While it is probably a feasible solution that results in saving disk space on the local drive with local clones (due to the internal hard linking) it wouldn't help with all the remote nodes that need to clone and build it, as on the remote clones I'm still facing the issue of only getting the large repo. Is there no way to rewrite the history for a certain branch only (the subtree branch in this case) so that it only contains the files really needed. This is actually what I expected the subtree split command to do in the first place to be honest. At least that's what I understood (wished) by reading the doc (see split::) : https://github.com/apenwarr/git-subtree/blob/master/git-subtree.txt Extract a new, synthetic project history from the > history of the <prefix> subtree. The new history > includes only the commits (including merges) that > affected <prefix>, and each of those commits now has the > contents of <prefix> at the root of the project instead > of in a subdirectory. Thus, the newly created history > is suitable for export as a separate git repository. I guess the part I was wrong about is that I was hoping that it contains not only the commit that affected <prefix> but also has this commit kinda rewritten to only contain the files that are needed for this <prefix> as well (was hoping that working with squashing commit or somehow creating some artificial ones... but might be totally off here). More ideas on that? Would just love to understand what is going on in detail. Am Donnerstag, 30. August 2012 14:12:07 UTC+2 schrieb Thomas Ferris Nicolaisen: > > On Thursday, August 30, 2012 1:04:56 PM UTC+2, Haasip Satang wrote: >> >> Thanks Thomas, >> >> I was looking into these Jenkins options as well, but as you said >> yourself, it would be kind of a "short-term hack" and definitely would have >> big impact on our feedback loop as we have a whole bunch of projects and >> branches and of course would like to get timely feedback. (which is hard >> with only one Executor ;-) ). >> > > You can go beyond this limitation by having several build nodes: > > *Jenkins-node 1** (with one executor)* > /workspace/big-repo > > *Jenkins-node 2 (with one executor)* > /workspace/big-repo > > Now. two example jobs: > > *Job: Foo* > Custom workspace: /workspace/big-repo > > *Job: Bar* > Custom workspace: /workspace/big-repo > > Now these two jobs can build in parallell, and will use the respective > workspace on their own build node. > > > As for the hard links, please see my reply to Philips answer above. I >> don't understand why cloning locally (with file:// prefix though) leads too >> a small repository while cloning from remote results in a big one. When I >> clone the small local clone again from remotely though, it stays small and >> I end up with what I want to have... just that I need to jump across too >> many repos: >> > > This is just an optimization Git does on filesystems that support > hardlinks. Quoting the "--local" option from git clone: > > * --local, -l* > > > > *When the repository to clone from is on a local machine, this flag >> bypasses the normal "git aware" transport mechanism and clones the >> repository by making a copy of HEAD and everything under objects and refs >> directories. The files under .git/objects/ directory are hardlinked to save >> space when possible. This is now the default when the source repository is >> specified with /path/to/repo syntax, so it essentially is a no-op option. >> To force copying instead of hardlinking (which may be desirable if you are >> trying to make a back-up of your repository), but still avoid the usual >> "git aware" transport mechanism, --no-hardlinks can be used.* > > > If you again combine this with the sparse checkout, you will be saving a > lot of space. If I went for this approach, I wouldn't bother with subtrees > at all. > > About splitting up to small repos... yes, would love to do that... the >> problem is more of political nature at the moment and we are having to >> merge sources between clear case and git all the time for now (which >> sucks!!!). So have the whole clear case view under git control as well just >> makes this already cumbersome process a lot easier for us. Don't see that I >> will be able to change that anytime soon until some people finally >> understand that it just doesn't make sense and git is definitely a good >> thing and can be used on enterprise scope projects (as all the references >> we gave are not enough ;) )... anyway... let's not go there... as I said... >> politics ;( >> >> > I think we've all been there :) > -- You received this message because you are subscribed to the Google Groups "Git for human beings" group. To view this discussion on the web visit https://groups.google.com/d/msg/git-users/-/BKlC0y36LI0J. To post to this group, send email to email@example.com. To unsubscribe from this group, send email to git-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/git-users?hl=en.