I am in the process of getting familiar with git myself, but I think 'git archive' will get you the repo without the .git metadata.
----- Original Message ----- From: [email protected] To: [email protected] At: 1/04 9:55:59 This is a great idea except that I have no clue how git works, obviously.... I had confused "git checkout" with "svn export", and now that I look, I can't find a way to accomlish this after all. What I wanted might not be possible with git -- namely a way to download the repo, and just get a working tree with no repo metadata. What I want is the equivalant result of "svn export", which gives you HEAD of your SVN repo, without all the .svn dirs. Now, obviously, you can do this: git clone $url . rm -r .git But that will NEVER scale, as the size of the git history grows. Maybe the better mechanism is to have a commit hook which does this, and publishes a tarball on ftp.openefs.org with a "latest" symlink. Then the code can use wget and tar to achieve this goal, rather than using git directly. If one of you knows of a means to do this using git, directly, please let me know. I will continue researching this... On Wed, Jan 4, 2012 at 8:28 AM, Phillip Moore <[email protected]> wrote: > I came up with an alternate way to manage deploying these > deploy-config projects, that will make it trivial to keep them > uptodate, AND deal with the fact that we're managing them in multiple > repos. > > First of all, for flexibility, I'm still going to implement the search > mechanism for the efsdeploy directory as I described before. However, > based on the way I've structured the git repos, you can actually do a > "git checkout" and drop them all into the same root directory? > > I'm going to try this today, since it so damn simple. > efsdeploy_config_update will be the script that does the following: > > efs create autorelease efs deploy-config > cd /efs/dev/efs/deploy-config/next/install/common > git checkout http://git.openefs.org/deploy-config > git checkout http://git.openefs.org/deploy-config-aix > git checkout http://git.openefs.org/deploy-config-gnu > .... > efs dist autorelease efs deploy-config > > Now, you have /efs/dist/efs/deploy-config/current/common with ALL of > the published git configs. > > Note that because ALL of these repos are structures with a > metaproj/project structure, they can ALL co-exist in the same > directory tree (if you use checkout, I think -- I haven't tried this > yet, but since you don't get the .git directory, I don't see why this > won't work -- I'll figure out how to make it work :-P) > > Even better, we can drop a simple file into the root of each repo, > giving the name of the "child" repos in the obvious hierarchy here. > For example, in the root of deploy-config, the contents of > subrepos.txt might be: > > deploy-config-aix > deploy-config-gnu > deploy-config-rhel > deploy-config-sunos > > The subrepos.txt file in deploy-config-gnu will have to live in the > gnu subdir, to avoid clashes, but then, since the top tells us to > checkout deploy-config-gnu, we then know to look for the next > subrepos.txt file in ./gnu. This will then contain: > > deploy-config-gnu-gcc > deploy-config-gnu-gcclib > > This will give us the full flexibility of an easy to use, well managed > default (you only get the published, commited master branch), with the > ability to create and manage your own local repos as well. For > example, there will never be an "fsf" metaproj in the OpenEFS > namespace, and in practice, you've going to be migrating stuff to gnu, > I assume, but if you wanted to maintain your own deploy-config-fsf git > repo, that works fine. You would simply manage it in: > > /efs/dist/fsf/deploy-config-fsf > > I can even support publishing this using efsdeploy_config_update via > CLI args, if you wanted to use the same, simple mechanism. > > This is starting to come together very nicely, and now all we really > need are.... > > Users :-( > > On Fri, Dec 30, 2011 at 12:57 PM, Phillip Moore > <[email protected]> wrote: >> On Fri, Dec 30, 2011 at 12:09 PM, Phillip Moore >> <[email protected]> wrote: >>> More thoughts, and some significant progress in this area.... >>> >>> I spent most of yesterday collecting the efsedploy rules for >>> EVERYTHING I've built into /efs/dist over the last few months (it's a >>> lot), by copying the src directory to: >>> >>> ~/dev/efs/deploy-config/$metaproj/$project >> >> OK, so once everything in that directory has been sanitized of ALL >> site-specific information, then we have to figure out how to manage >> it. Here's what I'm currently thinking, although this is going to >> evolve, of course. >> >> First of all, note that efsdeploy is going to start whining at you to >> switch from efs/deploy-config to efs/deploy-site, because I want to >> use the name deploy-config for all of this data. Deal with it.... >> It's *trivial* to switch, and takes about 5-10 minutes, if you type >> slow. >> >> I want to create 3 types of git repo to manage this data: >> >> deploy-config-$metaproj-$project.git >> deploy-config-$metaproj.git >> deploy-config.git >> >> For things like gnu/gcc, we'll obviously create a project-specific git >> repo, and for large metaprojs where we expect a lot of similarity >> among the projects, we can create metaproj-specific ones. The >> default, global git repo would contain all the small, simple stuff, >> like oss/zlib. For starters, I expect to create these: >> >> deploy-config-gnu-gcc.git (which will be used for rhel/gcc as well) >> deploy-config-gnu-gcclib.git (also for rhel/gcclib) >> deploy-config-gnu.git >> deploy-config-perl5-core.git >> deploy-config-perl5.git >> deploy-config-apache.git (might get it's own system, too -- we'll see...) >> >> And of course the generic one. What I like about this is we always >> migrate things from one to the other pretty easily. if we find that, >> say oss/openssl has grown complex enough, we can yank it out of >> deploy-config, and create deploy-config-oss-openssl. >> >> So how do we deploy this data? Having it well managed is git is >> great, but how to we access it when building things with efsdeploy, >> and where does it get copied/cached? >> >> Let's start with the generic repo first. Just as we use >> efs/deploy-site/current to abstract the site-specific config >> information, I think we can do the following: >> >> deploy-config.git => /efs/dist/efs/deploy-config/current >> >> The metaproj- and project-specific ones would then map to: >> >> deploy-config-$metaproj.git => >> /efs/dist/$metaproj/deploy-config-$metaproj/current >> deploy-config-$metaproj-project.git => >> /efs/dist/$metaproj/deploy-config-$metaproj-$project/current >> >> This would allow us to publish, probably date-based, any of these >> repositories with the "latest" set of efsdeploy build rules. >> Note that the default rules go into the efs metaproj, obviously, but >> we can still have a "deploy-config-efs.git" repo if we want, with no >> conflict. >> >> It is very straight forward to code a solution that allows us to >> automate keeping the local copies of these rules uptodate as they >> change. I will almost certainly have a first pass at this within the >> next month. However, what is NOT clear is just how to use this >> information in efsdeploy when building release. >> >> Reproducibility concerns me. The rules are going to evolve, and when >> we make gnu/gcc rule changes to build, say 4.7.0, we don't want to >> break builds of 4.4.6, and yet *testing* that is extremely expensive. >> For that reason, I think the contents of the efsdeploy directory >> should be CACHED in the release, rather than read from these projects >> during the build. Just as we are going to provide generic dependency >> specs (see email from 30 minutes ago), and expanding those into >> specific releasealiases to be used for the duration of the build, I >> think we should do the same for the project-specific build rules, or >> at least make it optional. >> >> In theory, if we just have efsdeploy search for these rules the same >> way it searches for system-specific (i,e, gnu, perl5, etc) rules, and >> then site-specific rules, then I could actually build EVERYTHING I >> have in /efs/dist with EMPTY source directories!! If a project is >> supported by one of these repos, then you can build a new release with >> nothing more than: >> >> efs create project ... >> efs create release ... >> cd ..../src >> efsdeploy down:up >> >> The contents of the src directory would contain NOTHING but the >> changes you had to make (hooks, configs, whatever) to get the release >> to build. Those changes should then be re-integrated with the git >> repo in a controlled fashion, so that the next person building that >> MPR has no pain. The specific workflow for how a new change gets >> rolled into the published git repos will need to be worked out, but I >> think that will be straight forward. >> >> Now, obviously, in order to *develop* changes to the rules, we'll need >> a simply means of overriding the path to these published rules. >> Maybe you want to install the latest set of gnu/gcc rules, but not >> make them current until you've actually done a test-build of the >> releases you care about. Maybe something in efsdeploy.conf (which >> will now be a site/release-specific file, by definition) like this. >> Say we wanted to test out some local changes right from the source >> tree (I've been doing this with symlinks for now): >> >> [rules] >> $metaproj/$project = /home/efsops/dev/efs/deploy-config-gnu-gcc >> >> or, perhaps, if we use date-based releases, you could install the >> latest update into /efs/dist, and test it out this way: >> >> [rules] >> $metaproj/$project = /efs/dist/gnu/deploy-config-gnu-gcc/20111230 >> >> Alternately, you could just rsync the efsdeploy directory right into a >> release, and work with a copy. >> >> OK, that's enough of Phil's rantings for one day. Not that anyone's >> paying attention, but you will see commits that implement many of >> these features over the next few weeks. _______________________________________________ EFS-dev mailing list [email protected] http://mailman.openefs.org/mailman/listinfo/efs-dev _______________________________________________ EFS-dev mailing list [email protected] http://mailman.openefs.org/mailman/listinfo/efs-dev
