I'm very close to being able to create this git repo, and begin
experimenting with git2efs implementations, but I'm really not sure
that a git-repo per-project is going to scale. If we follow this
path, we'll end up with a LOT of git repos on openefs.org, since there
will easily be close to 1000 projects to manage this way, over time.
Having said that, I'm going to move forward with this for now, and not
get hung up prematurely optimizing something. I fully expect this to
evolve, just like everything else in EFS.
The bigger problem with managing the efsdeploy data is that the data
isn't segregated correctly. There are site-specific configuration
values interleaved with generic values, and you can't commit most of
the existing efsdeploy.conf files into a shared repo because of this.
For example, the specific releases used for dependencies, and the
list of platforms to build are site-specific, NOT generic, yet
everything else in efsdeploy.conf is generic. Refactoring the
gnu/gcc and gnu/gcclib rules exposed this, but I think I have a
solution.
efsdeploy has always supported a hierarchy of config files, and it was
always possible to override things in commands.conf, etc (as of a
patch yesterday, this includes globals.conf, too). This allows me to
reduce efsdeploy.conf to ONLY the site-specific customizations, by
moving things like:
[options]
use_vpath_build = 1
from efsdeploy.conf to efsdeploy/globals.conf. In the code I will
shortly be committing to the new gnu-gcc repo, the efsdeploy.conf file
will be empty, with nothing but comments. I'm also considering naming
this file efsdeploy.conf.tmpl, and adding efsdeploy.conf to
.gitignore. RIght now, efsdeploy requires that the efsdeploy.conf
file exist, but I can probably make it optional.
This will make efsdeploy.conf 100% site specific, and solves the data
segregation problem fairly well. The last issue is then how to
manage the efsdeploy.conf file itself with git (or svn, or whatever).
The challenge here is that I don't think you can have a single
directory that contains files managed by two different git repos, and
even if you could, I suspect it's a fragile configuration. That
suggests we look for a way to manage these files in different
directory locations.
I do not have a solution for this handy yet, but this first pass at
the problem will at least reduce the scope of the data we are NOT
managing in git to a very small subset, and then once we "see" the
bigger picture and understand the nature of the data we are managing
in the reduced efsdeploy.conf file, a solution will present itself.
They do this to me, sometimes....
This will allow me to start sketching out a "git2efs" utility, which
can hopefully be made generic.
Plowing forward....
On Fri, Oct 28, 2011 at 9:49 AM, Phillip Moore
<[email protected]> wrote:
> I'm working on a solution to the problem of how to manage the efsdeploy
> configuration information, hooks, patches, etc.
>
> Right now, we're not putting this information into source code control
> ANYWHERE. Not in EFS 2 land, not in EFS 3, nowhere. This isn't such a
> major problem because the /efs/dev namespace effectively versionizes this
> stuff everytime you checkpoint a new release, but it's a problem because
> we're not sharing this information, and we're not tracking ANY of the
> changes anywhere.
>
> I'm got about 5 pages of notes I made on how to tackle this and a solution
> is starting to form in my mind. The basic idea is that each project will
> have it's own git repo, and efsdeploy will be enhanced to know how to talk
> to it, very basically. In order to start playing around with the idea, and
> experimenting with automation for it, I need to create a few new git repos
> on openefs.org.
>
> The convention I'm currently considering is:
>
> git://git.openefs.org/efs-deploy-config-$metaproj-$project.git
>
> Then efsdeploy will get some new commands, for example:
>
> scminit
> scmpull
> scmpush
>
> For metaproj's like gnu, where we install each project by hand, this
> approach will work pretty well. For one's llke perl5, where most of the
> projects are created automatically by cpan2efs, I don't want to require a
> git repo, since in the overwhelming majority of cases, installation is 100%
> automated now. We're almost to the point where even some gnu projects can
> be installed with NO config (if we supported a default download->url for
> gnu/tar, not even efsdeploy.conf is necessary).
>
> This has become a higher priority issue because of the changes I have
> recently made to gnu/gcc's build scripts. I want to make sure we have this
> critical information available, and right now, it's not.
>
> Jerry -- can you create the above repo for me for gnu/gcc, and document how
> it's done, so we can automate it? I have some reservations about the git
> repo namespace growing linearly this way, so this is NOT a final design, but
> merely an experiment for a proof of concept. Once I have a basic workflow
> functioning, I want to write it up and get a discussion going (if possible
> -- this is one of the lowest volume mailing lists in the history of the
> Internet) about how to generalize the solution.
>
> Without a good solution to this, anyone bootstrapping an EFS 3 domain will
> have to struggle with ALL the same issues that are solved in the currently
> unavailable gnu/gcc rules, among other things. This is the missing piece of
> the puzzle for complete reproducibility of an EFS domain.
>
_______________________________________________
EFS-dev mailing list
[email protected]
http://mailman.openefs.org/mailman/listinfo/efs-dev
