Victor Lowther ([email protected]) wrote:
> On Fri, Jan 3, 2014 at 8:52 AM, Adam Spiers <[email protected]> wrote:
> > Victor Lowther ([email protected]) wrote:
> > > On Fri, Jan 3, 2014 at 5:33 AM, Adam Spiers <[email protected]> wrote:
> > > > Victor Lowther ([email protected]) wrote:
> > > > > This is intended to work as a starting point for a discussion of what
> > > > > tool
> > > > > we should use to use to track inter-repository dependencies for CB2.0
> > > > > in
> > > > > the opencrowbar org.
> > > >
> > > > Before evaluating the tools I think it would be worth trying to
> > > > clarify the requirements. For example, all the approaches you list
> > > > seem to assume that there will only be dependencies from a single
> > > > superproject repository onto multiple slave repositories, but why is
> > > > it safe to exclude the possible need to track dependencies between
> > > > slave repositories?
> > >
> > > Then you are assuming too much. None of the tools I mentioned enforce
> > > such
> > > a relationship between git repositories
> >
> > OK, maybe I didn't express myself clearly enough then, or maybe we
> > are talking about different types of dependencies.
> >
> > I assumed that by "inter-repository dependencies" you were talking
> > about dependencies at the version control layer, e.g. one of the
> > following:
[snipped]
> > (Probably not all of these would be useful, but I'm just listing them
> > now for clarity, to help us avoid talking at cross-purposes.)
> >
> > These types of dependencies are obviously very different in nature to
> > the crowbar.yml types of dependencies such as "barclamp-nova requires
> > barclamp-keystone".
> >
> > So are you talking about the first type of dependency or the second,
> > or both, or something else?
>
> The first type of dependency. However, that all the tools express their
> relations in terms of parent/child relationships does not mean that we are
> restricted to a one superproject -> many subprojects structure -- I am
> leaning towards a one-superproject-per-release structure myself, and the
> superprojects would pick and choose whatever subset of the slaves were
> appropriate.
You really mean a superproject (i.e. separate git repo) per release,
or per build? I can *maybe* understand per build, but per release
would dispose of the many benefits which git branches provide.
> > With regards to the first type of dependency listed above
> > (i.e. version control layer):
> >
> > ./dev, git submodule, and git subtree, and gitslave *all* assume a
> > topology which involves a single parent repository and one or more
> > child repositories. I'm not sure about gitslave, but for the other
> > three, version control layer dependencies can only be expressed from
> > the parent ("superproject" as gitslave calls it) onto children
> > (e.g. to define a "snapshot" representing a particular release), not
> > between children. So my question was: is this limitation safe, or
> > might it get in the way later on? The answer may well be "yes, it's
> > safe"; I just wanted to raise awareness that adoption of any of the
> > tools you proposed will impose that limitation.
>
> That is an area I would like to discuss -- what sort of restrictions are we
> willing to live with here?
Maybe easier to ask what things we are not willing to live without?
For example:
- the ability to express dependencies between barclamps
(e.g. for handling barclamp API changes or any other changes
where we would currently use PR bundles)
- CI which is easily run per pull-request, without having to
build and maintain a complex layer of custom automation
- making it easy for newcomers to start developing within minutes
rather than hours or days (e.g. without having to build/download
a large ISO or learn a complex workflow)
> > > > What do these repositories contain?
> > >
> > > Source code, docs, and metadata.
> > >
> > > > Multiple barclamps in the superproject and one per slave?
> > >
> > > Doesn't matter for the purposes of this conversation.
> >
> > It does, because ./dev currently assumes no barclamps in the
> > superproject and one per slave whereas the other tools you proposed
> > don't, so if you want to achieve something different to the current
> > layout then that will certainly impact the decision whether to
> > continue using ./dev or not.
>
> Or splitting ./dev into several smaller tools, one of which would be the
> bit that handles git repo interdependencies.
Right. Of course I'm not saying it's impossible to change ./dev in
order to accommodate a different topology (and as you already know I
am a big fan of the idea of splitting it up into smaller tools with
decoupled responsibilities) - but the effort of doing so would need to
be considered versus switching to another approach, and that's why it
impacts the decision around tool choice.
> > > > Where do tools live?
> > >
> > > Wherever we want them to.
> >
> > Will the superproject be release-specific or not? Development of
> > ./dev and its associated files was held back by the fear of breaking
> > reproducible builds for older releases. But then keeping any future
> > repository "meta" tool in a separate repository to the build toolchain
> > should alleviate this problem.
>
> One of the things I am interested in talking about.
OK, but let's first ascertain whether we really need a custom tool at
all.
> > > > I have a strong preference for reusing existing technology over
> > > > rolling custom tools. For example, packaging barclamps as gems would
> > > > provide the functionality of the whole gem/bundler ecosystem for free,
> > > > which AFAICS would in one stroke solve the whole problem.
> > >
> > > I don't see how introducing a packaging conversation here is useful,
> > > unless
> > > you prefer downloading a gem over cloning a repo when you want to hack on
> > > some random ruby project (as opposed to just using it)
> >
> > Yes, I *hugely* prefer that - that's the whole point! If dependencies
> > are available as gems then this whole discussion becomes a non-issue.
>
> > Example advantages:
> >
> > - Any barclamp repo could easily be unit-tested standalone in Travis
> > simply by ensuring its Gemfile expressed the right dependencies
> > (e.g. on the core framework and barclamp gems), rather than the
> > awful https://github.com/crowbar/travis-ci-crowbar tree flattening
> > hack we were forced to build previously, which was never able to
> > automatically test pull requests.
> >
> > - Dependencies could be expressed using version numbers adhering to
> > semantic versioning, which gives much more flexibility during
> > testing than a monolithic approach which insists git cloning an
> > exact set of commits/branches/tags from a set of remotes chosen
> > by an opinionated tool.
> >
> > - Doesn't reinvent any wheels.
> >
> > - Proven technology already adopted by the whole Ruby community.
> >
> > - Everyone already knows bundler / Gemfile etc. very well => very
> > shallow learning curve => more welcoming for newbies
>
> OK.
>
> > Packaging and dependency handling are NOT mutually orthogonal problems.
>
> True, but I wish to focus on dependencies at the VCS level for the purposes
> of this conversation, not the packaging level.
Why? If you agree that they are not orthogonal, then how would you
attempt to solve one without solving the other?
> If you want to do otherwise, please start a new thread.
That doesn't make sense to me either, given that we apparently agree
the problems can't be isolated ... but very well - two threads
tackling strongly coupled problems is better than one which ignores
one ;-p
_______________________________________________
Crowbar mailing list
[email protected]
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/
