Tight submodule bindings (was: Preferred local submodule branches)
On Wed, Jan 08, 2014 at 10:17:51PM -0800, W. Trevor King wrote: > In another branch of the submodule thread Francesco kicked off, I > mentioned that we could store the preferred local submodule branch on > a per-superbranch level if we used the > .git/modules//config for local overrides [1]. Here's > a patch series that greatly extends my v2 "submodule: Respect > requested branch on all clones" series [2] to also support automatic, > recursive submodule checkouts, as I outlined here [3]. > > [1]: http://article.gmane.org/gmane.comp.version-control.git/240240 > [2]: http://article.gmane.org/gmane.comp.version-control.git/239967 > [3]: http://article.gmane.org/gmane.comp.version-control.git/240192 While mulling over better ways to explain my local-branch idea, I've come up with a more tightly bound model that may help break the silence that has greeted the “Preferred local submodule branches” series ;). That series doesn't have strong options on update mechanics, which leads to wishy-washy exchanges where nobody has a clear mental picture: On Thu, Jan 09, 2014 at 10:40:52PM +0100, Jens Lehmann wrote: > Am 09.01.2014 20:55, schrieb W. Trevor King: > > On Thu, Jan 09, 2014 at 08:23:07PM +0100, Jens Lehmann wrote: > >> Am 09.01.2014 18:32, schrieb W. Trevor King: > >>>> when superproject branches are merged (with and without conflicts), > >>> > >>> I don't think this currently does anything to the submodule itself, > >>> and that makes sense to me (use 'submodule update' or my 'submodule > >>> checkout' if you want such effects). We should keep the current logic > >>> for updating the gitlinked $sha. In the case that the > >>> .gitmodule-configured local-branches disagree, we should give the > >>> usual conflict warning (and <<<===>>> markup) and let the user resolve > >>> the conflict in the usual way. > >> > >> For me it makes lots of sense that in recursive checkout mode the > >> merged submodules are already checked out (if possible) right after > >> a superproject merge, making another "submodule update" unnecessary > >> (the whole point of recursive update is to make "submodule update" > >> obsolete, except for "--remote"). > > > > If you force the user to have the configured local-branch checked out > > before a non-checkout operations with checkout side-effects (as we > > currently do for other kinds of dirty trees), I think you'll avoid > > most (all?) of the branch-clobbering problems. > > I'm thinking that a local branch works in two directions: It should > make it easy to follow an upstream branch and also make changes to it > (and publish those) if necessary. But neither local nor upstream > changes take precedence, so the user should either use "merge" or > "rebase" as update strategy or be asked to resolve the conflict > manually when "checkout" is configured and the branches diverged. > Does that make sense? The current series is only weakly bound (you can explicitly call git submodule checkout' to change to the preferred local submodule branch), and the current Git is extremely weakly bound (you have to cd into the submodule and change branches by hand). The following extrapolates the “Preferred local submodule branches” series to a tightly-bound ideal. Gitlinked commit hash - The submodule model revolves around links to commits (“gitlinks”): $ git ls-tree HEAD 100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f.gitmodules 16 commit fbfa124c29362f180026bf0074630e8bd0ff4550 submod These are effectively switchable trees. The tree referenced by commit fbfa124 is 492781c: $ (cd submod/ && git cat-file commit fbfa124) tree 492781c581d4dec380a61ef5ec69a104de448a74 … If you init the submodule, subsequent checkouts will check out that tree, just like 'git checkout' would do if you'd had a superproject tree like: $ git ls-tree HEAD 100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f.gitmodules 04 tree 492781c581d4dec380a61ef5ec69a104de448a74submod For folks who treat the submodule as a black box (and do no local development), switchable trees are all they care about. They can easily checkout (or not, with deinit), the submodule tree at a gitlinked hash, and everything is nice and reproducible. The fact that 'submod' is stored as a commit object and not a tree, is just a convenient marker for optional init/deinit/remote-update-integration functionality. Additional metadata, the initial checkout, and syncing down --- Howev
[RFC v3 0/4] Preferred local submodule branches
From: "W. Trevor King" In another branch of the submodule thread Francesco kicked off, I mentioned that we could store the preferred local submodule branch on a per-superbranch level if we used the .git/modules//config for local overrides [1]. Here's a patch series that greatly extends my v2 "submodule: Respect requested branch on all clones" series [2] to also support automatic, recursive submodule checkouts, as I outlined here [3]. After this series, I can get through: # create the subproject mkdir subproject && ( cd subproject && git init && echo 'Hello, world' > README && git add README && git commit -m 'Subproject v1' ) && # create the superproject mkdir superproject ( cd superproject && git init && git submodule add ../subproject submod && git config -f .gitmodules submodule.submod.update merge && git commit -am 'Superproject v1' && ( # 'submodule update' doesn't look in .gitmodules (yet [4]) for a # default update mode. Copy submodule.submod.update over to # .git/config git submodule init ) ) && # start a feature branch on the superproject ( cd superproject && #git checkout -b my-feature --recurse-submodules && ( # 'git submodule checkout --recurse-submodules' doesn't exist yet, so... git checkout -b my-feature && git submodule checkout -b --gitmodules ) && ( cd submod && echo 'Add the subproject side of this feature' > my-feature && git add my-feature && git commit -m 'Add my feature to the subproject' ) && echo 'Add the superproject side of this feature' > my-feature && git add my-feature && git commit -am 'Add the feature to the superproject' ) && # meanwhile, the subproject has been advancing ( cd subproject && echo 'Goodbye, world' >> README && git commit -am 'Subproject v2' ) && # we need to get that critical advance into the superproject quick! ( cd superproject && # update the master branch #git checkout --recurse-submodules master ( # 'git checkout --recurse-submodules' doesn't exist yet [5,6]. # Even with that patch, 'git checkout' won't respect # submodule..local-branch without further work. git checkout master && git submodule checkout ) && git submodule update --remote && git commit -am 'Catch submod up with Subproject v2' && # update the my-feature branch #git checkout --recurse-submodules my-feature && ( # 'git checkout --recurse-submodules' doesn't exist yet [5,6]. git checkout my-feature && git submodule checkout ) && git submodule update --remote && git commit -am 'Catch submod up with Subproject v2' && # what does the history look like? ( cd submod && git --no-pager log --graph --date-order --oneline --decorate --all # * 16d9e3e (HEAD, my-feature) Merge commit 'f5e134d5747ee4a206e96d8c017f92f5b29a07f3' into my-feature # |\ # | * f5e134d (origin/master, origin/HEAD, master) Subproject v2 # * | 0a1cd07 Add my feature to the subproject # |/ # * c2d32ba Subproject v1 ) && printf 'master: ' && git ls-tree master submod && # master: 16 commit f5e134d5747ee4a206e96d8c017f92f5b29a07f3 submod printf 'my-feature: ' && git ls-tree my-feature submod # my-feature: 16 commit 16d9e3ea2fb57e7a166587203abdb328f90895d1 submod ) git --version # git version 1.8.5.2.237.g01c62c6 I think the first three patches are fairly solid. The last one gets through the above script, but I'd need a more thorough test suite before I trusted it. I tried to be detailed in the commit messages, but of course, we'd want some user-facing documentation if we actually merged something like this series. I'm sending it to the list mostly to explain my current views and re-focus debate [1]. [1]: http://article.gmane.org/gmane.comp.version-control.git/240240 [2]: http://article.gmane.org/gmane.comp.version-control.git/239967 [3]: http://article.gmane.org/gmane.comp.version-control.git/240192 [4]: http://article.gmane.org/gmane.comp.version-control.git/239246 [5]: http://thread.gmane.org/gmane.comp.version-control.git/239695 [6]: http://article.gmane.org/gmane.comp.version-control.git/240117 Cheers, Trevor W. Trevor King (4): submodule: Add helpers for configurable local branches submodule: Teach 'update' to preserve local branches submodule: Teach 'add' about a configurable local-branch submodule: Add a new 'checkout' command git-submodule.sh | 152 ++- 1 file changed, 138 insertions(+), 14 deletions(-) -- 1.8.5.2.237.g01c62c6 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Preferred local submodule branches
On Tue, Jan 07, 2014 at 07:47:08PM -0800, W. Trevor King wrote: > #git checkout --recurse-submodules master > ( # 'git checkout --recurse-submodules' doesn't exist yet [2,3]. > # Even with that patch, 'git checkout' won't respect > # submodule..local-branch without further work. > git checkout master && > cd submod && > git checkout master # don't pull in our my-feature work > ) > git submodule update --remote && > git commit -am 'Catch submod up with Subproject v2' && > # update the my-feature branch > git checkout my-feature > ( # 'git checkout' doesn't mess with submodules > cd submod && > git checkout my-feature > ) Oops, the my-feature checkout block should have been: #git checkout --recurse-submodules my-feature ( # 'git checkout --recurse-submodules' doesn't exist yet... git checkout my-feature && cd submod && git checkout my-feature ) mirroring the earlier master checkout block. Sorry for the sloppy editing. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
Re: Preferred local submodule branches
On Wed, Jan 08, 2014 at 03:12:44AM +0100, Francesco Pretto wrote:
> 2014/1/8 W. Trevor King :
> > Note that I've moved away from “submodule..branch
> > set means attached” towards “we should set per-superproject-branch
> > submodule..local-branch explicitly” [1].
>
> Honestly, I'm having an hard time to follow this thread.
I tried to refocus things (with a new subject) in this sub-thread.
Hopefully that helps make the discussion more linear ;).
> Also, you didn't update the patch.
I'm waiting [1] to see how the C-level checkout by Jens and Jonathan
progresses [2,3] before writing more code.
> If you were endorsed by someone (Junio, Heiko, ...) for the
> "submodule..local-branch" feature please show me where.
As far as I know, no-one else has endorsed this idea (yet :). Heiko
has expressed concern [4], but not convincingly enough (yet :) to win
me over ;).
> I somehow understand the point of the
> "submodule..local-branch" property, but I can't "see" the the
> workflow. Please, show me some hypothetical scripting example with
> as much complete as possible workflow (creation, developer update,
> mantainers creates feature branch, developer update, developer
> attach to another branch).
I've put this at the bottom of the message to avoid bothering the
tl;dr crowd, although they have probably long since tuned us out ;).
> Also, consider I proposed to support the attached HEAD path to
> reduce complexity and support a simpler use case for git
> submodules. I would be disappointed if the complexity is reduced in
> a way and augmented in another.
Agreed. I think we're all looking for the least-complex solution that
covers all (or most) reasonable workflows.
> > On Wed, Jan 08, 2014 at 01:17:49AM +0100, Francesco Pretto wrote:
> >> # Attach the submodule HEAD to .
> >> # Also set ".git/config" 'submodule..branch' to
> >> $ git submodule head -b --attach
> > [...]
> > I also prefer 'checkout' to 'head', because 'checkout'
> > already exists in non-submodule Git for switching between local
> > branches.
>
> I can agree with similarity to other git commands, but 'checkout'
> does not give me the idea of something that writes to ".git/config"
> or ".gitmodules".
Neither does 'head'. We have precedence in 'git submodule add' for
embracing and extending a core git command with additional .gitmodules
manipulation. I think it's easier to pick up the submodule jargon
when we add submodule-specific side-effects to submodule-specific
commands named after their core analogs than it would be if we pick
unique names for the submodule-specific commands.
> >> # Unset ".git/config" 'submodule..branch'
> >> # Also attach or detach the HEAD according to what is in ".gitmodules":
> >> # with Trevor's patch 'submodule..branch' set means attached,
> >> # unset means detached
> >> $ git submodule head --reset
> >
> > To me this reads “always detach HEAD” (because it unsets
> > submodule..branch, and submodule..branch unset means
> > detached).
>
> I disagree: this would remove only the value in ".git/config". If the
> value is till present in ".gitmodules", as I wrote above, the behavior
> of what is in the index should be respected as for the other
> properties. Also it gives a nice meaning to a switch like --reset :
> return to how it was before.
Ah, that makes more sense. I had confused .git/config with
“.gitmodules and .git/config”.
> >> NOTE: feature branch part!
> >>
> >> # Set ".gitmodules" 'submodule..branch' to
> >> $ git submodule head -b --attach --index
> >>
> >> # Unset ".gitmodules" 'submodule..branch'
> >> $ git submodule head --reset --index
> >> -
> >
> > These are just manipulating .gitmodules. I think we also need
> > per-superproject-branch configs under the superproject's .git/ for
> > developer overrides.
>
> I disagree: in my idea the --index switch is a maintainer only command
> to modify the behavior of the developers and touch only indexed files
> (.gitmodules, or create a new submodule branch). It expressly don't
> touch .git/config.
Something that just touches the config files is syntactic sugar, so I
avoided a more detailed review and moved on to address what I saw as a
more fundamental issue (preferred submodule local branches on a
per-superproject-branch level).
Here's a detailed workflow for the {my-feature, my-feature, master}
example I roughed out before [5].
# create the subproject
mkdir subproject &&
(
cd subproject &&
git init &&
echo 'Hello, world' > README &&
git add README &&
git commit -m 'Subproject v1'
) &&
# create the superproject
mkdir superproject
(
cd superproject &&
git init &&
git submodule add ../subproject submod &&
git config -f .gitmodules submodule.submod.update merge &&
git commit -am 'Superproject v1' &&
( # 'submodule update' doesn't look in .gitmodules (yet [6]) for a
# default update mode. Copy submodule.submod
Re: Preferred local submodule branches (was: Introduce git submodule attached update)
On Tue, Jan 07, 2014 at 02:36:25PM -0800, W. Trevor King wrote: > There are three branches that submodule folks usually care about: > > 1. The linked $sha1 in the superproject (set explicitly for every >superproject commit, and thus for every superproject branch). > 2. The remote-tracking submodule..branch that lives in the >upstream submodule..url repository. > 3. The submodule's locally checked out branch, which we currently let >the developer setup by hand, which is used integrated with one of >the other two branches during non-checkout updates. > > Git is currently a bit weak on conveniently handling type-3 branches. > “Just use what the developer has setup” works well for many basic > workflows, but falls short for: > > * Cloning-updates, where we currently always setup a detached HEAD. > * Workflows where the preferred type-3 branch depends on the > superproject branch. > > The former is easy to fix [1] if you accept submodule..branch as > a guess, but this conflates the type-2 and type-3 branches. > > For the latter, you'd want something like: > > On Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote: > > * Auto checkout of the preferred branch > > * Can do this at clone-update time with my patch. > > * For later submodule branch switches, maybe we want: > > > > git submodule checkout [-b ] […] > > > > Then if a user blows off their detached HEAD, at least they'll > > feel a bit sheepish afterwards. > > which would likely need some of Jens' new core checkout handling [2]. > > [1]: Using something along the lines of my > http://article.gmane.org/gmane.comp.version-control.git/239967 > [2]: http://article.gmane.org/gmane.comp.version-control.git/240117 For example, in Jonathan's recent version of Jens' series, the initial-setup and update functionality are moving into C. See: * populate_submodule() [1] for the initial-clone setup (calling 'read-tree'), and * update_submodule() [2] for subsequent updates (calling 'checkout -q' with an optional '-f') this is where any submodule..local-branch would come into play, if we decide to go down that route. It doesn't look like the C updates have the auto-clone functionality that the Bash updates have. I'm not sure if that's in the pipe or not. I'm not as familiar with the C implementation though, so maybe I'm missing the mark here. Cheers, Trevor [1]: http://article.gmane.org/gmane.comp.version-control.git/239698 [2]: http://article.gmane.org/gmane.comp.version-control.git/239699 -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
Preferred local submodule branches (was: Introduce git submodule attached update)
On Tue, Jan 07, 2014 at 10:51:34PM +0100, Francesco Pretto wrote: > 2014/1/7 W. Trevor King : > > > > I'd be happy to hear ideas about superproject-branch-specific local > > overrides to a hypothetical submodule..local-branch, in the > > event that a developer doesn't like a default set in .gitmodules. If > > I could think of a way to do that, we could avoid this heuristic > > approach, and make the local submodule..local-branch > > vs. remote-tracking submodule..branch distinction more obvious. > > Uh, I think you got it wrong in the other thread: I'm grafting this discussion back on to the thread where I proposed submodule..local-branch. > I didn't proposed such feature. Right. I proposed this feature after reading your proposed workflow. > I just wanted the attached submodule use case to be supported and of > course "--branch means attached" is even easier to get this. As I understood it, the '--branch means attached' stuff was tied up with automatic --remote updates. There are three branches that submodule folks usually care about: 1. The linked $sha1 in the superproject (set explicitly for every superproject commit, and thus for every superproject branch). 2. The remote-tracking submodule..branch that lives in the upstream submodule..url repository. 3. The submodule's locally checked out branch, which we currently let the developer setup by hand, which is used integrated with one of the other two branches during non-checkout updates. Git is currently a bit weak on conveniently handling type-3 branches. “Just use what the developer has setup” works well for many basic workflows, but falls short for: * Cloning-updates, where we currently always setup a detached HEAD. * Workflows where the preferred type-3 branch depends on the superproject branch. The former is easy to fix [1] if you accept submodule..branch as a guess, but this conflates the type-2 and type-3 branches. For the latter, you'd want something like: On Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote: > * Auto checkout of the preferred branch > * Can do this at clone-update time with my patch. > * For later submodule branch switches, maybe we want: > > git submodule checkout [-b ] […] > > Then if a user blows off their detached HEAD, at least they'll > feel a bit sheepish afterwards. which would likely need some of Jens' new core checkout handling [2]. Cheers, Trevor [1]: Using something along the lines of my http://article.gmane.org/gmane.comp.version-control.git/239967 [2]: http://article.gmane.org/gmane.comp.version-control.git/240117 -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
