Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
Heiko Voigtwrites: > On Thu, Sep 15, 2016 at 11:27:54AM -0700, Junio C Hamano wrote: > >> If the trend in Git community collectively these days is to make >> usage of submodules easier and smoother, I'd imagine that you would >> want to teach "git add" that was given a submodule to "git submodule >> add" instead by default, with an option "git add --no-gitmodules >> sub" to disable it, or something like that. >> >> > $ git submodule add --fixup-modules-file ./sub sub >> > Adding .gitmodule entry only for `sub` to use `git -C remote >> > show origin` as URL. >> >> I agree that a feature like this is needed regardless of what >> happens at "git add" time. > > How about just > >git submodule add When I said "a feature like this is needed", I didn't care about exact syntax. I am not sure how often people need the "fixup", what kind of causes there are that they need the "fixup", and what the distribution of vaious causes would be like. If the _ONLY_ kind of fixup necessary is "I meant to say 'git submodule add ./path path' but I said 'git add path' instead", then I think it makes sense to teach "submodule add" that the form "git submodule add " is a short-pand for "git submodule add ./ ". I am not sure if we want to _ignore_ a gitlink that is already in the index unconditionally, i.e. if it is a good idea to let the second one override the first one git submodule add $URL sub && git submodule add sub in this sequence, though. > ? I remember back in the days when I started with submodules thats the > way I imagined submodules would work: > > 1. clone the submodule into a directory > 2. git submodule add it > 3. git commit everything > > Because that how you basically work with files. So instead of adding > another option I would rather like to autodetect that: > > * its a relative path inside this repo that is passed to >'git submodule add' > * there is no .gitmodules entry > * and no .git/config > ==> create those from a remote in the submodule In other words, I agree with the general direction but I'd add another condition to the above three, i.e. * and there is no gitlink for that path in the index yet.
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
On Fri, Sep 16, 2016 at 7:11 AM, Heiko Voigtwrote: > How about just > >git submodule add > > ? I remember back in the days when I started with submodules thats the > way I imagined submodules would work: > > 1. clone the submodule into a directory > 2. git submodule add it > 3. git commit everything > > Because that how you basically work with files. So instead of adding > another option I would rather like to autodetect that: > > * its a relative path inside this repo that is passed to >'git submodule add' > * there is no .gitmodules entry > * and no .git/config > ==> create those from a remote in the submodule > > Corner cases: > > * If there is more than one remote we could tell the user to use an >option to specify which one to use. > * Barf in case there is no remote (not adding the submodule except -f >is used). > * If the gitlink is already there but no .gitmodules entry, 'git >submodule add' will just add the entry as if it was initially added. > > Instead of giving an error message that the submodule is already added > we could actually be nicer to the user and try to fix things for him > instead. > This makes sense to me. Possibly we could warn in this case, so that the user knows that something was "off" but I don't think we should be failing here... Regards, Jake > Cheers Heiko
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
On Thu, Sep 15, 2016 at 11:27:54AM -0700, Junio C Hamano wrote: > Stefan Bellerwrites: > > So how about this fictional work flow: > > > > $ git init top > > $ cd top > > $ git commit --allow-empty -m 'initial in top' > > $ git init sub > > $ git -C sub commit --allow-empty -m 'initial in sub' > > $ git add sub > > You added a gitlink, but no corresponding entry in > > .gitmodules is found. This is fine for gits core functionality, but > > the submodule command gets confused by this unless you add 'sub' > > to your .gitmodules via `git submodule add --already-in-tree \ > > --reuse-submodules-origin-as-URL sub`. Alternatively you can make > > this > > message disappear by configuring advice.gitlinkPitfalls. > > I am not sure if I agree with that direction. > > If the trend in Git community collectively these days is to make > usage of submodules easier and smoother, I'd imagine that you would > want to teach "git add" that was given a submodule to "git submodule > add" instead by default, with an option "git add --no-gitmodules > sub" to disable it, or something like that. > > > $ git submodule add --fixup-modules-file ./sub sub > > Adding .gitmodule entry only for `sub` to use `git -C remote > > show origin` as URL. > > I agree that a feature like this is needed regardless of what > happens at "git add" time. How about just git submodule add ? I remember back in the days when I started with submodules thats the way I imagined submodules would work: 1. clone the submodule into a directory 2. git submodule add it 3. git commit everything Because that how you basically work with files. So instead of adding another option I would rather like to autodetect that: * its a relative path inside this repo that is passed to 'git submodule add' * there is no .gitmodules entry * and no .git/config ==> create those from a remote in the submodule Corner cases: * If there is more than one remote we could tell the user to use an option to specify which one to use. * Barf in case there is no remote (not adding the submodule except -f is used). * If the gitlink is already there but no .gitmodules entry, 'git submodule add' will just add the entry as if it was initially added. Instead of giving an error message that the submodule is already added we could actually be nicer to the user and try to fix things for him instead. Cheers Heiko
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
Yaroslav Halchenkowrites: > do you foresee any unpleasant side-effects from above manual editing > .gitmodules/submodule update --init ? I do not think so; you essentially did what a canned command we should have had should have done by hand because of a lack of such a feature.
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
Stefan Bellerwrites: > When searching around the net, some people use half > initialized submodules intentionally,... > > Not sure I agree with such a setup, but people use it. In such a top-level project, people would not use "git submodule" command, would they? I do not think anybody in this thread was pushing to forbid such a use, and it may be perfectly fine if "git submodule" does not work for such a gitlink; after all such a subdirectory is not even meant to be a submodule. > So how about this fictional work flow: > > $ git init top > $ cd top > $ git commit --allow-empty -m 'initial in top' > $ git init sub > $ git -C sub commit --allow-empty -m 'initial in sub' > $ git add sub > You added a gitlink, but no corresponding entry in > .gitmodules is found. This is fine for gits core functionality, but > the submodule command gets confused by this unless you add 'sub' > to your .gitmodules via `git submodule add --already-in-tree \ > --reuse-submodules-origin-as-URL sub`. Alternatively you can make this > message disappear by configuring advice.gitlinkPitfalls. I am not sure if I agree with that direction. If the trend in Git community collectively these days is to make usage of submodules easier and smoother, I'd imagine that you would want to teach "git add" that was given a submodule to "git submodule add" instead by default, with an option "git add --no-gitmodules sub" to disable it, or something like that. > $ git submodule add --fixup-modules-file ./sub sub > Adding .gitmodule entry only for `sub` to use `git -C remote > show origin` as URL. I agree that a feature like this is needed regardless of what happens at "git add" time.
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
On Thu, Sep 15, 2016 at 11:12 AM, Yaroslav Halchenkowrote: > > do you foresee any unpleasant side-effects from above manual editing > .gitmodules/submodule update --init ? I think that is fine, but un(der)documented. So you have to figure it out from experience what to do exactly.
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
On Thu, Sep 15, 2016 at 11:02 AM, Junio C Hamanowrote: > I think that is a more pressing thing to address. Once we make it > easier for the user to bring a half-initialized submodule properly > into the world view of the submodule subsystem, we would have to > worry about the reported failure case even less and you do not need > to pile on workaround options to let things continue in a state that > is half-broken (that is, in a state that is perfectly sane to the > core layer, but is not liked by the submodule layer). Heh, I see. When searching around the net, some people use half initialized submodules intentionally, e.g. I'll store some private keys in sub and I publish the superproject asking for collaborators to my new shiny webbased thing. The submodule containing the private keys never leaves my hard drive, hence no .gitmodules entry is necessary. Not sure I agree with such a setup, but people use it. So how about this fictional work flow: $ git init top $ cd top $ git commit --allow-empty -m 'initial in top' $ git init sub $ git -C sub commit --allow-empty -m 'initial in sub' $ git add sub You added a gitlink, but no corresponding entry in .gitmodules is found. This is fine for gits core functionality, but the submodule command gets confused by this unless you add 'sub' to your .gitmodules via `git submodule add --already-in-tree \ --reuse-submodules-origin-as-URL sub`. Alternatively you can make this message disappear by configuring advice.gitlinkPitfalls. $ git submodule ... similar advice goes here... fatal: no submodule mapping found in .gitmodules for path 'sub' $ git submodule add --fixup-modules-file ./sub sub Adding .gitmodule entry only for `sub` to use `git -C remote show origin` as URL. # user is happy now.
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
On Thu, 15 Sep 2016, Junio C Hamano wrote: > >> which then stops without even looking at other submodules. > >> I think it would be more logical to make it a 'warning:' not a 'fatal:' and > >> proceed. > Making "git submodule" listing to continue from that point may be > one thing, but do we have a sensible way in "git submodule add" to > allow the user to recover from this condition? That is, "git add" > is a right way to tell the core level that there is a gitlink, but > as Yaroslav correctly observed in the early part of his message, > having that gitlink alone is not good enough for the world view of > "git submodule" that sits at higher-layer. And the usual way to > tell the submodule layer that there is a submodule is with "git > submodule add", but > $ git init top > $ cd top > $ git commit --allow-empty -m 'initial in top' > $ git init sub > $ git -C sub commit --allow-empty -m 'initial in sub' > $ git add sub > $ git submodule > fatal: no submodule mapping found in .gitmodules for path 'sub' > $ git submodule add ./sub sub > 'sub' already exists in the index > $ git submodule add -f ./sub sub > 'sub' already exists in the index FWIW I could have sworn that I have tried to 'submodule add' it and it worked... but pragmatically I just did edit .gitmodules, added the record for it, committed it, and then iirc git submodule update --init which seemed to make git happy... FTR: $> git submodule add ./sub ./sub 'sub' already exists in the index $> git submodule add ./sub/ ./sub/ 'sub' already exists in the index $> vim .gitmodules $> git add .gitmodules cached/staged changes: .gitmodules | 4 sub | 1 + $> git submodule update --init Submodule 'sub' (/tmp/111/top/sub) registered for path 'sub' cached/staged changes: .gitmodules | 4 sub | 1 + $> git commit -m 'added finally' [master aa6d912] added finally 2 files changed, 5 insertions(+) create mode 100644 .gitmodules create mode 16 sub $> git submodule 6f574b298ef51aebd36daafad450a3e38802ca03 sub (heads/master) > I highly suspect that the user will then get stuck at this point, > after trying to "submodule add" and then even attempting to force > it. > I think that is a more pressing thing to address. Once we make it > easier for the user to bring a half-initialized submodule properly > into the world view of the submodule subsystem, we would have to > worry about the reported failure case even less and you do not need > to pile on workaround options to let things continue in a state that > is half-broken (that is, in a state that is perfectly sane to the > core layer, but is not liked by the submodule layer). do you foresee any unpleasant side-effects from above manual editing .gitmodules/submodule update --init ? -- Yaroslav O. Halchenko Center for Open Neuroscience http://centerforopenneuroscience.org Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
Stefan Bellerwrites: > On Thu, Sep 15, 2016 at 6:02 AM, Yaroslav Halchenko > wrote: >> >> If e.g. you just 'git add' a subdirectory which is a git repository, git >> adds it as a subproject but doesn't initiate any entry in .gitmodules >> since it is the job done by submodule and git core itself is >> agnostic of those beasts. >> ... >> $> git submodule >> cc6a09ac06c13cf06b4f4c8b54cda9a535e4e385 ds01 (2.0.0+4) >> 0a9f3b66e06a2137311a537b7377c336f1fb30ad ds02 (1.0.0-3-g0a9f3b6) >> 9da7e4f4221699915645ac2003298c6aba2db109 ds03 (1.1.0+4) >> fe16cacb5cb9b4d53c50e498298fab182500e147 ds05 (2.0.0+3) >> 6898d99ff3ba26880183ed3672a458a7fcde1737 ds06 (2.0.0+2) >> bbd10f634fe87e9d5853df3a891edbdb18cda7f9 ds07 (2.0.0+3) >> 138e6730193c0585a69b8baf5b9d7a4439e83ecc ds08 (2.0.0+2) >> ddf3a4cf7ce51a01a664e6faff4b8334b8414b1f ds09 (2.0.1+1) >> 7fa73b4df8166dba950c7dc07c3f8cdd50fca313 ds11 (1.0.0-5-g7fa73b4) >> fatal: no submodule mapping found in .gitmodules for path 'ds17 >> >> which then stops without even looking at other submodules. >> >> I think it would be more logical to make it a 'warning:' not a 'fatal:' and >> proceed. Making "git submodule" listing to continue from that point may be one thing, but do we have a sensible way in "git submodule add" to allow the user to recover from this condition? That is, "git add" is a right way to tell the core level that there is a gitlink, but as Yaroslav correctly observed in the early part of his message, having that gitlink alone is not good enough for the world view of "git submodule" that sits at higher-layer. And the usual way to tell the submodule layer that there is a submodule is with "git submodule add", but $ git init top $ cd top $ git commit --allow-empty -m 'initial in top' $ git init sub $ git -C sub commit --allow-empty -m 'initial in sub' $ git add sub $ git submodule fatal: no submodule mapping found in .gitmodules for path 'sub' $ git submodule add ./sub sub 'sub' already exists in the index $ git submodule add -f ./sub sub 'sub' already exists in the index I highly suspect that the user will then get stuck at this point, after trying to "submodule add" and then even attempting to force it. I think that is a more pressing thing to address. Once we make it easier for the user to bring a half-initialized submodule properly into the world view of the submodule subsystem, we would have to worry about the reported failure case even less and you do not need to pile on workaround options to let things continue in a state that is half-broken (that is, in a state that is perfectly sane to the core layer, but is not liked by the submodule layer).
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
On Thu, Sep 15, 2016 at 9:40 AM, Yaroslav Halchenkowrote: > > On Thu, 15 Sep 2016, Stefan Beller wrote: > >> > I think it would be more logical to make it a 'warning:' not a 'fatal:' and >> > proceed. > >> So maybe we would want to introduce a switch >> `--existing-but-unconfigure-gitlinks=(warn|ignore)` >> as well as >> `git config submodule.existing-but-unconfigured (warn|ignore)` >> for a more permanent solution? > > possibly ignorant question: is gitlink === Subprojector a > Subproject is a kinda of a gitlink and there are other gitlinks which > aren't Subprojects? ;) > gitlink is the internal name (just like a file is called blob, or a directory is called tree; there is no file system equivalent for e.g. commits, or gitlinks) gitlinks as a basic building block only points at a sha1 to be part of the repository. Submodules use gitlinks to point at (usually) different projects, e.g. a library. Subprojects in Git is a loose term, it could mean submodule or part of your repo merged in via the git-subtree command. Or a sub project can be just a loose repository inside your repository. gitlinks do not necessarily need to track other projects; it's "just a pointer" to a specific version of a repository. So to come back to your question: yeah In that response I meant submodule==gitlink, though naming the internal data structure to be more explicit. (What if we invent a new way to do submodules not using gitlinks? Then that config option may be less confusing and backwards compatible. That is unlikely though. So I guess we'll settle with a different name)
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
On Thu, 15 Sep 2016, Stefan Beller wrote: > > I think it would be more logical to make it a 'warning:' not a 'fatal:' and > > proceed. > So maybe we would want to introduce a switch > `--existing-but-unconfigure-gitlinks=(warn|ignore)` > as well as > `git config submodule.existing-but-unconfigured (warn|ignore)` > for a more permanent solution? possibly ignorant question: is gitlink === Subprojector a Subproject is a kinda of a gitlink and there are other gitlinks which aren't Subprojects? ;) -- Yaroslav O. Halchenko Center for Open Neuroscience http://centerforopenneuroscience.org Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik
Re: [wishlist?] make submodule commands robust to having non-submodule Subprojects
On Thu, Sep 15, 2016 at 6:02 AM, Yaroslav Halchenkowrote: > NB echos some questions of mine a few days back on IRC about Subprojects > and submodules > > If e.g. you just 'git add' a subdirectory which is a git repository, git > adds it as a subproject but doesn't initiate any entry in .gitmodules > since it is the job done by submodule and git core itself is > agnostic of those beasts. > > But having then this "Subproject"s which aren't registered as submodules > (and I haven't found any other use for them besides being a submodule) > brakes "git submodule" commands, e.g. > > $> git submodule > cc6a09ac06c13cf06b4f4c8b54cda9a535e4e385 ds01 (2.0.0+4) > 0a9f3b66e06a2137311a537b7377c336f1fb30ad ds02 (1.0.0-3-g0a9f3b6) > 9da7e4f4221699915645ac2003298c6aba2db109 ds03 (1.1.0+4) > fe16cacb5cb9b4d53c50e498298fab182500e147 ds05 (2.0.0+3) > 6898d99ff3ba26880183ed3672a458a7fcde1737 ds06 (2.0.0+2) > bbd10f634fe87e9d5853df3a891edbdb18cda7f9 ds07 (2.0.0+3) > 138e6730193c0585a69b8baf5b9d7a4439e83ecc ds08 (2.0.0+2) > ddf3a4cf7ce51a01a664e6faff4b8334b8414b1f ds09 (2.0.1+1) > 7fa73b4df8166dba950c7dc07c3f8cdd50fca313 ds11 (1.0.0-5-g7fa73b4) > fatal: no submodule mapping found in .gitmodules for path 'ds17 > > which then stops without even looking at other submodules. > > I think it would be more logical to make it a 'warning:' not a 'fatal:' and > proceed. So maybe we would want to introduce a switch `--existing-but-unconfigure-gitlinks=(warn|ignore)` as well as `git config submodule.existing-but-unconfigured (warn|ignore)` for a more permanent solution? > > Thank you for consideration > -- > Yaroslav O. Halchenko > Center for Open Neuroscience http://centerforopenneuroscience.org > Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 > Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 > WWW: http://www.linkedin.com/in/yarik
[wishlist?] make submodule commands robust to having non-submodule Subprojects
NB echos some questions of mine a few days back on IRC about Subprojects and submodules If e.g. you just 'git add' a subdirectory which is a git repository, git adds it as a subproject but doesn't initiate any entry in .gitmodules since it is the job done by submodule and git core itself is agnostic of those beasts. But having then this "Subproject"s which aren't registered as submodules (and I haven't found any other use for them besides being a submodule) brakes "git submodule" commands, e.g. $> git submodule cc6a09ac06c13cf06b4f4c8b54cda9a535e4e385 ds01 (2.0.0+4) 0a9f3b66e06a2137311a537b7377c336f1fb30ad ds02 (1.0.0-3-g0a9f3b6) 9da7e4f4221699915645ac2003298c6aba2db109 ds03 (1.1.0+4) fe16cacb5cb9b4d53c50e498298fab182500e147 ds05 (2.0.0+3) 6898d99ff3ba26880183ed3672a458a7fcde1737 ds06 (2.0.0+2) bbd10f634fe87e9d5853df3a891edbdb18cda7f9 ds07 (2.0.0+3) 138e6730193c0585a69b8baf5b9d7a4439e83ecc ds08 (2.0.0+2) ddf3a4cf7ce51a01a664e6faff4b8334b8414b1f ds09 (2.0.1+1) 7fa73b4df8166dba950c7dc07c3f8cdd50fca313 ds11 (1.0.0-5-g7fa73b4) fatal: no submodule mapping found in .gitmodules for path 'ds17 which then stops without even looking at other submodules. I think it would be more logical to make it a 'warning:' not a 'fatal:' and proceed. Thank you for consideration -- Yaroslav O. Halchenko Center for Open Neuroscience http://centerforopenneuroscience.org Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik