Re: Moving (renaming) submodules, recipe/script
On Sun, Feb 03, 2013 at 10:36:17PM +, TJ wrote: I've recently had need to re-arrange more than ten submodules within a project and discovered there is apparently no easy way to do it. I ran into a similar problem last month, and wrote a similar script [1] ;). There are a few other related threads you might be interested in: On Sun, Jan 06, 2013 at 07:36:03PM -0500, W. Trevor King wrote: Today I had to move my first submodule, and I discovered that Git's support for this is pretty limited. There have been a few patch series attempting to address this [1,2], but none of them seems to have pushed through into master (although I can't put my finger on a reason for why). There are also some SO postings discussing this [3,4]. It would be nice if `git mv` worked out of the box on … [1]: http://thread.gmane.org/gmane.comp.version-control.git/88720 [2]: http://thread.gmane.org/gmane.comp.version-control.git/143250 [4]: http://stackoverflow.com/questions/4323558/moving-submodules-with-git [3]: http://stackoverflow.com/questions/4604486/how-do-i-move-an-existing-git-submodule-within-a-git-repository The long-term solution is probably Jens' branch: On Mon, Jan 07, 2013 at 07:59:53AM +0100, Jens Lehmann wrote: I´m currently working on teaching mv to move submodules and intend to send those patches to the list after finishing submodule deinit. Please see https://github.com/jlehmann/git-submod-enhancements/commits/mv-submodules for the current state of this series. Cheers, Trevor [1]: http://thread.gmane.org/gmane.comp.version-control.git/212861 -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
Re: Moving (renaming) submodules, recipe/script
On Mon, Jan 07, 2013 at 07:59:53AM +0100, Jens Lehmann wrote: Am 07.01.2013 02:39, schrieb Jonathan Nieder: (just cc-ing Jens and Peter, who might be interested) I´m currently working on teaching mv to move submodules and intend to send those patches to the list after finishing submodule deinit. Please see https://github.com/jlehmann/git-submod-enhancements/commits/mv-submodules for the current state of this series. Thinking about this a bit more, I'm not clear on how out-of-tree updates (i.e. worktree in .git/modules/*/config) propogated during branch checkouts (merges, rebases, etc.). I just got a broken rebase trying to move a trivial patch back before the submodule move, and Git was confused about what had happened to the submodules. Here's a simple script that illustrates the problem: #!/bin/sh rm -rf a b c mkdir a (cd a git init echo a README git add README git commit -am 'a' ) git clone a b (cd b git submodule add ../a submod-1 git commit -am 'add submodule at submod-1' ) git clone b c (cd c git submodule update --init ) (cd b git-submodule-mv.sh submod-1 submod-2 git commit -am 'move submodule from submod-1 to submod-2' ) (cd c git pull ls -d .git/modules/* cat .git/modules/submod-1/config ls -a submod* ) The end result is that `c` gets the `.gitmodules` path updates and new gitlinked directory from the submodule move in `b` (using my git-submodule-mv.sh posted earlier in this thread), but `submod-1` is left lying around (because Git doesn't know that it can remove submod-1/.git and submod-1/README). The Git directory for the submodule stays in .git/modules/submod-1/ (good), but the worktree in .git/modules/submod-1/config still points to ../../../submod-1 (bad). This means that submodule moves are possible, but anyone trying to share them between several repositories (or trying to rebase across the move within their own repository) is in for a world of suffering ;). I'm not sure how this should be addressed, but I didn't see anything handling it in Jens' new series. Thanks, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
Re: Moving (renaming) submodules, recipe/script
On Tue, Jan 08, 2013 at 09:32:14AM -0500, W. Trevor King wrote: Thinking about this a bit more, I'm not clear on how out-of-tree updates (i.e. worktree in .git/modules/*/config) propogated during branch checkouts (merges, rebases, etc.). Actually, I don't understand why storing `worktree` in .git/modules/*/config is useful at all…. This may be related to my lack of clarity on the why can't we have multiple working directories checked out at the same time issue: On Tue, Oct 23, 2012 at 06:09:55PM -0400, W. Trevor King wrote: On Tue, Oct 23, 2012 at 10:57:57PM +0200, Jens Lehmann wrote: Am 22.10.2012 14:37, schrieb W. Trevor King: but cloning a remote repository (vs. checking out a local branch) seems to be baked into the submodule implementation. Should I be thinking about generalizing git-submodule.sh, or am I looking under the wrong rock? My ideal syntax would be something like $ git submodule add -b c --local dir-for-c/ But then we'd have to be able to have two (or more) work trees using the same git directory, which current submodule code can't. And that's the problem I'm trying to solve ;). Can someone with a better feeling for why this won't work. Is is just that there's only one `.git/HEAD`? Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
Re: Moving (renaming) submodules, recipe/script
Am 08.01.2013 15:32, schrieb W. Trevor King: On Mon, Jan 07, 2013 at 07:59:53AM +0100, Jens Lehmann wrote: Am 07.01.2013 02:39, schrieb Jonathan Nieder: (just cc-ing Jens and Peter, who might be interested) I´m currently working on teaching mv to move submodules and intend to send those patches to the list after finishing submodule deinit. Please see https://github.com/jlehmann/git-submod-enhancements/commits/mv-submodules for the current state of this series. Thinking about this a bit more, I'm not clear on how out-of-tree updates (i.e. worktree in .git/modules/*/config) propogated during branch checkouts (merges, rebases, etc.). I just got a broken rebase trying to move a trivial patch back before the submodule move, and Git was confused about what had happened to the submodules. Here's a simple script that illustrates the problem: #!/bin/sh rm -rf a b c mkdir a (cd a git init echo a README git add README git commit -am 'a' ) git clone a b (cd b git submodule add ../a submod-1 git commit -am 'add submodule at submod-1' ) git clone b c (cd c git submodule update --init ) (cd b git-submodule-mv.sh submod-1 submod-2 git commit -am 'move submodule from submod-1 to submod-2' ) (cd c git pull ls -d .git/modules/* cat .git/modules/submod-1/config ls -a submod* ) The end result is that `c` gets the `.gitmodules` path updates and new gitlinked directory from the submodule move in `b` (using my git-submodule-mv.sh posted earlier in this thread), but `submod-1` is left lying around (because Git doesn't know that it can remove submod-1/.git and submod-1/README). That's just what current git does with removed submodules (but my recursive submodule update series will handle that just fine). The Git directory for the submodule stays in .git/modules/submod-1/ (good), but the worktree in .git/modules/submod-1/config still points to ../../../submod-1 (bad). You'll not only have to update the gitfile but also the core.worktree setting in the repo. Sorry I missed that when you posted your script. This means that submodule moves are possible, but anyone trying to share them between several repositories (or trying to rebase across the move within their own repository) is in for a world of suffering ;). I'm not sure how this should be addressed, but I didn't see anything handling it in Jens' new series. If you adjust core.worktree properly you'll just have the old submodule work tree lying around (just like you do after you rm'd it) and everything apart from that should just work. As I mentioned that will be fixed by recursive submodule checkout. I'll see if I can polish my preliminary branch so that interested people can play around with it if anyone is interested. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Moving (renaming) submodules, recipe/script
On Tue, Jan 08, 2013 at 06:12:13PM +0100, Jens Lehmann wrote: Am 08.01.2013 15:32, schrieb W. Trevor King: The Git directory for the submodule stays in .git/modules/submod-1/ (good), but the worktree in .git/modules/submod-1/config still points to ../../../submod-1 (bad). You'll not only have to update the gitfile but also the core.worktree setting in the repo. Sorry I missed that when you posted your script. My git-submodule-mv.sh script does update core.worktree. The problem is that `git checkout`, `git merge`, etc. do not. This means that submodule moves are possible, but anyone trying to share them between several repositories (or trying to rebase across the move within their own repository) is in for a world of suffering ;). I'm not sure how this should be addressed, but I didn't see anything handling it in Jens' new series. If you adjust core.worktree properly you'll just have the old submodule work tree lying around (just like you do after you rm'd it) and everything apart from that should just work. As I mentioned that will be fixed by recursive submodule checkout. I'll see if I can polish my preliminary branch so that interested people can play around with it if anyone is interested. Sounds like a fix will be in here. I'll definitely help put the branch through its paces ;). Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
Re: Moving (renaming) submodules, recipe/script
Am 07.01.2013 08:44, schrieb Junio C Hamano: Jens Lehmann jens.lehm...@web.de writes: Am 07.01.2013 02:39, schrieb Jonathan Nieder: (just cc-ing Jens and Peter, who might be interested) I´m currently working on teaching mv to move submodules and intend to send those patches to the list after finishing submodule deinit. Thanks for a heads-up. As a couple of recent What's cooking message has stated, I'll shortly kick jl/submodule-deinit topic out of 'next' back to 'pu', so please make an update a replacement, not an incremental. If I recall the discussion correctly, I think we agreed that deinit should clear the slate, which means the submodule working tree should be removed and made into an empty directory without .git in it, and the last round we saw on the list didn't do that. Right, and me thinks that would warrant a --force option for deinit to do that even if the submodule contains local changes (which would make deinit fail otherwise). Additionally Michael and Marc spoke up that they would rather have a --all option to deinit all initialized submodules and git submodule deinit without any arguments should just produce a usage message. As I saw no voices against it that'll be part of the next iteration too. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Moving (renaming) submodules, recipe/script
Jens Lehmann jens.lehm...@web.de writes: Right, and me thinks that would warrant a --force option for deinit to do that even if the submodule contains local changes (which would make deinit fail otherwise). Probably. Additionally Michael and Marc spoke up that they would rather have a --all option to deinit all initialized submodules and git submodule deinit without any arguments should just produce a usage message. As I saw no voices against it that'll be part of the next iteration too. Yeah, I forgot about that possible surprise of deiniting everything under the sun by default. I am not sure if --all is a good way forward, though. Can you defeat it with git submodule deinit ./--all or something to limit the target only to one submodule whose unfortunate location is named as such? If you have such a support, I have this suspicion that you already get a short and explicit way to say everything under the current directory with git submodule deinit . for free. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Moving (renaming) submodules, recipe/script
Today I had to move my first submodule, and I discovered that Git's support for this is pretty limited. There have been a few patch series attempting to address this [1,2], but none of them seems to have pushed through into master (although I can't put my finger on a reason for why). There are also some SO postings discussing this [3,4]. It would be nice if `git mv` worked out of the box on submodules. Failing that, there could be a `git submodule mv` command that casts the appropriate spell. Failing that, there could be a recipe in Documentation/git-submodule.txt. Here's the best I could come up with for a `git-submodule-mv.sh`: #!/bin/sh # usage: git-submodule-mv.sh OLD NEW OLD=$(realpath --relative-to . $1) NEW=$(realpath --relative-to . $2) SHA=$(git ls-files -s $OLD | sed 's|^[0-9]* \([0-9a-f]*\) .*|\1|') NAME=$(git config -f .gitmodules --get-regexp 'submodule\..*\.path' $OLD | sed -e 's|^submodule.||' -e s|.path $OLD\$||) GITDIR=$(realpath --relative-to $NEW .git/modules/$NAME) git config -f .gitmodules submodule.$NAME.path $NEW git config -f .git/modules/$NAME/config core.worktree ../../../$NEW git rm --cached $OLD mv $OLD $NEW echo gitdir: $GITDIR $NEW/.git git update-index --add --cacheinfo 16 $SHA $NEW This only works from the repository root directory, and I'm sure makes a number of poor assumptions (e.g. old-style submodules that don't use `gitdir` links are not supported). It does work for some simple test cases. The tricky parts (e.g. path - name conversion) are already worked out more robustly git-submodule.sh, so adding a new cmd_mv shouldn't be very difficult. Could something like this live somewhere in Git, or are we waiting for a more integrated solution? Cheers, Trevor [1]: http://thread.gmane.org/gmane.comp.version-control.git/88720 [2]: http://thread.gmane.org/gmane.comp.version-control.git/143250 [4]: http://stackoverflow.com/questions/4323558/moving-submodules-with-git [3]: http://stackoverflow.com/questions/4604486/how-do-i-move-an-existing-git-submodule-within-a-git-repository -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
Re: Moving (renaming) submodules, recipe/script
(just cc-ing Jens and Peter, who might be interested) W. Trevor King wrote: Today I had to move my first submodule, and I discovered that Git's support for this is pretty limited. There have been a few patch series attempting to address this [1,2], but none of them seems to have pushed through into master (although I can't put my finger on a reason for why). There are also some SO postings discussing this [3,4]. It would be nice if `git mv` worked out of the box on submodules. Failing that, there could be a `git submodule mv` command that casts the appropriate spell. Failing that, there could be a recipe in Documentation/git-submodule.txt. Here's the best I could come up with for a `git-submodule-mv.sh`: #!/bin/sh # usage: git-submodule-mv.sh OLD NEW OLD=$(realpath --relative-to . $1) NEW=$(realpath --relative-to . $2) SHA=$(git ls-files -s $OLD | sed 's|^[0-9]* \([0-9a-f]*\) .*|\1|') NAME=$(git config -f .gitmodules --get-regexp 'submodule\..*\.path' $OLD | sed -e 's|^submodule.||' -e s|.path $OLD\$||) GITDIR=$(realpath --relative-to $NEW .git/modules/$NAME) git config -f .gitmodules submodule.$NAME.path $NEW git config -f .git/modules/$NAME/config core.worktree ../../../$NEW git rm --cached $OLD mv $OLD $NEW echo gitdir: $GITDIR $NEW/.git git update-index --add --cacheinfo 16 $SHA $NEW This only works from the repository root directory, and I'm sure makes a number of poor assumptions (e.g. old-style submodules that don't use `gitdir` links are not supported). It does work for some simple test cases. The tricky parts (e.g. path - name conversion) are already worked out more robustly git-submodule.sh, so adding a new cmd_mv shouldn't be very difficult. Could something like this live somewhere in Git, or are we waiting for a more integrated solution? Cheers, Trevor [1]: http://thread.gmane.org/gmane.comp.version-control.git/88720 [2]: http://thread.gmane.org/gmane.comp.version-control.git/143250 [4]: http://stackoverflow.com/questions/4323558/moving-submodules-with-git [3]: http://stackoverflow.com/questions/4604486/how-do-i-move-an-existing-git-submodule-within-a-git-repository -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Moving (renaming) submodules, recipe/script
Am 07.01.2013 02:39, schrieb Jonathan Nieder: (just cc-ing Jens and Peter, who might be interested) I´m currently working on teaching mv to move submodules and intend to send those patches to the list after finishing submodule deinit. Please see https://github.com/jlehmann/git-submod-enhancements/commits/mv-submodules for the current state of this series. W. Trevor King wrote: Today I had to move my first submodule, and I discovered that Git's support for this is pretty limited. There have been a few patch series attempting to address this [1,2], but none of them seems to have pushed through into master (although I can't put my finger on a reason for why). There are also some SO postings discussing this [3,4]. It would be nice if `git mv` worked out of the box on submodules. Failing that, there could be a `git submodule mv` command that casts the appropriate spell. Failing that, there could be a recipe in Documentation/git-submodule.txt. Here's the best I could come up with for a `git-submodule-mv.sh`: #!/bin/sh # usage: git-submodule-mv.sh OLD NEW OLD=$(realpath --relative-to . $1) NEW=$(realpath --relative-to . $2) SHA=$(git ls-files -s $OLD | sed 's|^[0-9]* \([0-9a-f]*\) .*|\1|') NAME=$(git config -f .gitmodules --get-regexp 'submodule\..*\.path' $OLD | sed -e 's|^submodule.||' -e s|.path $OLD\$||) GITDIR=$(realpath --relative-to $NEW .git/modules/$NAME) git config -f .gitmodules submodule.$NAME.path $NEW git config -f .git/modules/$NAME/config core.worktree ../../../$NEW git rm --cached $OLD mv $OLD $NEW echo gitdir: $GITDIR $NEW/.git git update-index --add --cacheinfo 16 $SHA $NEW This only works from the repository root directory, and I'm sure makes a number of poor assumptions (e.g. old-style submodules that don't use `gitdir` links are not supported). It does work for some simple test cases. The tricky parts (e.g. path - name conversion) are already worked out more robustly git-submodule.sh, so adding a new cmd_mv shouldn't be very difficult. Could something like this live somewhere in Git, or are we waiting for a more integrated solution? Cheers, Trevor [1]: http://thread.gmane.org/gmane.comp.version-control.git/88720 [2]: http://thread.gmane.org/gmane.comp.version-control.git/143250 [4]: http://stackoverflow.com/questions/4323558/moving-submodules-with-git [3]: http://stackoverflow.com/questions/4604486/how-do-i-move-an-existing-git-submodule-within-a-git-repository -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Moving (renaming) submodules, recipe/script
Jens Lehmann jens.lehm...@web.de writes: Am 07.01.2013 02:39, schrieb Jonathan Nieder: (just cc-ing Jens and Peter, who might be interested) I´m currently working on teaching mv to move submodules and intend to send those patches to the list after finishing submodule deinit. Thanks for a heads-up. As a couple of recent What's cooking message has stated, I'll shortly kick jl/submodule-deinit topic out of 'next' back to 'pu', so please make an update a replacement, not an incremental. If I recall the discussion correctly, I think we agreed that deinit should clear the slate, which means the submodule working tree should be removed and made into an empty directory without .git in it, and the last round we saw on the list didn't do that. Thanks. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
