AW: Restoring detached HEADs after Git operations
Hello Stefan, I never have tapped into the DLL Hell trap. That's maybe I never did C++ development or I started with VB .NET / C# as .NET solved major parts of the DLL Hell :). That doesn't mean my new beloved language Python doesn't have a similar problem ... Thinking about DLL Hell is a thinking in big version numbers like 1.0, 2.0 oder even 2.1, 2.2, ... We are here talking about revisions in the build numbers which need to be synchronized between the parent repository and the sub modules (IP cores). Both sides are under heavy development and interfaces evolving from day to day because hardware design can't be planned as easy as software design. So by using Git submodules a developer - responsible for a submodule / IP core - can after he finished interface level 1 now go on and implement interface level 2. The parent project can finish it's integration and testing of the level 1 interface before proceeding with level 2. More over if the same IP core is used multiple time in different sub IP cores, it's possible to update one usage place to interface level 2 by a second developer so he can finish his IP core at level 2, which other usage places can still use the level 1 interface. Start situation: -- TOPLEVEL (developer A) o- IP_1 @level1 (developer B) o- IP_2 @level1 (developer C) o- IP_3 @level1 (developer D) o- IP_2 @level1 Developer C creates interface level 2, but all instances use level1 of IP_2: -- TOPLEVEL (developer A) o- IP_1 @level1 (developer B) o- IP_2 @level1 (developer C) o- IP_3 @level1 (developer D) o- IP_2 @level1 Developer D updates instance of IP_2 to level 2 and completes level 2 of IP_3: -- TOPLEVEL (developer A) o- IP_1 @level1 (developer B) o- IP_2 @level1 (developer C) o- IP_3 @level1 (developer D) o- IP_2 @level2 Developer A updates instance of IP_3 to level 2: -- TOPLEVEL (developer A) o- IP_1 @level1 (developer B) o- IP_2 @level1 (developer C) o- IP_3 @level2 (developer D) o- IP_2 @level2 Developer B has finished his testing for IP_1 and can now update the instance if IP_2: -- TOPLEVEL (developer A) o- IP_1 @level1 (developer B) o- IP_2 @level2 (developer C) o- IP_3 @level2 (developer D) o- IP_2 @level2 So now imaging 8 developers, whereof 6 are working remote on the project. There is one responsible developer per IP core (maintainer) and an overall maintainer overseeing all integration merges and test results (CI). Kind regards Patrick Von: Stefan Beller [sbel...@google.com] Gesendet: Montag, 19. Juni 2017 21:21 Bis: Patrick Lehmann Cc: Lars Schneider; Git Mailinglist Betreff: Re: Restoring detached HEADs after Git operations On Mon, Jun 19, 2017 at 11:09 AM, Patrick Lehmann wrote: > Hello Stefan, > > the use case is as follows: > > The projects consists of circa 18 IP cores. Each IP core is represented by a > Git repository. Think of an IP core as of a lonestanding DLL or SO file > project. Each IP core references 2 submodules, which bring the verification > environments for testing the IP core standalone. So phrased differently: You are using submodules to avoid "DLL hell" (sharing a lib, with ease of versioning as the submodules in the different IP cores may be pointing at different versions). > > These 18 IP cores are grouped to bigger IP cores, referencing the low-level > IP cores and each again the 2 verification submodules. Finally, the main > project references the bigger IP cores and again the 2 verification cores. > > TOPLEVEL > o- IP1 >o- UVVM >o- VUnit > o- IP2 >o- UVVM >o- VUnit > o- IP3 >o- UVVM >o- VUnit > o- IP4 >o- UVVM >o- VUnit >o- IP5 >o- UVVM >o- VUnit >o- IP6 >o- UVVM >o- VUnit >o- IP7 >o- UVVM >o- VUnit > o- IP8 >o- UVVM >o- VUnit >o- IP9 >o- UVVM >o- VUnit >o- IP10 >o- UVVM >o- VUnit > o- IP11 >o- UVVM >o- VUnit >o- IP9 >o- UVVM >o- VUnit >o- IP12 >o- UVVM >o- VUnit >o- UVVM >o- VUnit > > That's the simplified structure. I can't write more, because it's a closed > source project. You can find other usecases e.g. in my other open source > projects. E.g. The PoC-Library or The PicoBlaze-Library and the corresponding > PoC-Examples re
Re: Restoring detached HEADs after Git operations
On Mon, Jun 19, 2017 at 11:09 AM, Patrick Lehmann wrote: > Hello Stefan, > > the use case is as follows: > > The projects consists of circa 18 IP cores. Each IP core is represented by a > Git repository. Think of an IP core as of a lonestanding DLL or SO file > project. Each IP core references 2 submodules, which bring the verification > environments for testing the IP core standalone. So phrased differently: You are using submodules to avoid "DLL hell" (sharing a lib, with ease of versioning as the submodules in the different IP cores may be pointing at different versions). > > These 18 IP cores are grouped to bigger IP cores, referencing the low-level > IP cores and each again the 2 verification submodules. Finally, the main > project references the bigger IP cores and again the 2 verification cores. > > TOPLEVEL > o- IP1 >o- UVVM >o- VUnit > o- IP2 >o- UVVM >o- VUnit > o- IP3 >o- UVVM >o- VUnit > o- IP4 >o- UVVM >o- VUnit >o- IP5 >o- UVVM >o- VUnit >o- IP6 >o- UVVM >o- VUnit >o- IP7 >o- UVVM >o- VUnit > o- IP8 >o- UVVM >o- VUnit >o- IP9 >o- UVVM >o- VUnit >o- IP10 >o- UVVM >o- VUnit > o- IP11 >o- UVVM >o- VUnit >o- IP9 >o- UVVM >o- VUnit >o- IP12 >o- UVVM >o- VUnit >o- UVVM >o- VUnit > > That's the simplified structure. I can't write more, because it's a closed > source project. You can find other usecases e.g. in my other open source > projects. E.g. The PoC-Library or The PicoBlaze-Library and the corresponding > PoC-Examples repository. > > Example: PoC > Pile of Cores includes 4 Git submodules and is itself an IP core library. > So PoC-Examples again references PoC. This looks like this tree: > > PoC-Examples > |- lib/ >o- PoC > |- lib > o- Cocotb > o- OSVVM > o- VUnit > o- OSVVM > o- UVVM > > The library VUnit itself already includes OSVVM as a library. > > -- > Forcast: > I'll write a new question / idea about multiple equal submodules and the > memory footprint soon... > Here is my original question posted on StackOverflow: > https://stackoverflow.com/questions/44585425/how-to-reduce-the-memory-footprint-for-multiple-submodules-of-the-same-source > -- > > Do you need more use cases? > Well this use case points out a different issue than I hoped for. ;) >From the stackoverflow post and from looking at the layout here, one of the major questions is how to deduplicate the submodule object store for example. By use case I rather meant a sales pitch for your initial email: I use this bash script because it fits in my workflow because I need branches instead of detached HEADS, because $REASONS and I'd be interested in these $REASONS, which I assumed to be * easier to work with branches than detached HEADS (it aids the workflow) * we're not challenging the underlying mental model of tracking sha1s in the superproject rather than branches. At least I gave these reasons in the "reattach HEAD" stuff that I wrote, but maybe there are others? (I know the code base of submodules very well, but I do not work with submodules on a day-to-day basis myself...)
Re: Restoring detached HEADs after Git operations
On Mon, Jun 19, 2017 at 10:55 AM, Junio C Hamano wrote: > Stefan Beller writes: > >> On Mon, Jun 19, 2017 at 2:52 AM, Patrick Lehmann >> wrote: >>> Hello Lars, >>> >>> for your questions: If there are multiple branches with the same hash then your script would pick the first one. Can you imagine a situation where this would be a problem? >>> >>> I can't think of a good solution to resolve it automatically. Maybe a >>> script could print that there are multiple possibilities and it choose the >>> first branch in the list. >>> >>> Plus, you are looking only at local branches. Wouldn't it make sense to look at remote branches, too? >>> >>> This is also related to restoring tags. If we go this way, we should have >>> this priority list: >>> - local branches >>> - remote branches >> >> For remote branches you would create a local branch of the same name >> (if such a branch would not exist, possibly setting it up to track that >> remote >> branch)? >> >>> - tags >> >> as said in the other email and similar to remote branches, we'd not want to >> have >> HEAD pointing to them directly but somehow have a local branch. > > Let's step back a bit. We detach the HEAD for a good reason, no? And the 'good reason' being that at the time git-submodule was written we did not know what would be best, and having a detached HEAD would be (a) easy to implement, and (b) removing one moving thing from the whole construction, hence making it a bit safer, (c) it sort of follows the mental model: the superproject said it had the submodule at X (and not at branch Y!) the submodule itself is a whole repo on its own (it doesn't need to be aware of the superproject) so in this world detaching at X is the best we can do. > Why is it a good idea to move them back on to a branch picked among > multiple ones that all happen to be pointing at the same commit? This (rhetorical?) question reads like 2 questions actually: (a) "Why is it a good idea to move them back on to a branch?" It makes working easier as the submodule is not detached, but on a proper branch (b) "picked among multiple ones that all ..." I think this is a bad idea and we'd rather want to follow some configuration instead of wild-guessing by Git. > The user may build on a history of a submodule, and then may push > the result out to a particular branch at the other side; that is > when being on a named branch in the submodule becomes useful, but > even then I do not think randomly picking one branch and be on it > is a good thing to do. so you provide one reason why it is useful, but then claiming it is 'not a good thing' (yet). Can you give a reason why this is a 'bad thing'? > I would understand the workflow would go more like so: > > - You do something at the superproject (e.g. create a new branch X >from an existing commit and check it out), which results in >submodules' HEADs getting detached at the commits bound to the >superproject's tree. And here we'd want to discuss if we *really* want to detach the HEADs or rather have a symbolic ref "following the superproject". > - Because you want to make changes to both submodules and the >superproject in a consistent way, you'd want to commit changes to >all of these repositories and the push the result out in an >atomic way. Committing and pushing are different things. You should not care if I commit atomically as you (in the general "upstream" sense) cannot observe my local commits. For pushing we would want to have an atomic push, but that is not the scope of this discussion. (As a Gerrit user, we implemented the submodule atomicity serverside, but in plain Git server you'd not need the atomicity either: git commit -a -m "update submodule pointers" git submodule foreach git push git push should be fine w.r.t. any non-atomic race condition.) > - Hence you tell "Hey, Git, I want all the submodules that I >modified to be on branch X" from the superproject. > >- This may succeed in a submodule where X is a new name, or the > current tip of branch X is an ancestor of the detached HEAD. so we'd allow fast forward for X. This seems arbitrary to me. I could also say "If X exists I allow a merge to be made between old X and the object name given by the superproject". (maybe as a config option) >- This may fail in a submodule where there is branch X that does > not want to move to the detached HEAD's state. In this latter > case, the user needs to deal with the situation (perhaps the > old X is expendable; perhaps the HEAD's commit may need to be > merged to old X; perhaps there are other cases). makes sense. > > though.
AW: Restoring detached HEADs after Git operations
Hello Stefan, the use case is as follows: The projects consists of circa 18 IP cores. Each IP core is represented by a Git repository. Think of an IP core as of a lonestanding DLL or SO file project. Each IP core references 2 submodules, which bring the verification environments for testing the IP core standalone. These 18 IP cores are grouped to bigger IP cores, referencing the low-level IP cores and each again the 2 verification submodules. Finally, the main project references the bigger IP cores and again the 2 verification cores. TOPLEVEL o- IP1 o- UVVM o- VUnit o- IP2 o- UVVM o- VUnit o- IP3 o- UVVM o- VUnit o- IP4 o- UVVM o- VUnit o- IP5 o- UVVM o- VUnit o- IP6 o- UVVM o- VUnit o- IP7 o- UVVM o- VUnit o- IP8 o- UVVM o- VUnit o- IP9 o- UVVM o- VUnit o- IP10 o- UVVM o- VUnit o- IP11 o- UVVM o- VUnit o- IP9 o- UVVM o- VUnit o- IP12 o- UVVM o- VUnit o- UVVM o- VUnit That's the simplified structure. I can't write more, because it's a closed source project. You can find other usecases e.g. in my other open source projects. E.g. The PoC-Library or The PicoBlaze-Library and the corresponding PoC-Examples repository. Example: PoC Pile of Cores includes 4 Git submodules and is itself an IP core library. So PoC-Examples again references PoC. This looks like this tree: PoC-Examples |- lib/ o- PoC |- lib o- Cocotb o- OSVVM o- VUnit o- OSVVM o- UVVM The library VUnit itself already includes OSVVM as a library. -- Forcast: I'll write a new question / idea about multiple equal submodules and the memory footprint soon... Here is my original question posted on StackOverflow: https://stackoverflow.com/questions/44585425/how-to-reduce-the-memory-footprint-for-multiple-submodules-of-the-same-source -- Do you need more use cases? Kind regards Patrick Von: git-ow...@vger.kernel.org [git-ow...@vger.kernel.org]" im Auftrag von "Stefan Beller [sbel...@google.com] Gesendet: Montag, 19. Juni 2017 19:47 Bis: Patrick Lehmann Cc: Lars Schneider; Git Mailinglist Betreff: Re: Restoring detached HEADs after Git operations On Mon, Jun 19, 2017 at 10:34 AM, Patrick Lehmann wrote: > Hello, > > I'm just an advanced Git user, not a Git developer. So I might find some time > to improve the suggested script, which I provided with the hints given on the > mailing list, but I have no time to do a complete feature release in your > patch based Git flow. ok, thanks for letting us know. I may re-prioritize the "reattach HEAD" patches that I referenced earlier. I would have hoped that additionally to the shell lines you'd have given a good use case/summary. > I have no experience with other shells then Bash. So if you rely on a Bash > with less features, please port the syntax to such a shell system. (I > personally do not support legacy programs or out-date programs). > > -- > We are talking about circa 50 submodules in total with a maximum depth of 4. > The platforms are: > - Mint OS with Git in Bash > - Windows 7 with Git-Bash > - Windows 10 with Git-Bash > - Windows 10 with Posh-Git Thanks, Stefan
Re: Restoring detached HEADs after Git operations
On Mon, Jun 19 2017, Patrick Lehmann jotted: > Hello, > > I wrote a Bash script to recover branch names after Git operations have > create detached HEADs in a Git repository containing lots of Git submodules. > The script works recursively. > > I would like to see: > a) that script or algorithm being integrated into Git by default > b) that as a default behavior for all Git operations creating detached HEADs > > That's the command: > > git submodule foreach --recursive 'HEAD=$(git branch --list | head -n 1); if > [[ "$HEAD" == *HEAD* ]]; then REF=$(git rev-parse HEAD); FOUND=0; for Branch > in $(git branch --list | grep "^ " | sed -e "s/ //" ); do if [[ "$(git > rev-parse "$Branch")" == $REF ]]; then echo -e " \e[36mCheckout > $Branch...\e[0m"; git checkout $Branch; FOUND=1; break; fi done; if [[ $FOUND > -eq 0 ]]; then echo -e " \e[31mNo matching branch found.\e[0m"; fi else echo > -e " \e[36mNothing to do.\e[0m"; fi' > > > How does it work: > 1. It uses git submodule foreach to dive into each Git submodule and execute > a series of Bash commands. > 2. It's reading the list of branches and checks if the submodule is in > detached mode. The first line contains the string HEAD. > 3. Retrieve the hash of the detached HEAD > 4. Iterate all local branches and get their hashes > 5. Compare the branch hashes with the detached HEAD's hash. If it matches do > a checkout. > 6. Report if no branch name was found or if a HEAD was not in detached mode. > > The Bash code with line breaks and indentation: > > HEAD=$(git branch --list | head -n 1) > if [[ "$HEAD" == *HEAD* ]]; then > REF=$(git rev-parse HEAD) > FOUND=0 > for Branch in $(git branch --list | grep "^ " | sed -e "s/ //" ); do > if [[ "$(git rev-parse "$Branch")" == $REF ]]; then > echo -e " \e[36mCheckout $Branch...\e[0m" > git checkout $Branch > FOUND=1 > break > fi > done > if [[ $FOUND -eq 0 ]]; then > echo -e " \e[31mNo matching branch found.\e[0m" > fi > else > echo -e " \e[36mNothing to do.\e[0m" > fi > > > Are their any chances to get it integrated into Git? > > I tried to register that code as a Git alias, but git config complains about > quote problem not showing where. It neither specifies if it's a single or > double quote problem. Any advice on how to register that piece of code as an > alias? > > If wished, I think I could expand the script to also recover hash values to > Git tags if no branch was found. I have something similar to this, this written before git-submodule learned --branch (or at least before I knew about it): $ git config alias.sm-pull-all !git submodule foreach 'git checkout $(NAME=$name git sm-mainbranch) && git pull' $ git config alias.sm-mainbranch !git config --file ../.gitmodules submodule.$NAME.branch || git describe --all --always | sed 's!^heads/!!' So with this I can run `git sm-pull-all` to update all the submodules in a superproject to update them all. This relies on the branch name being in the .gitmodules config.a Now if you add a module with e.g. 'git submodule add -b master ...' you'll have it checked out at the master branch, but I'm not familiar enough with all the workflows around them to say if there are any holes in that implementation. But that seems like a fundimentally better approach to me than what you're suggesting. Why would we try to work our way back from the SHA-1 and guess what branch it lives on when we can just encode the branch name we'd like to be on in the superproject?
Re: Restoring detached HEADs after Git operations
Stefan Beller writes: > On Mon, Jun 19, 2017 at 2:52 AM, Patrick Lehmann > wrote: >> Hello Lars, >> >> for your questions: >>> If there are multiple branches with the same hash then your script would >>> pick the first one. Can you imagine a situation where this would be a >>> problem? >> >> I can't think of a good solution to resolve it automatically. Maybe a script >> could print that there are multiple possibilities and it choose the first >> branch in the list. >> >> >>> Plus, you are looking only at local branches. Wouldn't it make sense to >>> look at remote branches, too? >> >> This is also related to restoring tags. If we go this way, we should have >> this priority list: >> - local branches >> - remote branches > > For remote branches you would create a local branch of the same name > (if such a branch would not exist, possibly setting it up to track that remote > branch)? > >> - tags > > as said in the other email and similar to remote branches, we'd not want to > have > HEAD pointing to them directly but somehow have a local branch. Let's step back a bit. We detach the HEAD for a good reason, no? Why is it a good idea to move them back on to a branch picked among multiple ones that all happen to be pointing at the same commit? The user may build on a history of a submodule, and then may push the result out to a particular branch at the other side; that is when being on a named branch in the submodule becomes useful, but even then I do not think randomly picking one branch and be on it is a good thing to do. I would understand the workflow would go more like so: - You do something at the superproject (e.g. create a new branch X from an existing commit and check it out), which results in submodules' HEADs getting detached at the commits bound to the superproject's tree. - Because you want to make changes to both submodules and the superproject in a consistent way, you'd want to commit changes to all of these repositories and the push the result out in an atomic way. - Hence you tell "Hey, Git, I want all the submodules that I modified to be on branch X" from the superproject. - This may succeed in a submodule where X is a new name, or the current tip of branch X is an ancestor of the detached HEAD. - This may fail in a submodule where there is branch X that does not want to move to the detached HEAD's state. In this latter case, the user needs to deal with the situation (perhaps the old X is expendable; perhaps the HEAD's commit may need to be merged to old X; perhaps there are other cases). though.
Re: Restoring detached HEADs after Git operations
On Mon, Jun 19, 2017 at 10:34 AM, Patrick Lehmann wrote: > Hello, > > I'm just an advanced Git user, not a Git developer. So I might find some time > to improve the suggested script, which I provided with the hints given on the > mailing list, but I have no time to do a complete feature release in your > patch based Git flow. ok, thanks for letting us know. I may re-prioritize the "reattach HEAD" patches that I referenced earlier. I would have hoped that additionally to the shell lines you'd have given a good use case/summary. > I have no experience with other shells then Bash. So if you rely on a Bash > with less features, please port the syntax to such a shell system. (I > personally do not support legacy programs or out-date programs). > > -- > We are talking about circa 50 submodules in total with a maximum depth of 4. > The platforms are: > - Mint OS with Git in Bash > - Windows 7 with Git-Bash > - Windows 10 with Git-Bash > - Windows 10 with Posh-Git Thanks, Stefan
AW: Restoring detached HEADs after Git operations
Hello, I'm just an advanced Git user, not a Git developer. So I might find some time to improve the suggested script, which I provided with the hints given on the mailing list, but I have no time to do a complete feature release in your patch based Git flow. I'm currently involved in 8 other open source projects. One can't improve the world alone by supplying patches to any open source project one is using... I have no experience with other shells then Bash. So if you rely on a Bash with less features, please port the syntax to such a shell system. (I personally do not support legacy programs or out-date programs). -- We are talking about circa 50 submodules in total with a maximum depth of 4. The platforms are: - Mint OS with Git in Bash - Windows 7 with Git-Bash - Windows 10 with Git-Bash - Windows 10 with Posh-Git Kind regards Patrick Von: Stefan Beller [sbel...@google.com] Gesendet: Montag, 19. Juni 2017 18:37 Bis: Patrick Lehmann Cc: Lars Schneider; Git Mailinglist Betreff: Re: Restoring detached HEADs after Git operations On Mon, Jun 19, 2017 at 2:52 AM, Patrick Lehmann wrote: > Hello Lars, > > for your questions: >> If there are multiple branches with the same hash then your script would >> pick the first one. Can you imagine a situation where this would be a >> problem? > > I can't think of a good solution to resolve it automatically. Maybe a script > could print that there are multiple possibilities and it choose the first > branch in the list. > > >> Plus, you are looking only at local branches. Wouldn't it make sense to look >> at remote branches, too? > > This is also related to restoring tags. If we go this way, we should have > this priority list: > - local branches > - remote branches For remote branches you would create a local branch of the same name (if such a branch would not exist, possibly setting it up to track that remote branch)? > - tags as said in the other email and similar to remote branches, we'd not want to have HEAD pointing to them directly but somehow have a local branch. >> Submodule processing is already quite slow if you have many of them. I >> wonder how much this approach would affect the performance. > > Yes. It takes a few seconds to iterate all the submodules. It could be > improved if the processing wouldn't be based on slow Bash scripts spawning > lot's of sub-shells to execute multiple Git commands. How many submodules are we talking about? (Are you on Windows to make shell even more fun?)
Re: Restoring detached HEADs after Git operations
On Mon, Jun 19, 2017 at 08:46:45AM +, Patrick Lehmann wrote: > for Branch in $(git branch --list | grep "^ " | sed -e "s/ //" ); do > if [[ "$(git rev-parse "$Branch")" == $REF ]]; then > echo -e " \e[36mCheckout $Branch...\e[0m" > git checkout $Branch > FOUND=1 > break > fi > done I see Lars pointed you at for-each-ref, which is the preferred way for scripts to enumerate branches. But you can also use --points-at to skip the inner part of your loop entirely, like: git for-each-ref --points-at=HEAD --format='%(refname:short)' refs/heads That should run much faster, as it only has to spawn one process rather than one for each branch (it will print all of them, of course. You can pipe through head -n 1 to get the first, and check out --sort if you want to prioritize by recency or similar). -Peff
Re: Restoring detached HEADs after Git operations
On Mon, Jun 19, 2017 at 2:52 AM, Patrick Lehmann wrote: > Hello Lars, > > for your questions: >> If there are multiple branches with the same hash then your script would >> pick the first one. Can you imagine a situation where this would be a >> problem? > > I can't think of a good solution to resolve it automatically. Maybe a script > could print that there are multiple possibilities and it choose the first > branch in the list. > > >> Plus, you are looking only at local branches. Wouldn't it make sense to look >> at remote branches, too? > > This is also related to restoring tags. If we go this way, we should have > this priority list: > - local branches > - remote branches For remote branches you would create a local branch of the same name (if such a branch would not exist, possibly setting it up to track that remote branch)? > - tags as said in the other email and similar to remote branches, we'd not want to have HEAD pointing to them directly but somehow have a local branch. >> Submodule processing is already quite slow if you have many of them. I >> wonder how much this approach would affect the performance. > > Yes. It takes a few seconds to iterate all the submodules. It could be > improved if the processing wouldn't be based on slow Bash scripts spawning > lot's of sub-shells to execute multiple Git commands. How many submodules are we talking about? (Are you on Windows to make shell even more fun?)
Re: Restoring detached HEADs after Git operations
On Mon, Jun 19, 2017 at 1:46 AM, Patrick Lehmann
wrote:
> Hello,
>
> I wrote a Bash script to recover branch names after Git operations have
> create detached HEADs in a Git repository containing lots of Git submodules.
> The script works recursively.
Cool. :)
You may also like
https://public-inbox.org/git/20170501180058.8063-5-sbel...@google.com/
https://public-inbox.org/git/20170501180058.8063-6-sbel...@google.com/
These patches are still on my plate, they are not landed yet as I had issues
coming up with a good convincing commit message.
They are essentially putting submodules back on a branch (if configured).
Let's see how this differs from your solution.
> I would like to see:
> a) that script or algorithm being integrated into Git by default
For that you'd want to send a patch, see Documentation/SubmittingPatches.
We'd want to discuss if this command is an independent command
("git submodule reattachHEADs", name subject to bikeshedding ;) )
or if it is a configurable option that is obeyed by anything that touches
submodules (which I would prefer, as this mode seems to be the
"correct default". When having it as a mode we can switch the default
eventually such that submodules are always on a branch).
> b) that as a default behavior for all Git operations creating detached HEADs
changing defaults is hard. Let's go with a) first and then people will
report how
awesome the new mode/command is and then it is easier to see how this
may be a good default. :)
>
> That's the command:
>
(reformatted for readability:)
git submodule foreach --recursive
'HEAD=$(git branch --list | head -n 1);
if [[ "$HEAD" == *HEAD* ]]; then
REF=$(git rev-parse HEAD);
FOUND=0;
for Branch in $(git branch --list | grep "^ " | sed -e "s/ //" );
do
if [[ "$(git rev-parse "$Branch")" == $REF ]]; then
echo -e " \e[36mCheckout $Branch...\e[0m";
git checkout $Branch;
FOUND=1;
break;
fi
done;
if [[ $FOUND -eq 0 ]]; then
echo -e " \e[31mNo matching branch found.\e[0m";
fi
else
echo -e " \e[36mNothing to do.\e[0m";
fi'
>
>
> How does it work:
> 1. It uses git submodule foreach to dive into each Git submodule and execute
> a series of Bash commands.
If you want to see it upstream eventually, we'd make it shell commands.
There are some subtle differences between shell and bash,
one of them is the way conditions are written. I think plain shell
does not support [[ ]], so that would become
if test $FOUND -eq 0
then
echo ...
Maybe look at git-submodule.sh for coding style suggestions.
> 2. It's reading the list of branches and checks if the submodule is in
> detached mode. The first line contains the string HEAD.
This works for you but some crazy person may have a branch containing
HEAD in their branch name. ;)
("git checkout -b notADetachedHEAD")
I think that check can be improved via
if test $(git symbolic-ref HEAD 2>/dev/null >/dev/null) -eq 128
then
# detached HEAD
else
# on a branch
fi
so if the output of symbolic-ref starts with ref then it is on a
branch. In detached HEAD
> 3. Retrieve the hash of the detached HEAD
> 4. Iterate all local branches and get their hashes
What happens (/should happen) when multiple branches have the same sha1?
With this implementation the first wins? Is this 'lazy guessing' desired?
The patches referenced above assumed you'd have submodule.NAME.branch
set and we'd reattach to that branch only (if matching hashes)
> 5. Compare the branch hashes with the detached HEAD's hash. If it matches do
> a checkout.
Speaking of checkout: checkout --recurse-submodules is a
thing in the latest version of Git, but it also detaches HEADs.
I'd like to have reattaching HEADs in there and then combined with
"git config submodule.recurse true", which is in master but no release
a plain "git checkout " in the superproject would put the submodules
on branches.
Using checkout within git submodule-foreach works of course just as fine.
Note: Currently Prathamesh Chavan converts git-submodule-foreach to C
https://public-inbox.org/git/CAME+mvUrzVxpRdPDvA1ZyatNm2R27QGJVjSB3=kx85ceedm...@mail.gmail.com/
so it will be faster. In the process of doing so, we surfaced a couple
of bugs, but
they would not impact this script AFAICT.
> 6. Report if no branch name was found or if a HEAD was not in detached mode.
... and it is colored unconditionally in red. Maybe have a look at
git config --get-color[bool]
which can help in figuring out if we want to print color codes.
> The Bash code with line breaks and indentation:
>
> HEAD=$(git branch --list | head -n 1)
> if [[ "$HEAD" == *HEAD* ]]; then
> REF=$(git rev-parse HEAD)
> FOUND=0
> for Branch in $(git branch --list | grep "^ " | sed -e "s/ //" ); do
> if [[ "$(git rev-parse "
AW: Restoring detached HEADs after Git operations
Hello Lars, for your questions: > If there are multiple branches with the same hash then your script would pick > the first one. Can you imagine a situation where this would be a problem? I can't think of a good solution to resolve it automatically. Maybe a script could print that there are multiple possibilities and it choose the first branch in the list. > Plus, you are looking only at local branches. Wouldn't it make sense to look > at remote branches, too? This is also related to restoring tags. If we go this way, we should have this priority list: - local branches - remote branches - tags > Submodule processing is already quite slow if you have many of them. I wonder > how much this approach would affect the performance. Yes. It takes a few seconds to iterate all the submodules. It could be improved if the processing wouldn't be based on slow Bash scripts spawning lot's of sub-shells to execute multiple Git commands. Is there a way to avoid detached DEADs at the beginning? Many submodules are attached to a reference and get detached to a hash of the same reference. It would be better, if they never get detached when the current and new hash are the same. Kind regards Patrick Von: git-ow...@vger.kernel.org [git-ow...@vger.kernel.org]" im Auftrag von "Lars Schneider [larsxschnei...@gmail.com] Gesendet: Montag, 19. Juni 2017 11:30 Bis: Patrick Lehmann Cc: Git Mailinglist; Stefan Beller Betreff: Re: Restoring detached HEADs after Git operations > On 19 Jun 2017, at 10:46, Patrick Lehmann wrote: > > Hello, > > I wrote a Bash script to recover branch names after Git operations have > create detached HEADs in a Git repository containing lots of Git submodules. > The script works recursively. I did run into this situation myself and therefore I understand your motivation. I've CC'ed Stefan as he is a Submodule expert! > I would like to see: > a) that script or algorithm being integrated into Git by default > b) that as a default behavior for all Git operations creating detached HEADs > > That's the command: > > git submodule foreach --recursive 'HEAD=$(git branch --list | head -n 1); if > [[ "$HEAD" == *HEAD* ]]; then REF=$(git rev-parse HEAD); FOUND=0; for Branch > in $(git branch --list | grep "^ " | sed -e "s/ //" ); do if [[ "$(git > rev-parse "$Branch")" == $REF ]]; then echo -e " \e[36mCheckout > $Branch...\e[0m"; git checkout $Branch; FOUND=1; break; fi done; if [[ $FOUND > -eq 0 ]]; then echo -e " \e[31mNo matching branch found.\e[0m"; fi else echo > -e " \e[36mNothing to do.\e[0m"; fi' > > > How does it work: > 1. It uses git submodule foreach to dive into each Git submodule and execute > a series of Bash commands. > 2. It's reading the list of branches and checks if the submodule is in > detached mode. The first line contains the string HEAD. > 3. Retrieve the hash of the detached HEAD > 4. Iterate all local branches and get their hashes > 5. Compare the branch hashes with the detached HEAD's hash. If it matches do > a checkout. If there are multiple branches with the same hash then your script would pick the first one. Can you imagine a situation where this would be a problem? Plus, you are looking only at local branches. Wouldn't it make sense to look at remote branches, too? > 6. Report if no branch name was found or if a HEAD was not in detached mode. > > The Bash code with line breaks and indentation: > > HEAD=$(git branch --list | head -n 1) > if [[ "$HEAD" == *HEAD* ]]; then > REF=$(git rev-parse HEAD) > FOUND=0 > for Branch in $(git branch --list | grep "^ " | sed -e "s/ //" ); do There is a convenient "git for-each-ref" function to iterate over branches in scripts. See here an example: https://github.com/larsxschneider/scotty/blob/master/admin/oss-fork.sh#L88 >if [[ "$(git rev-parse "$Branch")" == $REF ]]; then > echo -e " \e[36mCheckout $Branch...\e[0m" > git checkout $Branch > FOUND=1 > break >fi > done > if [[ $FOUND -eq 0 ]]; then >echo -e " \e[31mNo matching branch found.\e[0m" > fi > else > echo -e " \e[36mNothing to do.\e[0m" > fi > > > Are their any chances to get it integrated into Git? > > I tried to register that code as a Git alias, but git config complains about > quote problem not showing where. It neither specifies if it's a single or > double quote problem. Any advice
Re: Restoring detached HEADs after Git operations
> On 19 Jun 2017, at 10:46, Patrick Lehmann wrote: > > Hello, > > I wrote a Bash script to recover branch names after Git operations have > create detached HEADs in a Git repository containing lots of Git submodules. > The script works recursively. I did run into this situation myself and therefore I understand your motivation. I've CC'ed Stefan as he is a Submodule expert! > I would like to see: > a) that script or algorithm being integrated into Git by default > b) that as a default behavior for all Git operations creating detached HEADs > > That's the command: > > git submodule foreach --recursive 'HEAD=$(git branch --list | head -n 1); if > [[ "$HEAD" == *HEAD* ]]; then REF=$(git rev-parse HEAD); FOUND=0; for Branch > in $(git branch --list | grep "^ " | sed -e "s/ //" ); do if [[ "$(git > rev-parse "$Branch")" == $REF ]]; then echo -e " \e[36mCheckout > $Branch...\e[0m"; git checkout $Branch; FOUND=1; break; fi done; if [[ $FOUND > -eq 0 ]]; then echo -e " \e[31mNo matching branch found.\e[0m"; fi else echo > -e " \e[36mNothing to do.\e[0m"; fi' > > > How does it work: > 1. It uses git submodule foreach to dive into each Git submodule and execute > a series of Bash commands. > 2. It's reading the list of branches and checks if the submodule is in > detached mode. The first line contains the string HEAD. > 3. Retrieve the hash of the detached HEAD > 4. Iterate all local branches and get their hashes > 5. Compare the branch hashes with the detached HEAD's hash. If it matches do > a checkout. If there are multiple branches with the same hash then your script would pick the first one. Can you imagine a situation where this would be a problem? Plus, you are looking only at local branches. Wouldn't it make sense to look at remote branches, too? > 6. Report if no branch name was found or if a HEAD was not in detached mode. > > The Bash code with line breaks and indentation: > > HEAD=$(git branch --list | head -n 1) > if [[ "$HEAD" == *HEAD* ]]; then > REF=$(git rev-parse HEAD) > FOUND=0 > for Branch in $(git branch --list | grep "^ " | sed -e "s/ //" ); do There is a convenient "git for-each-ref" function to iterate over branches in scripts. See here an example: https://github.com/larsxschneider/scotty/blob/master/admin/oss-fork.sh#L88 >if [[ "$(git rev-parse "$Branch")" == $REF ]]; then > echo -e " \e[36mCheckout $Branch...\e[0m" > git checkout $Branch > FOUND=1 > break >fi > done > if [[ $FOUND -eq 0 ]]; then >echo -e " \e[31mNo matching branch found.\e[0m" > fi > else > echo -e " \e[36mNothing to do.\e[0m" > fi > > > Are their any chances to get it integrated into Git? > > I tried to register that code as a Git alias, but git config complains about > quote problem not showing where. It neither specifies if it's a single or > double quote problem. Any advice on how to register that piece of code as an > alias? Try to escape ". See here for an example: https://github.com/Autodesk/enterprise-config-for-git/blob/master/config.include#L76-L94 > If wished, I think I could expand the script to also recover hash values to > Git tags if no branch was found. It would be indeed nice to see the tagged version on my prompt. -- Submodule processing is already quite slow if you have many of them. I wonder how much this approach would affect the performance. - Lars
Restoring detached HEADs after Git operations
Hello, I wrote a Bash script to recover branch names after Git operations have create detached HEADs in a Git repository containing lots of Git submodules. The script works recursively. I would like to see: a) that script or algorithm being integrated into Git by default b) that as a default behavior for all Git operations creating detached HEADs That's the command: git submodule foreach --recursive 'HEAD=$(git branch --list | head -n 1); if [[ "$HEAD" == *HEAD* ]]; then REF=$(git rev-parse HEAD); FOUND=0; for Branch in $(git branch --list | grep "^ " | sed -e "s/ //" ); do if [[ "$(git rev-parse "$Branch")" == $REF ]]; then echo -e " \e[36mCheckout $Branch...\e[0m"; git checkout $Branch; FOUND=1; break; fi done; if [[ $FOUND -eq 0 ]]; then echo -e " \e[31mNo matching branch found.\e[0m"; fi else echo -e " \e[36mNothing to do.\e[0m"; fi' How does it work: 1. It uses git submodule foreach to dive into each Git submodule and execute a series of Bash commands. 2. It's reading the list of branches and checks if the submodule is in detached mode. The first line contains the string HEAD. 3. Retrieve the hash of the detached HEAD 4. Iterate all local branches and get their hashes 5. Compare the branch hashes with the detached HEAD's hash. If it matches do a checkout. 6. Report if no branch name was found or if a HEAD was not in detached mode. The Bash code with line breaks and indentation: HEAD=$(git branch --list | head -n 1) if [[ "$HEAD" == *HEAD* ]]; then REF=$(git rev-parse HEAD) FOUND=0 for Branch in $(git branch --list | grep "^ " | sed -e "s/ //" ); do if [[ "$(git rev-parse "$Branch")" == $REF ]]; then echo -e " \e[36mCheckout $Branch...\e[0m" git checkout $Branch FOUND=1 break fi done if [[ $FOUND -eq 0 ]]; then echo -e " \e[31mNo matching branch found.\e[0m" fi else echo -e " \e[36mNothing to do.\e[0m" fi Are their any chances to get it integrated into Git? I tried to register that code as a Git alias, but git config complains about quote problem not showing where. It neither specifies if it's a single or double quote problem. Any advice on how to register that piece of code as an alias? If wished, I think I could expand the script to also recover hash values to Git tags if no branch was found. Kind regards Patrick Lehmann
