Re: Re: Re: Relative submodule URLs
On Mon, Aug 25, 2014 at 09:29:07AM -0500, Robert Dailey wrote: On Sun, Aug 24, 2014 at 8:34 AM, Heiko Voigt hvo...@hvoigt.net wrote: New --with--remote parameter for 'git submodule' While having said all that about submodule settings I think a much much simpler start is to go ahead with a commandline setting, like Robert proposed here[2]. For that we do not have to worry about how it can be stored, transported, defined per submodule or on a branch, since answers to this are given at the commandline (and current repository state). There are still open questions about this though: * Should the name in the submodule be 'origin' even though you specified --with-remote=somewhere? For me its always confusing to have the same/similar remotes named differently in different repositories. That why I try to keep the names the same in all my clones of repositories (i.e. for my private, github, upstream remotes). * When you do a 'git submodule sync --with-remote=somewhere' should the remote be added or replaced. My opinion on these are: The remote should be named as in the superproject so --with-remote=somewhere adds/replaces the remote 'somewhere' in the submodules named on the commandline (or all in case no submodule is specified). In case of a fresh clone of the submodule, there would be no origin but only a remote under the new name. Would the --with-remote feature I describe be a feasible start for you Robert? What do others think? Is the naming of the parameter '--with-remote' alright? Cheers Heiko [1] http://article.gmane.org/gmane.comp.version-control.git/255512 [2] http://article.gmane.org/gmane.comp.version-control.git/255512 [3] https://github.com/jlehmann/git-submod-enhancements/wiki#special-ref-overriding-gitmodules-values Hi Heiko, My last email response was in violation of your request to keep the two topics separate, sorry about that. I started typing it this weekend and completed the draft this morning, without having read this response from you first. Thats fine, no problem. Here is what I think would make the feature most usable. I think you went over some of these ideas but I just want to clarify, to make sure we're on the same page. Please correct me as needed. 1. Running `git submodule update --with-remote name` shall fail the command unconditionally. I am not sure but I think you mean git submodule update --with-remote=name With the equals sign, without it you would name the submodule paths to update. No I think that should just add the remote name to all submodules that would be updated and do the normal update operation on them (with the new remote of course). 2. Using the `--with-remote` option on submodule `update` or `sync` will fail if it detects absolute submodule URLs in .gitmodule Yes, almost. Since you can have a mixture I suggest to only fail if the submodules that would be processed have an absolute url in them. If processed submodules are all relative it can go ahead. 3. Running `git submodule update --init --with-remote name` shall fail the command ONLY if a submodule is being processed that is NOT also being initialized. No since the --init flag just tells update to initialize submodules on-demand. It should just go ahead the same way as without --with-remote. 4. The behavior of git submodule's `update` or `sync` commands combined with `--with-remote` will REPLACE or CREATE the 'origin' remote in each submodule it is run in. We will not allow the user to configure what the submodule remote name will end up being (I think this is current behavior and forces good practice; I consider `origin` an adopted standard for git, and actually wish it was more enforced for super projects as well!) No please carefully read my email again. I specifically was describing the opposite. --with-remote=name creates/replaces the remote name in the submodule. I do not see a benefit in restricting the user from creating different remote names in the submodule. I think it would be more confusing if the remote 'origin' in the superproject does not point to the same location as 'origin' in the submodule. Let me know if I've missed anything. Once we clarify requirements I'll attempt to start work on this during my free time. I'll start by testing this through msysgit, since I do not have linux installed, but I have Linux Mint running in a Virtual Machine so I can test on both platforms as needed (I don't have a lot of experience on Linux though). I think it does not matter which development environment you use. In my experience though Linux is around 30x faster when it comes to the typical operations you do when developing git. Especially for running the testsuite that makes a difference between a few hours and minutes. I hope you won't mind me reaching out for questions as needed, however I
Re: Re: Re: Relative submodule URLs
On Tue, Aug 26, 2014 at 1:28 AM, Heiko Voigt hvo...@hvoigt.net wrote: Hi Heiko, My last email response was in violation of your request to keep the two topics separate, sorry about that. I started typing it this weekend and completed the draft this morning, without having read this response from you first. Thats fine, no problem. Here is what I think would make the feature most usable. I think you went over some of these ideas but I just want to clarify, to make sure we're on the same page. Please correct me as needed. 1. Running `git submodule update --with-remote name` shall fail the command unconditionally. I am not sure but I think you mean git submodule update --with-remote=name With the equals sign, without it you would name the submodule paths to update. No I think that should just add the remote name to all submodules that would be updated and do the normal update operation on them (with the new remote of course). I'm not sure about Linux but at least with msysgit on Windows, typing a two-dash option (such as --with-remote) forces command line evaluation to use the next placement parameter as the parameter for it. I've seen this work the same way with argparse in python too. In my experience, command line has worked that way, I'm not sure if that is by design or not though. I never use equal signs with git commands, never had a problem for some reason. For example: git rebase --onto release/1.0 head~3 head The `--onto` option knows to use `release/1.0` as its parameter. 2. Using the `--with-remote` option on submodule `update` or `sync` will fail if it detects absolute submodule URLs in .gitmodule Yes, almost. Since you can have a mixture I suggest to only fail if the submodules that would be processed have an absolute url in them. If processed submodules are all relative it can go ahead. For example if it processes 3 submodules in the following order: 1. relative 2. absolute 3. relative Should it fail before or after processing the 3rd relative submodule? I was thinking it would fail while trying to sync/update the 2nd one (which is absolute) and stop before processing the 3rd. 3. Running `git submodule update --init --with-remote name` shall fail the command ONLY if a submodule is being processed that is NOT also being initialized. No since the --init flag just tells update to initialize submodules on-demand. It should just go ahead the same way as without --with-remote. But doesn't the on-demand initialization need to evaluate relative URLs and convert them to absolute based on the .gitmodules configuration? I thought the idea was to make `--with-remote` invalid for initialization/sync of absolute URLs. In other words if I did: git submodule init --with-remote fork my-submodule-dir and if my-submodule-dir was not relative in .gitmodules, then the `--with-remote` flag becomes useless. We could fail silently but for educational purposes to the user I thought we were failing in these scenarios. Maybe I misunderstood your original intent with the failures? Is init not doing the relative to absolute evaluation like I'm thinking? Please correct me where I'm wrong. 4. The behavior of git submodule's `update` or `sync` commands combined with `--with-remote` will REPLACE or CREATE the 'origin' remote in each submodule it is run in. We will not allow the user to configure what the submodule remote name will end up being (I think this is current behavior and forces good practice; I consider `origin` an adopted standard for git, and actually wish it was more enforced for super projects as well!) No please carefully read my email again. I specifically was describing the opposite. --with-remote=name creates/replaces the remote name in the submodule. I do not see a benefit in restricting the user from creating different remote names in the submodule. I think it would be more confusing if the remote 'origin' in the superproject does not point to the same location as 'origin' in the submodule. Well the reason why I said it would be 'origin' is so that the submodule knows which remote to use internally during an update. I'm assuming 'update' uses 'origin' internally in the submodule to know which remote to pull from. My understanding of how `git submodule update` knows which URL to pull from is probably incorrect. I'm not familiar on the internal mechanics of how this works. Perhaps you could explain or send me to some reading material on it? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Re: Re: Re: Relative submodule URLs
On Tue, Aug 26, 2014 at 10:18:48AM -0500, Robert Dailey wrote: On Tue, Aug 26, 2014 at 1:28 AM, Heiko Voigt hvo...@hvoigt.net wrote: My last email response was in violation of your request to keep the two topics separate, sorry about that. I started typing it this weekend and completed the draft this morning, without having read this response from you first. Thats fine, no problem. Here is what I think would make the feature most usable. I think you went over some of these ideas but I just want to clarify, to make sure we're on the same page. Please correct me as needed. 1. Running `git submodule update --with-remote name` shall fail the command unconditionally. I am not sure but I think you mean git submodule update --with-remote=name With the equals sign, without it you would name the submodule paths to update. No I think that should just add the remote name to all submodules that would be updated and do the normal update operation on them (with the new remote of course). I'm not sure about Linux but at least with msysgit on Windows, typing a two-dash option (such as --with-remote) forces command line evaluation to use the next placement parameter as the parameter for it. I've seen this work the same way with argparse in python too. In my experience, command line has worked that way, I'm not sure if that is by design or not though. I never use equal signs with git commands, never had a problem for some reason. For example: git rebase --onto release/1.0 head~3 head The `--onto` option knows to use `release/1.0` as its parameter. If you are on Window or Linux does not make a difference here. I just realized we are quite inconsistent: $ git grep -E -e --\w+=\w+ -- Documentation/ | wc -l 226 $ git grep -E -e --\w+ \w+ -- Documentation/ | wc -l 75 I would prefer the first though since that one is used more often. But we can leave that for later, once we have some code to talk about. 2. Using the `--with-remote` option on submodule `update` or `sync` will fail if it detects absolute submodule URLs in .gitmodule Yes, almost. Since you can have a mixture I suggest to only fail if the submodules that would be processed have an absolute url in them. If processed submodules are all relative it can go ahead. For example if it processes 3 submodules in the following order: 1. relative 2. absolute 3. relative Should it fail before or after processing the 3rd relative submodule? I was thinking it would fail while trying to sync/update the 2nd one (which is absolute) and stop before processing the 3rd. For consistency I would prefer if it fails right from the beginning in this situation since the command can not be completed. 3. Running `git submodule update --init --with-remote name` shall fail the command ONLY if a submodule is being processed that is NOT also being initialized. No since the --init flag just tells update to initialize submodules on-demand. It should just go ahead the same way as without --with-remote. But doesn't the on-demand initialization need to evaluate relative URLs and convert them to absolute based on the .gitmodules configuration? I thought the idea was to make `--with-remote` invalid for initialization/sync of absolute URLs. In other words if I did: git submodule init --with-remote fork my-submodule-dir and if my-submodule-dir was not relative in .gitmodules, then the `--with-remote` flag becomes useless. We could fail silently but for educational purposes to the user I thought we were failing in these scenarios. Maybe I misunderstood your original intent with the failures? Is init not doing the relative to absolute evaluation like I'm thinking? Please correct me where I'm wrong. Yes it does the relative to absolute evaluation. But that is a different topic. For absolute urls in .gitmodules it should fail, but you were talking about --init in general and in general that should not fail IMO. So e.g. git submodule update --init --with-remote=name when all submodule urls are relative in .gitmodules and some submodules have already been initialized should succeed. 4. The behavior of git submodule's `update` or `sync` commands combined with `--with-remote` will REPLACE or CREATE the 'origin' remote in each submodule it is run in. We will not allow the user to configure what the submodule remote name will end up being (I think this is current behavior and forces good practice; I consider `origin` an adopted standard for git, and actually wish it was more enforced for super projects as well!) No please carefully read my email again. I specifically was describing the opposite. --with-remote=name creates/replaces the remote name in the submodule. I do not see a benefit in restricting the user from creating different remote names in the submodule. I think it would be more confusing if the remote 'origin' in the
Re: Re: Re: Relative submodule URLs
On Wed, Aug 20, 2014 at 08:18:12AM -0500, Robert Dailey wrote: On Tue, Aug 19, 2014 at 3:57 PM, Heiko Voigt hvo...@hvoigt.net wrote: I would actually error out when specified in already cloned state. Because otherwise the user might expect the remote to be updated. Since we are currently busy implementing recursive fetch and checkout I have added that to our ideas list[1] so we do not forget about it. In the meantime you can either use the branch.name.remote configuration to define a remote to use or just use 'origin'. Cheers Heiko [1] https://github.com/jlehmann/git-submod-enhancements/wiki#add-with-remote--switch-to-submodule-update Thanks Heiko. I would offer to help implement this for you, if you find it to be a good idea, but I've never done git development before and based on what I've seen it seems like you need to know at least 2-3 languages to contribute: bash, perl, C++. I know C++ Python but I don't know perl or bash scripting language. What would it take to help you guys out? It's easy to complain file bugs but as a developer I feel like I should offer more, if it suits you. For this particular case shell scripting should be sufficient. And it should not take too much time. Have a look at the git-submodule.sh script in the repository. That is the one implementing the git submodule command. Additionally you need to extend the documentation and write a test or two. Writing a test is also done in shell script. The documentation[1] is in asciidoc which is pretty self explanatory. The test should probably go into t/t7406-submodule-update.sh and, as Phil pointed out, in t7403-submodule-sync.sh). Also make sure to read the shell scripting part in Documentation/CodingGuidelines and as a general rule: Keep close to the style you find in the file. And when you are ready to send a patch: Documentation/SubmittingPatches. If you are happy but unsure about anything just send a patch with your implementation (CC me and everyone involved) and we will discuss it here on the list. Cheers Heiko [1] Documentation/git-submodule.txt -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Re: Re: Relative submodule URLs
On Tue, Aug 19, 2014 at 3:57 PM, Heiko Voigt hvo...@hvoigt.net wrote: I would actually error out when specified in already cloned state. Because otherwise the user might expect the remote to be updated. Since we are currently busy implementing recursive fetch and checkout I have added that to our ideas list[1] so we do not forget about it. In the meantime you can either use the branch.name.remote configuration to define a remote to use or just use 'origin'. Cheers Heiko [1] https://github.com/jlehmann/git-submod-enhancements/wiki#add-with-remote--switch-to-submodule-update Thanks Heiko. I would offer to help implement this for you, if you find it to be a good idea, but I've never done git development before and based on what I've seen it seems like you need to know at least 2-3 languages to contribute: bash, perl, C++. I know C++ Python but I don't know perl or bash scripting language. What would it take to help you guys out? It's easy to complain file bugs but as a developer I feel like I should offer more, if it suits you. Let me know I'm happy to help with anything. Thanks again!! -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Re: Re: Relative submodule URLs
On Tue, Aug 19, 2014 at 03:23:36PM -0500, Robert Dailey wrote: On Tue, Aug 19, 2014 at 2:30 PM, Heiko Voigt hvo...@hvoigt.net wrote: Well the remote for the submodule is currently only calculated once, when you do the initial git submodule update --init that clones the submodule. Afterwards the fixed url is configured under the name 'origin' in the submodule like in a normal git repository that you have freshly cloned. Which remote is used for cloning depends on the configured remote for the current branch or 'origin'. When you do a fetch or push with --recurse-submodules it only executes a 'git fetch' or 'git push' without any specific remote. For fetch the same commandline options (but only the options) are passed on. Here it might make sense to guess the remote in the submodule somehow and not do what fetch without remotes would do. For the triangular workflow not much work has been done in regards to submodule support. But since a submodule behaves like a normal git repository maybe there is not much work needed and we can just point to the workflow without submodules most times. We still have to figure that out properly. Maybe then the only thing we need is a --with-remote option for git submodule? :: git submodule update --init --with-remote myremote The --with-remote option would be a NOOP if it's already initialized, as you say. But I could create an alias for this as needed to make sure it is always specified. I would actually error out when specified in already cloned state. Because otherwise the user might expect the remote to be updated. Since we are currently busy implementing recursive fetch and checkout I have added that to our ideas list[1] so we do not forget about it. In the meantime you can either use the branch.name.remote configuration to define a remote to use or just use 'origin'. Cheers Heiko [1] https://github.com/jlehmann/git-submod-enhancements/wiki#add-with-remote--switch-to-submodule-update -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
