`git commit -a` stages ignored submodules?
Hello everyone, There seems to be a difference how Git 2.15.0 handles submodules in comparison with 2.14.2. In Git 2.14.2 `git commit -a` will not stage submodules which has `ignore = all` set in their .gitmodule section. However, in Git 2.15.0 `git commit -a` will stage all submodules no matter what their "ignore" setting is set to, and also no matter if submodule path is in .gitignore. Didn't see anything explicit about this in release logs. Is it an expected new behavior?It is till possible to keep `git commit -a` to ignore submodules which are configured to be ignored? -- With best regards, Sergey Sharybin
Re: git:// protocol over SSL/TLS
Yeah, i understand this. We can not protect self from every single possible attack.. On Fri, Dec 27, 2013 at 10:26 PM, Bernhard R. Link brl+...@mail.brlink.eu wrote: * Sergey Sharybin sergey@gmail.com [131227 15:25]: Security in this case is about being sure everyone gets exactly the same repository as stored on the server, without any modifications to the sources cased by MITM. Note that ssl (and thus https) only helps here against a resource-less man-in-the-middle. Getting catch-all CA-signed certificates is said to no longer available to everyone as easily as it was some years ago, but unless you allow only one private CA (and even there clients often fail) you still should assume everyone resourceful enough to still be able to do MITM. Bernhard R. Link -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
git:// protocol over SSL/TLS
Hello everyone! Quick question is, is it possible to use git:// protocol over SSL/TLS/other secure transport? Or the recommended way to do secure anonymous checkout is to simply use https:// ? Thanks in advance! -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git:// protocol over SSL/TLS
Hi, On Fri, Dec 27, 2013 at 7:36 PM, Konstantin Khomoutov flatw...@users.sourceforge.net wrote: The Git protocol does not implement it itself but you can channel it over a TLS tunnel (via stunnel for instance). Unfortunately, this means a specialized software and setup on both ends so if the question was about a general client using stock Git then the answer is no, it's impossible. Ok, got it. Yes, but it will only be secure if you've managed to verify the server's certificate and do trust its issuer (or a CA higher up the cert's trust chain) -- people tend to confuse encrypted with secure which is not at all the same thing. We've got CA-signed certificate atm and it's about to be also EV-signed for our server (git.blender.org). So this is not gonna to be an issue. Cloning over https:// works fine, but we wanted to be sure all the bits are secure. So guess we just need to recommend using https:// protocol instead of git:// for our users? -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git:// protocol over SSL/TLS
Our sysadmns are mainly worried about possible MITM which might give users completely wrong repo. For sure users might simply compare hash of HEAD from https'ed site with repo browser with what they've got in the checkout. But that's an extra step which we'd like to avoid without security harm :) On Fri, Dec 27, 2013 at 8:12 PM, Andreas Schwab sch...@linux-m68k.org wrote: Sergey Sharybin sergey@gmail.com writes: So guess we just need to recommend using https:// protocol instead of git:// for our users? Given how easy it is to verify the integrity of a git repository out of band there isn't really much of added security by using TLS for transport. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 And now for something completely different. -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git:// protocol over SSL/TLS
Security in this case is about being sure everyone gets exactly the same repository as stored on the server, without any modifications to the sources cased by MITM. As for smart http, this seems pretty much cool.However, we're currently using lighthttpd, so it might be an issue. We'll check on whether smart http is used there, and if not guess it wouldn't be a big deal to switch to apache. On Fri, Dec 27, 2013 at 8:20 PM, Matthieu Moy matthieu@grenoble-inp.fr wrote: Andreas Schwab sch...@linux-m68k.org writes: Sergey Sharybin sergey@gmail.com writes: So guess we just need to recommend using https:// protocol instead of git:// for our users? Given how easy it is to verify the integrity of a git repository out of band there isn't really much of added security by using TLS for transport. You can verify integrity after the fact, but not guarantee confidentiality ... so it again depends on the definition of security. -- Matthieu Moy http://www-verimag.imag.fr/~moy/ -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: git:// protocol over SSL/TLS
As for smart http, this seems pretty much cool.However, we're currently using lighthttpd, so it might be an issue. We'll check on whether smart http is used there, and if not guess it wouldn't be a big deal to switch to apache. The web server software has nothing to do with HTTP[S] used by Git being smart, I think, it just has to be set up properly. Misunderstood git doc then which says it has to be Apache, currently - other CGI servers don't work, last I checked. As discussed in an earlier thread here, a good indication of the dumb version of the protocol being in use is no display of the fetching progress on the client while doing `git clone` because this information (like compressing objects ... etc) is sent by the server-side Git process which is only there if HTTP[S] was smart. Otherwise the client just GETs packs of objects, traverses them, GETs more and so on, so batches of HTTP GET requests correlating to clone sessions in the web server logs should also be indicative of the problem. Just to verify, if i see messages like Receiving objects: 1% (7289/705777), 1.72 MiB | 340.00 KiB/s it means server is smart ? -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git issues with submodules
Hey!
Sorry for the delayed reply.
Am i right the intention is to make it so `git add .` and `git commit
.` doesn't include changes to submodule hash unless -f argument is
provided?
On Sun, Nov 24, 2013 at 10:29 PM, Jens Lehmann jens.lehm...@web.de wrote:
Am 24.11.2013 01:52, schrieb Heiko Voigt:
Hi,
On Sat, Nov 23, 2013 at 09:10:44PM +0100, Jens Lehmann wrote:
Am 22.11.2013 23:09, schrieb Jonathan Nieder:
Heiko Voigt wrote:
After that we can discuss whether add should add submodules that are
tracked but not shown. How about commit -a ? Should it also ignore the
change? I am undecided here. There does not seem to be any good
decision. From the users point of view we should probably not add it
since its not visible in status. What do others think?
I agree --- it should not add.
I concur: adding a change that is hidden from the user during
the process is not a good idea.
Here is a patch achieving that. Still missing a test which I will add.
Looking good to me. Please add tests for diff.ignoreSubmodules
and submodule.name.ignore, the latter both in .gitmodules and
.git/config. While doing some testing for this thread I found an
inconsistency in git show which currently honors the submodule
specific option only from .git/config and ignores it in the
.gitmodules file (depending on the outcome of the discussion on
what '--ignore-submodules=all' should ignore we might have to fix
that one afterwards).
I'd suggest to also add the --ignore-submodules option in another
patch on top, because the user should be able to override the
configuration either way. And what about having the '-f' option
imply '--ignore-submodules=none'?
Cheers Heiko
---8
Subject: [PATCH] fix 'git add' to skip submodules configured as ignored
If submodules are configured as ignore=all they are not shown by status.
Lets also ignore them when adding files to the index. This avoids that
users accidentially add ignored submodules with: git add .
We achieve this by reading the submodule config and thus correctly
initializing the infrastructure to take the ignore decision.
Signed-off-by: Heiko Voigt hvo...@hvoigt.net
---
builtin/add.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/builtin/add.c b/builtin/add.c
index 226f758..2d0d2ef 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -15,6 +15,7 @@
#include diffcore.h
#include revision.h
#include bulk-checkin.h
+#include submodule.h
static const char * const builtin_add_usage[] = {
N_(git add [options] [--] pathspec...),
@@ -378,6 +379,10 @@ static int add_config(const char *var, const char
*value, void *cb)
ignore_add_errors = git_config_bool(var, value);
return 0;
}
+
+ if (!prefixcmp(var, submodule.))
+ return parse_submodule_config_option(var, value);
+
return git_default_config(var, value, cb);
}
@@ -415,6 +420,7 @@ int cmd_add(int argc, const char **argv, const char
*prefix)
int implicit_dot = 0;
struct update_callback_data update_data;
+ gitmodules_config();
git_config(add_config, NULL);
argc = parse_options(argc, argv, prefix, builtin_add_options,
--
With best regards, Sergey Sharybin
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH] disable complete ignorance of submodules for index - HEAD diff
; DIFF_OPT_SET(diffopt, IGNORE_SUBMODULES); - else if (!strcmp(arg, untracked)) + } else if (!strcmp(arg, untracked)) DIFF_OPT_SET(diffopt, IGNORE_UNTRACKED_IN_SUBMODULES); else if (!strcmp(arg, dirty)) DIFF_OPT_SET(diffopt, IGNORE_DIRTY_SUBMODULES); diff --git a/wt-status.c b/wt-status.c index b4e44ba..34be1cc 100644 --- a/wt-status.c +++ b/wt-status.c @@ -462,6 +462,9 @@ static void wt_status_collect_changes_index(struct wt_status *s) handle_ignore_submodules_arg(rev.diffopt, s-ignore_submodule_arg); } + /* for the index we need to disable complete ignorance of submodules */ + DIFF_OPT_SET(rev.diffopt, NO_IGNORE_SUBMODULE); + rev.diffopt.output_format |= DIFF_FORMAT_CALLBACK; rev.diffopt.format_callback = wt_status_collect_updated_cb; rev.diffopt.format_callback_data = s; -- 1.8.5.rc3.1.gcd6363f -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Re: Git issues with submodules
Heiko, yeah sure see what you mean. Changing existing behavior is pretty PITA. Just one more question for now, are you referencing to the patch [RFC PATCH] disable complete ignorance of submodules for index - HEAD diff? Coz i tested it and seems it doesn't change behavior of add/commit. Also, i'm around to test the all patches which are related on submodules :) On Mon, Nov 25, 2013 at 11:49 PM, Heiko Voigt hvo...@hvoigt.net wrote: On Mon, Nov 25, 2013 at 03:02:51PM +0600, Sergey Sharybin wrote: Am i right the intention is to make it so `git add .` and `git commit .` doesn't include changes to submodule hash unless -f argument is provided? Yes thats the goal. My patch currently only disables it when ignore is set to all. I will add another patch that implements the -f and --submodule-ignore option to both of them so the user has an easy way to bypass that. But having said that we changing existing behavior here so we have to investigate carefully whether we are not breaking peoples expectations (and script). That also applies to the other patch that enables showing them in diff and friends again. Cheers Heiko -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git issues with submodules
Hey, Answers are inlined. On Fri, Nov 22, 2013 at 5:16 PM, Ramkumar Ramachandra artag...@gmail.com wrote: [+CC: Jens, the goto-guy for submodules] Sergey Sharybin wrote: Namely, `git ls-files -m` will show addons as modified, regardless ignore=all configuration. In the same time `git diff-index --name-only HEAD --` will show no changes at all. This happens because diff-index handles submodules explicitly (see diff-lib.c), while ls-files doesn't (see builtin/ls-files.c). My opinion is that this is a bug, and git ls-files needs to be taught to handle submodules properly. Shall i fire report somewhere or it's being handled by the folks reading this ML? This leads to issues with Arcanist (which is a Phabricator's tool) who considers addons as uncommited changes and either complains on this or just adds this to commits. Does Arcanist use `git ls-files -m` to check? Yes, Arcanist uses `git ls-files -m` to check whether there're local modifications. We might also contact phab developers asking to change it to `git diff --name-only HEAD --`. Is there a preferable way to get list of modified files and are this command intended to output the same results? There're also some more issues which happens to our developers and which i can not quite reproduce. Do try to track down the other issues and let us know. I'm trying, but doesn't happen here on laptop yet. Will give it another try (do have some ideas). Also directed our developers here who experienced the issue and might give some details, Sometimes it happens so git checkout to another branch yields about uncommited changes to addons and doesn't checkout to another branch. I've seldom used submodules with branches, so I'll let others chime in. Ok, thanks anyway :) -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git issues with submodules
Ramkumar, not actually sure what you mean? For me `git diff --name-only HEAD --` ignores changes to submodules hash changes. Also apparently it became a known TODO for phabricator developers [1]. Jeff, kinda trying to match yes. Just don't want changes to submodules hash to be included. So, after all is it expected behavior of ls-files or not and if not shall i report it as a separate thread? :) [1] https://secure.phabricator.com/rARCe62b23e67deacc24469525cc5dea2b297a5073fb On Fri, Nov 22, 2013 at 9:11 PM, Jeff King p...@peff.net wrote: On Fri, Nov 22, 2013 at 06:38:47PM +0530, Ramkumar Ramachandra wrote: Does Arcanist use `git ls-files -m` to check? Yes, Arcanist uses `git ls-files -m` to check whether there're local modifications. We might also contact phab developers asking to change it to `git diff --name-only HEAD --`. Is there a preferable way to get list of modified files and are this command intended to output the same results? I just checked it out: it uses `git ls-files -m` to get the list of unstaged changes; `git diff --name-only HEAD --` will list staged changes as well. That diff command compares the working tree and HEAD; if you are trying to match `ls-files -m`, you probably wanted just `git diff --name-only` to compare the working tree and the index. Although in a script you'd probably want to use the plumbing `git diff-files` instead. -Peff -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git issues with submodules
Ah, didn't notice you're the author of that pull-request Ramkumar :) So guess issue with arc can be considered solved now. But i'm still collecting more details about how to manage to commit change addons hash without arc command even (it happens to Campbell Barton really often). Will report back when we'll know something. On Fri, Nov 22, 2013 at 10:35 PM, Ramkumar Ramachandra artag...@gmail.com wrote: Sergey Sharybin wrote: Ramkumar, not actually sure what you mean? For me `git diff --name-only HEAD --` ignores changes to submodules hash changes. `git diff --name-only HEAD --` compares the worktree to HEAD (listing both staged and unstaged changes); we want `git diff --name-only --` to compare the worktree to the index (listing only unstaged changes), as Peff notes. Also apparently it became a known TODO for phabricator developers [1]. That was me :) So, after all is it expected behavior of ls-files or not and if not shall i report it as a separate thread? :) Actually, I doubt it's worth fixing ls-files. Your problem should be fixed when this is merged (hopefully in a few hours): https://github.com/facebook/arcanist/pull/121 Cheers. -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git issues with submodules
Ok, got it now. To reproduce the issue: - Run git submodule update --recursive to make sure their SHA is changed. Then `git add /path/to/changed submodule` or just `git add .` - Modify any file from the parent repository - Neither of `git status`, `git diff` and `git diff-files --name-only` will show changes to a submodule, only changes to that file which was changed in parent repo. - Make a git commit. It will not list changes to submodule as wll. - `git show HEAD` will show changes to both file from and parent repository (which is expected) and will also show changes to the submodule hash (which is unexpected i'd say). On Fri, Nov 22, 2013 at 11:01 PM, Sergey Sharybin sergey@gmail.com wrote: Ah, didn't notice you're the author of that pull-request Ramkumar :) So guess issue with arc can be considered solved now. But i'm still collecting more details about how to manage to commit change addons hash without arc command even (it happens to Campbell Barton really often). Will report back when we'll know something. On Fri, Nov 22, 2013 at 10:35 PM, Ramkumar Ramachandra artag...@gmail.com wrote: Sergey Sharybin wrote: Ramkumar, not actually sure what you mean? For me `git diff --name-only HEAD --` ignores changes to submodules hash changes. `git diff --name-only HEAD --` compares the worktree to HEAD (listing both staged and unstaged changes); we want `git diff --name-only --` to compare the worktree to the index (listing only unstaged changes), as Peff notes. Also apparently it became a known TODO for phabricator developers [1]. That was me :) So, after all is it expected behavior of ls-files or not and if not shall i report it as a separate thread? :) Actually, I doubt it's worth fixing ls-files. Your problem should be fixed when this is merged (hopefully in a few hours): https://github.com/facebook/arcanist/pull/121 Cheers. -- With best regards, Sergey Sharybin -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Git issues with submodules
For some reason, the `git add .` is adding the ignored submodule to the index. The ignore setting is documented to only affect diff output (including what checkout, commit and status show as modified). While I agree that this behavior is confusing for Sergey and not optimal for the floating branch model he uses, git is currently doing exactly what it should. And for people using the ignore setting to not having to stat submodules with huge and/or many files that behavior is what they want: don't bother me with what changed, but commit what I did change on purpose. We may have to rethink what should happen for users of the floating branch model though. I totally see what's happening here and indeed current logic of `git add .` agree is correct from how it was designed to. I could also see why it might be useful to keep `git add .` and `git commit .` not to respect submodule ignore flag. The only confusing thing here is that if i stage changed submodule with this command i wouldn't see this submodule in changes to be committed wen doing a commit. So seems it's just matter of better communication of what's gonna to be committed in changes to be committed section? Or maybe even make it so `git status` will show staged changes from submdules hash regardless ignore flag? Just an ideas how to make communication what's going on a bit better :) And for sure don't think suppressing stuff from git show is a nice idea (if i understand your proposal f making submodule ignore option affect on other commands). -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Git issues with submodules
Hey everyone from Blender developers! As you might already know, we've recently switched from SVN to Git to host Blender sources. In general it works really awesome, but we've got some issues with submodules. in SVN we had separate repositories for addons and translations which were attached to main tree as svn:external. The reason for this was: 1. Separate commit access between core sources and addons so nobody accidentally breaks anything in the core. 2. Separate commit history to help tracking issues down. For the most developers and all artists (yes, we've got loads of artists who builds blender on their own) it makes sense to always checkout latest versions of addons and translations when updating working tree. We used Git submodules as a replacement for svn:external, with some tweaks and specific of update procedure. Namely, we always do `git submodule update --remote` to pull all the latest changes from submodules. This will mark checkout as modified because submodule hash changes. To avoid infinite commits of submodule hash we've added ignore=all to their configuration. In most cases it works fine, but there're some circumstances when it gives weirdo issues. Namely, `git ls-files -m` will show addons as modified, regardless ignore=all configuration. In the same time `git diff-index --name-only HEAD --` will show no changes at all. This leads to issues with Arcanist (which is a Phabricator's tool) who considers addons as uncommited changes and either complains on this or just adds this to commits. This issue i might easily reproduce on my laptop with latest Git 1.8.4.3. There're also some more issues which happens to our developers and which i can not quite reproduce. Sometimes it happens so git checkout to another branch yields about uncommited changes to addons and doesn't checkout to another branch. My guess here is that submodule hash in master and branch was different and having hash modified in master somehow prevented changes from another branch to be checked out. In this case question would be: what would be the proper way to checkout branches when having submodules configured this way? Second issue is that some developers still manages to commit changes to submodule hash, which i have totally no idea why Git allows to include such a changes. I could not do such a commits on purpose even. Here're some links to help understanding what's going on: - Blender repository browser: http://developer.blender.org/diffusion/B/ - Task in our tracker about issues we've got with Git: http://developer.blender.org/T37528 - History of changes to addons hash: http://developer.blender.org/diffusion/B/history/master/release/scripts/addons We're totally new to git submodules and clarification (and maybe even confirmed bug with ls-files -m :) would really be appreciated. We're also open for suggestions about re-configuring our submodules so they works in a way we'd expect this. Thanks in advance! -- With best regards, Sergey Sharybin -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
