Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)
Junio C Hamano gits...@pobox.com writes: You raised a good point on the issue of external dependencies may impact Git as a whole, even for those who are not interested in the particular remote helpers at all. I'll have to think about it. (As I'm sure you know, but starting from the beginning.) There are basically two ways that a program can be built. One is by a user on a particular system. There, checking to see if various libraries are installed and if so enabling some additional feature (that isn't that hard/time-consuming to build) is entirely reasonable. In a packaging system, dependencies are much more troublesome. Dependencies have to be declared, and the build limited to use only those declared dependencies, in order to get repeatable builds and binary packages that can be used on other systems. Dependencies that really are required are fine. But optional dependencies are a problem, because e.g. one doesn't want to require the presence of qt to build something (that isn't already enormous). So if git needs mercurial and subversion installed, plus perhaps 5 other things for less popular remote helpers, that starts to be a real burden. (I realize some packaging systems have a style where the union of the possible dependencies must be present to build, and then the resulting binaries are allocated to split packages. But that's not universal, and it still requires large amounts of unnecessary dependencies to build a package from source.) Ideally, the core of git would have a small set of dependencies, and optional language bindings or remote helpers could be built independently (by running a different build, after git proper was built and installed). It seems more likely that the property of independent builds of optional components will be preserved if the various git-foo pieces are in seaprate trees. But if they are in subdirs of the main git tree, and build by ./configure make make install in the subdir, that's arguably equivalent. pgppg0HzmvEjx.pgp Description: PGP signature
Re: What's cooking in git.git (Apr 2014, #09; Tue, 29)
Felipe Contreras felipe.contre...@gmail.com writes: Greg Troxel wrote: In a packaging system, dependencies are much more troublesome. Dependencies have to be declared, and the build limited to use only those declared dependencies, in order to get repeatable builds and binary packages that can be used on other systems. Dependencies that really are required are fine. But optional dependencies are a problem, because e.g. one doesn't want to require the presence of qt to build something (that isn't already enormous). So if git needs mercurial and subversion installed, plus perhaps 5 other things for less popular remote helpers, that starts to be a real burden. It doesn't *need* them to build. The Mercurial/Bazaar dependencies are optional, both at build-time and at run-time. Most distributions would want to test the functionality they are distributing, and for testing they do need these dependencies. My point is that a packaging of git needs to either decide to forego these optional parts, or to include them, in the default case. One choice means that anyone who builds the package from source has to have the dependencies, and the other means that users of the built package(s) can't use the features. I realize that in Linux it's perhaps typical to not worry about burdening builders because actually building is very rare, but that's only reasonable because of having only one OS and perhaps three CPUs; with dozens each, building from source becomes more frequent. So I'm merely trying to suggest that it's better to have a core package with a restrained set of dependencies, and then a way to build the other things independently (perhaps assuming the core is built/installed), each with their own dependencies. It turns out in pkgsrc that git-svn is a meta-package (with no files) that depends on git-base (no man pages, no gitk) and p5-subversion. hg-git appears to be a separate source distribution, depending on a python implementation of the git formats. So perhaps the situation is currently ok. I was just trying to point out the issue to avoid regressions in the packaging situation. pgptdIoL4IFDe.pgp Description: PGP signature
Re: [PATCH 0/3] Reject non-ff pulls by default
Junio C Hamano gits...@pobox.com writes: Peff already covered (1)---it is highly doubtful that a merge is almost always wrong. In fact, if that _were_ the case, we should simply be defaulting to rebase, not failing the command and asking between merge and rebase like jc/pull-training-wheel topic did. We simply do not know what the user wants, as it heavily depends on the project, so we ask the user to choose one (and stick to it). From my experience leading the first large project using git at BBN, evolving a workflow (most work on topic branches, which are rebased, banning 'git pull'-created merge commits, and explicit merge commits to preserve --first-parent, basically), and seeing many people struggle to learn all this, my take is that a user who does not understand non-ff merge vs ff-merge vs rebase will end up doing the wrong thing. So two thoughts: In the glorious future, perhaps git could have a way to express a machine-parseable representation of the workflow and rules for a repo, so that these choices could be made accordingly. In the current world, I think it makes sense to error out when there are multiple reasonable choices depending on workflow. One of my team members, Richard Hansen, has argued to us that 'git pull' is harmful, essentially because it creates non-ff merges sometimes, while our rules say those should be rebased out. So we use [alias] up = !git remote update -p git merge --ff-only @{u} which acts like pull if ff merge works, and otherwise errors out. I think the key question is: can a user who doesn't really understand the implications of ff vs non-ff merges and the local workflow rules actually function ok, or do they need to stop and go back and understand. I'm in the you just have to take the time to understand camp, which led to us having a semi-custom syllabus from github training covering rebase, explicit vs ff merges and the consequences for first-parent history, etc. Therefore, I think git pull should do the update (perhaps of just the remote corresponding to @{u}, perhaps without -p) and a --ff-only merge, absent a configuration asking for non-ff merge or rebase. (Arguably, an ff merge is a degenerate case of rebase and also of the ff/non-ff merge, so it's safe with either policy.) Greg pgpT4Be5FrhuZ.pgp Description: PGP signature
Re: git should not use a default user.email config value
Jeff King p...@peff.net writes: On Sat, Aug 10, 2013 at 11:59:21AM +0200, Michael Haggerty wrote: I intentionally don't set user.email in my ~/.gitconfig because I use different identities (on the same machine) depending on what project I am committing to (open-source vs. work). After I clone a repo, I *rely* on Git reminding me to set user.email on my first commit, because I invariably forget to set it myself. And for me, *any* universal, heuristically-determined email address would be wrong for me for at least some repos. So if I understand your use case, then you would be even happier if rather than giving a warning, git simply barfed and said please set your identity before committing? I also think it's a bug that git will create commits without an explicitly-set author. I've seen multiple cases of the author being something unreasonable in a shared/official repository because of this. One was a person's personal email address on a work-repo commit, apparently because on Mac there was some magic extraction of primary email address from Mail.app (but I'm not 100% clear on what happened). If name/mail are not explicitly set, failing and making the user set them seems like the right thing. I find all the discussion of /etc/mailname to be a bit perplexing. The notion that the externally-visible email of a person making a commit should be the same as if they sent mail from that machine seems to be a bit of a stretch. And their username might be different. I don't think it's possible to reliably figure out what ought to be in the git author field. Another reason to fail rather than use a possibly-wrong default is that it's very difficult (if not impossible, depending on local CM policy about forced updates in shared repos) to recover from pushing a commit with a bad email address. (And the people that don't set their email right are the same people that won't run git log -p @{u}.. before pushing.) But failing and having to set it manually is easy (people who are already competent will be slowed down a minute or two, and the others need to learn anyway), results in something that should have been done anyway, and has no long-term negative consequences. pgpGtvyDdMd2C.pgp Description: PGP signature
Re: Feature Request: Support logic or shell execution to control values in .gitconfig
Matthieu Moy matthieu@grenoble-inp.fr writes: What I suggest instead is to edit/track/share template configuration files like ~/.gitconfig.in email = me@HOSTNAME@ and then script something like sed -e s/@HOSTNAME@/$(hostname)/ ~/.gitconfig.in ~/.gitconfig. You may also use the include.path functionality to share most of your configuration, and include a small file which is different on each host. For what it's worth, I keep dotfiles checked in as m4 sources, which is more or less equivalent to Matthieu's suggestion of sed don't try to keep .gitconfig under source control, but instead have a git-config alias that executes a lot of git config --global commands. The downsides to the git-config shell function approach are: unwanted configurations are not removed one has to source (e.g.) .bash_aliases and then rerun git-config after updating the dotfile repo, I suspect what's really needed is some sort of two-way macro preprocessor that can take changes in an output file an back them into the source. pgpHDjNi8b3hD.pgp Description: PGP signature
repo consistency under crashes and power failures?
Clearly there is the possibility of creating a corrupt repository when receiving objects and updating refs, if a crash or power failure causes data not to get written to disk but that data is pointed to. Journaling mitigates this, but I'd argue that programs should function safely with only the guarantees from POSIX. I am curious if anyone has actual experiences to share, either a report of corruption after a crash (where corruption means that either 1) git fsck reports worse than dangling objects or 2) some ref did not either point to the old place or the new place) experiments intended to provoke corruption, like dropping power during pushes, or forced panics in the kernel due to timers, etc. Alternatively, is there somewhere a first-principles analysis vs POSIX specs (such as fsyncing object files before updating refs to point to them, which I realize has performance negatives)? (I have not done experiments, but have observed no corruption.) Thanks, Greg pgpgbG9bqc3bd.pgp Description: PGP signature
Re: Tracking vendor release with Git
I'm trying to setup a workflow to track vendor releases (upstream). Each new release are provided as an archive of source code, data, documentation, etc. I've been doing more or less this. A few comments: I suggest that you not view CRLF-LF as a patch. I would do EOL hygiene as a preprocessing script, with a checked-in script, after unpacking the tarball or whatever, and before 'import'. Otherwise it's just going to be too messy. I use vendor.foo as the branch name. If your repo is only for this program, you can ignore this, but otherwise you way want to use subtree merge so that vendor.foo: maps to master:foo (putting foo in a subtree in master). This lets you have multiple upstreams in one repo, which is useful for system building more than maintaining. I would avoid rebase. You are essentially merging someone else's branch (that they aren't putting in git, but you are with the vendor.foo) into your master. With regular merge, you can still diff, but the natural history will be right. With rebase each local version as you call it will have different commits that will not have clear ancestry. pgpKcjZY3cZsB.pgp Description: PGP signature
Re: [Administrivia] On ruby and contrib/
Felipe Contreras felipe.contre...@gmail.com writes: On Thu, Jun 6, 2013 at 9:54 AM, Greg Troxel g...@ir.bbn.com wrote: git is a core tool that people use on almost the smallest of boxes, perhaps even replacing rcs for managing local config files. On such machines, even perl may be large, but a second scripting language seems excessive. You can compile Git without any of them. That ignores the 99% of people who use packaged versions. The question is really Should the standared approach for building be to use or not use dependency X?. Really this should be expressed in the README, and it creates expectations for someone who just installs the git package in terms of whether pieces of functionality are there. Packagers generally should be reading the README and including required/recommended dependencies and not including optional dependencies (in the main package). The information in INSTALL is pretty reasonable, but it doesn't really clearly say if you hand someone git built without perl, it is { perfectly ok but missing a fringe optional feature | deficient because git add -p won't work }. I'm leaning towards the deficient camp. So you can compile git without X should really translate into when one runs the default build following the instructions, and does not take affirmative steps to use X, X should not be used or depended on. If it doesn't mean that, it doesn't help the packaging/expectations discussion. It's of course fine that one can hand-compile a smaller than standard but still useful subset. But that's entirely different from the definition of normal. On a NetBSD 6 i386 system, the size of the ruby193-base binary package (as installed) is 25 MB (vs 15 MB for the git base package, which lacks gitk and docs). (Presently, the git base package defaults to requiring python and installing the git_remote_helpers, but I think that's a bug.) perl is 54 MB. That's only the default, if the default doesn't suit you, don't use the default. It's not about what I want. It's about making choices that affect other people, and trying to find a plan that will be overall reasonable; that's the essence of stewardship in packaging. Compiling for just myself is far easier. pgpEWOFl1E9Ut.pgp Description: PGP signature
Re: [RFH] NetBSD 6?
Junio C Hamano gits...@pobox.com writes: [OLD_ICONV] It refers to the type of the second parameter to iconv(); OLD_ICONV makes it take const char *, as opposed to char *, the latter of which matches http://pubs.opengroup.org/onlinepubs/9699919799/functions/iconv.html I just wanted to follow up on this. It turns out that the old POSIX standard was buggy (header file and function spec were different), and they resolved it in favor of non-const. NetBSD followed the const way, and just now documented that with links to the standards email archives. Interestingly, GNU iconv 1.14 seems to define it as const also: https://www.gnu.org/savannah-checkouts/gnu/libiconv/documentation/libiconv-1.14/iconv.3.html (which matches man/iconv.3 in the tarball). When I build libiconv-1.14, it produces a .h with const. But it has a configure test to check if there is a host include file with const, and puts the const in the built header file or not to match! In include/iconv.h.in, there is: extern size_t iconv (iconv_t cd, @ICONV_CONST@ char* * inbuf, size_t *inbytesleft, char* * outbuf, size_t *outbytesleft); Someday, it would be nice to have the configure test not fail an iconv implementation just because of the const, unless the presence of const is causing a real problem. But I can understand that no one thinks that's important enough to get around to. pgpP2BDPxvhEM.pgp Description: PGP signature
Re: [RFH] NetBSD 6?
Junio C Hamano gits...@pobox.com writes: Don't get too offended by the OLD_ prefix to that symbol, by the way. I do not think old means old and broken hence fixed in newer version and you are low life if you live on a platform that has to define it ;-). Thanks - it did throw me at the beginning, because I expected that it lead to using a copy of GNU iconv and not using the native one. But it will probably confuse few enough people that changing to CONST_ICONV is not warranted... We just needed to have a boolean to tell which variant it is to let the compiler build objects without complaining, and we named that switch as OLD_ICONV. I get that, now that I read utf8.c. It's amusing that git's own function is const, and on non-OLD_ICONV platforms has to cast away the const for standards-compliant iconv. pgpMmmRFXFcvN.pgp Description: PGP signature
Re: [PATCH] t1402: work around shell quoting issue on NetBSD
René Scharfe rene.scha...@lsrfire.ath.cx writes: invalid_ref() constructs a test case description using its last argument, but the shell seems to split it up into two pieces if it contains a space. Minimal test case: This is indeed a bug in NetBSD's shell, which I reported after finding this test case problem, and I think someone is working on a fix. But because git does not intend to be a shell torture test, if it's possible to avoid bugs in a reasonable way, I think it's nice to do so. pgpeZdzuWnVQF.pgp Description: PGP signature
Re: [PATCH] t1402: work around shell quoting issue on NetBSD
René Scharfe rene.scha...@lsrfire.ath.cx writes: The test fails for me on NetBSD 6.0.1 and reports: ok 1 - ref name '' is invalid ok 2 - ref name '/' is invalid ok 3 - ref name '/' is invalid with options --allow-onelevel ok 4 - ref name '/' is invalid with options --normalize error: bug in the test script: not 2 or 3 parameters to test-expect-success The alleged bug is in this line: invalid_ref NOT_MINGW '/' '--allow-onelevel --normalize' The bug in NetBSD's sh has been fixed in -current: http://gnats.netbsd.org/47361 and the change will almost certainly make it to the -6 and -5 release branches. With the fixed sh: 414c78c (with the workaround): t1402 passes 69637e5 (without the workaround): t1402 passes With the buggy sh, 414c78c (with the workaround): t1402 passes 69637e5 (without workaround): t1402 fails so I can confirm that the workaround is successful on NetBSD 5. Thanks for addressing this, and sorry I didn't mention it on this list. Greg pgpTeLAlCVU0y.pgp Description: PGP signature
Re: [RFH] NetBSD 6?
Junio C Hamano gits...@pobox.com writes: I forgot to mention that we also ship configure (and keep track of configure.ac) so that optionally people can let autoconf machinery to create config.mak.autogen to be included at the same place as handcrafted config.mak in their build process. I do not offhand know if we do for p in python python2.6 python2.7; do ... kind of thing, though. pkgsrc uses the configure method, but it seems not to output a PYTHON_PATH. It looks like automake's python.m4 is not used by git's configure.ac. But pkgsrc passes PYTHON_PATH in the environment to make, so it works out currently. It refers to the type of the second parameter to iconv(); OLD_ICONV makes it take const char *, as opposed to char *, the latter of which matches http://pubs.opengroup.org/onlinepubs/9699919799/functions/iconv.html Thanks - I now see our extra const and am looking into it. pgpgVxwiC7ZH_.pgp Description: PGP signature
Re: [RFH] NetBSD 6?
Junio C Hamano gits...@pobox.com writes: [query about NetBSD-6] The 2.7 bit certainly looks fishy, as users should be able to choose between 2.6 and 2.7 (and possibly 3.0), IIUC. + PYTHON_PATH = /usr/pkg/bin/python2.7 + PERL_PATH = /usr/pkg/bin/perl (I am one of the people who maintain the git package in pkgsrc.) (Strictly, this is not really about NetBSD, but about all systems where the standard approach to get python is via pkgsrc. So that include DragonflyBSD as well. (pkgsrc runs on many other systems, but it isn't the standard approach, so from the git viewpoint that's irrelevant.)) You are entirely right that on e.g. NetBSD 6 the view is that users should be able to choose the python version. pkgsrc can install multiple versions of python at the same time, to cope with python-using packages that need different versions. pkgsrc chooses not to have a 'python' program, because that would result in installed packages changing their binding of which python version to use when the default was changed. The default python version is currently 2.7, so /usr/pkg/bin/python2.7 is the best guess for finding python on a NetBSD system, if you're only allowed one guess. A user can set a PYTHON_VERSION_DEFAULT variable to choose the version they want; each package expresses which versions will work. This isn't relevant for git, not being a pure python library, but pkgsrc supports installing multiple versions of some packages, so one can have two versions installed at once: py27-expat-0nb6 Python interface to expat py26-expat-0nb6 Python interface to expat The git package just depends on one version; by default the git package depends on python (but one can tell it not to). The python.m4 macro that comes with automake seems to find one of the various pythonX.Y binaries in $PATH just fine. pkgsrc has an entry for git (at 1.8.0.1). The key line for handling python is: MAKE_FLAGS+= PYTHON_PATH=${PYTHONBIN} and there PYTHONBIN is set up by pkgsrc infrastructure for the right prefix (99.9% but not always /usr/pkg) and version. After this, everything seems to come out right: head -1 /usr/pkg/libexec/git-core/git-p4 #!/usr/pkg/bin/python2.7 So I'd say that if PYTHON_PATH is set in the environment to configure, it should behave as it does now. And if not, it would be nice if the highest pythonX.Y found (that is known to work with git) is used. + PYTHON_PATH = /usr/pkg/bin/python2.7 + PERL_PATH = /usr/pkg/bin/perl So it would be nice to make these work as ?=, letting an environment variable win if set. pgpTtVz1Ch_vc.pgp Description: PGP signature
Re: [RFH] NetBSD 6?
Junio C Hamano gits...@pobox.com writes: I would appreciate if somebody with more familiarlity with the platform can suggest a better alternative than applying the following patch to our Makefile. Right now I have an equivalent of this change in config.mak locally when building on the said platform. I realized after sending my previous reply that you are probably trying to have a way to build and run tests on NetBSD-6 from a git checkout as part of development testing. One approach I've taken with other programs is to have a README.NetBSD file which is actually an executable /bin/sh script with comments, explaining the prereqs in terms of pkgsrc and invoking configure to get dependencies from pkgsrc (-I/usr/pkg/include plus -L/-R). So in the git case, this could set PYTHON_PATH in the environment. I realize a README.foo file for N different systems could be clutter, but having these checked in would provide the concise help that people on any of those platforms need. pgpjKLtsOgSYi.pgp Description: PGP signature
Re: GIT get corrupted on lustre
we are using git since may and all is working fine for all of us (almost 20 people) on our workstations. However, when we clone our repositories to the cluster, only and only there we are having many problems similiar to this post: What filesystem tests have you run on lustre? I would run every test you can find, and lustre should have a robust test suite. It's really hard to be certain, but given how many filesystems git is used with, your experience points to a lustre bug. I would also suggest using ktrace/ktruss/strace and perhaps poring over the logs to see if you can spot any bad behavior. pgp6qmR6soDBX.pgp Description: PGP signature
Re: Q: do people compile with NO_FNMATCH on OpenBSD 5.2?
[mkstemp truncating output on error] diff --git c/wrapper.c w/wrapper.c index 68739aa..a066e2e 100644 --- c/wrapper.c +++ w/wrapper.c @@ -229,7 +229,7 @@ int xmkstemp(char *template) int saved_errno = errno; const char *nonrelative_template; - if (!template[0]) + if (strlen(template) != strlen(origtemplate)) template = origtemplate; nonrelative_template = absolute_path(template); Thanks for the quick fix. I applied this patch to 1.8.0.1, and then the tests get all the way to t1402 (separate msg when I figure out why). pgpaC2CMgoYXA.pgp Description: PGP signature
[PATCH] Documentation/git-update-index: caution about tree objects
While one can add tree objects to the index, this is not currently useful. Therefore, use git ls-tree -r as the example to be fed to --index-info. Add a section explaining about expected index contents. (Thanks to Junio for explaining this to me in August of 2011.) Signed-off-by: Greg Troxel g...@ir.bbn.com --- Documentation/git-update-index.txt | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt index 9d0b151..6ce65fa 100644 --- a/Documentation/git-update-index.txt +++ b/Documentation/git-update-index.txt @@ -29,6 +29,11 @@ Modifies the index or directory cache. Each file mentioned is updated into the index and any 'unmerged' or 'needs updating' state is cleared. +Note that update-index does not check that the modifications preserve +the expected invariants. In particular, an index normally holds +regular blobs, executable blobs, symlink blobs, and gitlinks. +Therefore, adding a tree object is not likely useful. + See also linkgit:git-add[1] for a more user-friendly way to do some of the most common operations on the index. @@ -210,7 +215,7 @@ back on 3-way merge. . mode SP type SP sha1 TAB path + -The second format is to stuff 'git ls-tree' output +The second format is to stuff 'git ls-tree -r' output into the index file. . mode SP sha1 SP stage TAB path -- 1.8.0.1 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Q: do people compile with NO_FNMATCH on OpenBSD 5.2?
Junio C Hamano gits...@pobox.com writes: I seem to get a failure from git ls-files a* in t/t-basic.sh if I link with platform's fnmatch(). Not what you asked, but on NetBSD 5.1, libc fnmatch is used, and with git 1.8.0.1 that test passes. This prompted me to look at the rest of the tests. All tests pass (except for expected failures) until: *** t0070-fundamental.sh *** ok 1 - character classes (isspace, isalpha etc.) not ok - 2 mktemp to nonexistent directory prints filename # # test_must_fail test-mktemp doesnotexist/testXX 2err # grep doesnotexist/test err # ok 3 - mktemp to unwritable directory prints filename ok 4 - check for a bug in the regex routines # failed 1 among 4 test(s) 1..4 Running this by hand, I get: gdt 51 /usr/pkgsrc/devel/scmgit-base/work/git-1.8.0.1/t ../test-mktemp foo/barXX MKTEMP.stdout 2 MKTEMP.stderr; ls -l MKTEMP* -rw-r--r-- 1 gdt wheel 121 Dec 18 15:14 MKTEMP.stderr -rw-r--r-- 1 gdt wheel0 Dec 18 15:14 MKTEMP.stdout gdt 52 /usr/pkgsrc/devel/scmgit-base/work/git-1.8.0.1/t cat MKTEMP.stderr fatal: Unable to create temporary file '/usr/pkgsrc/devel/scmgit-base/work/git-1.8.0.1/t/foo': No such file or directory It seems ENOENT is correct for the directory not existing. I think the test is complaining that the failed call to mkstemp modified the argument. Looking at: http://pubs.opengroup.org/onlinepubs/9699919799/functions/mkstemp.html I can't see that it requires anything in particular for the in/out paramater when there is an error. pgpnwLq2iMf0J.pgp Description: PGP signature
Re: Message from 'git-rebase'; German translation
Ralf Thielow ralf.thie...@gmail.com writes: On Sat, Sep 22, 2012 at 11:57 PM, Sascha Cunz sascha...@babbelbox.org wrote: Original: ... and I wonder if you are in the middle of another rebase. German git translation: ... und es wäre verwunderlich, wenn ein Neuaufbau bereits im Gange ist. And a re-translation back to English from my understanding as native German speaker: ... and it would be astonishing (=i'd be surprised), if a rebase was already in progress. Perhaps the English wording should be made more clear; the fact that translation is difficult is a clue that it may be hard to understand, particularly for non-native-English-Speakers (albeit an ambiguous one; I realize translation is hard). We try to avoid translating terms like I wonder if.. and similar as ich wundere mich, because it's very unusual in German messages of computer programs. Translators should have the freedom to reword It's unusual in English too :-) pgpyhJYu3up07.pgp Description: PGP signature