Re: [PATCH] doc: Don't echo sed command for manpage-base-url.xsl

2018-08-28 Thread Eric Sunshine
On Tue, Aug 28, 2018 at 5:21 PM Tim Schumacher wrote: > Previously, the sed command for generating manpage-base-url.xsl > was printed to the console when being run. > > For the purpose of silencing it, define a $(QUIET) variable which > contains an '@' if verbose mode isn't enabled and which is

[PATCH] commit-reach: correct accidental #include of C file

2018-08-28 Thread Jonathan Nieder
Without this change, the build breaks with clang: libgit/ref-filter.pic.o: multiple definition of 'filter_refs' libgit/commit-reach.pic.o: previous definition here Signed-off-by: Jonathan Nieder --- Jonathan Nieder wrote: > Derrick Stolee wrote: >> --- a/commit-reach.c >> +++

Re: [PATCH v4 4/6] tests: use shorter here-docs in chainlint.sed for AIX sed

2018-08-28 Thread Ævar Arnfjörð Bjarmason
On Fri, Aug 24 2018, Eric Sunshine wrote: > On Fri, Aug 24, 2018 at 11:20 AM Ævar Arnfjörð Bjarmason > wrote: >> Improve the portability of chainlint by using shorter here-docs. On >> AIX sed will complain about: >> >> sed: 0602-417 The label :hereslurp is greater than eight >>

Re: [PATCH 0/1] Define GIT_TEST_COMMIT_GRAPH for commit-graph test coverage

2018-08-28 Thread Stefan Beller
On Tue, Aug 28, 2018 at 1:33 PM Derrick Stolee via GitGitGadget wrote: > > The commit-graph (and multi-pack-index) features are optional data > structures that can make Git operations faster. Since they are optional, we > do not enable them in most Git tests. The commit-graph is tested in >

[PATCH v2 2/9] introduce hasheq() and oideq()

2018-08-28 Thread Jeff King
The main comparison functions we provide for comparing object ids are hashcmp() and oidcmp(). These are more flexible than a strict equality check, since they also express ordering. That makes them useful for sorting and binary searching. However, it also makes them potentially slower than a

[PATCH v2 3/9] convert "oidcmp() == 0" to oideq()

2018-08-28 Thread Jeff King
Using the more restrictive oideq() should, in the long run, give the compiler more opportunities to optimize these callsites. For now, this conversion should be a complete noop with respect to the generated code. The result is also perhaps a little more readable, as it avoids the "zero is equal"

[PATCH v2 1/9] coccinelle: use <...> for function exclusion

2018-08-28 Thread Jeff King
Sometimes we want to suppress a coccinelle transformation inside a particular function. For example, in finding conversions of hashcmp() to oidcmp(), we should not convert the call in oidcmp() itself, since that would cause infinite recursion. We write that like this: @@ identifier f !=

[PATCH v2 4/9] convert "hashcmp() == 0" to hasheq()

2018-08-28 Thread Jeff King
This is the partner patch to the previous one, but covering the "hash" variants instead of "oid". Note that our coccinelle rule is slightly more complex to avoid triggering the call in hasheq(). I didn't bother to add a new rule to convert: - hasheq(E1->hash, E2->hash) + oideq(E1, E2)

[PATCH v2 6/9] convert "hashcmp() != 0" to "!hasheq()"

2018-08-28 Thread Jeff King
This rounds out the previous three patches, covering the inequality logic for the "hash" variant of the functions. As with the previous three, the accompanying code changes are the mechanical result of applying the coccinelle patch; see those patches for more discussion. Signed-off-by: Jeff King

[PATCH v2 7/9] convert hashmap comparison functions to oideq()

2018-08-28 Thread Jeff King
The comparison functions used for hashmaps don't care about strict ordering; they only want to compare entries for equality. Let's use the oideq() function instead, which can potentially be better optimized. Note that unlike the previous patches mass-converting calls like "!oidcmp()", this patch

[PATCH v2 8/9] read-cache: use oideq() in ce_compare functions

2018-08-28 Thread Jeff King
These functions return the full oidcmp() value, but the callers really only care whether it is non-zero. We can use the more strict !oideq(), which a compiler may be able to optimize further. This does change the meaning of the return value subtly, but it's unlikely that anybody would try to use

[PATCH v2 5/9] convert "oidcmp() != 0" to "!oideq()"

2018-08-28 Thread Jeff King
This is the flip side of the previous two patches: checking for a non-zero oidcmp() can be more strictly expressed as inequality. Like those patches, we write "!= 0" in the coccinelle transformation, which covers by isomorphism the more common: if (oidcmp(E1, E2)) As with the previous two

[PATCH v2 9/9] show_dirstat: simplify same-content check

2018-08-28 Thread Jeff King
We use two nested conditionals to store a content_changed variable, but only bother to look at the result once, directly after we set it. We can drop the variable entirely and just use a single "if". This needless complexity is the result of 2ff3a80334 (Teach --dirstat not to completely ignore

Re: [PATCH 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-08-28 Thread Derrick Stolee
On 8/28/2018 4:41 PM, Stefan Beller wrote: On Tue, Aug 28, 2018 at 1:33 PM Derrick Stolee via GitGitGadget wrote: From: Derrick Stolee The commit-graph feature is tested in isolation by t5318-commit-graph.sh and t6600-test-reach.sh, but there are many more interesting scenarios involving

Re: [PATCH v2 04/18] commit-reach: move commit_contains from ref-filter

2018-08-28 Thread Derrick Stolee
On 8/28/2018 5:24 PM, Jonathan Nieder wrote: Hi, Derrick Stolee wrote: There are several commit walks in the codebase. Group them together into a new commit-reach.c file and corresponding header. After we group these walks into one place, we can reduce duplicate logic by calling equivalent

Re: [PATCH 0/1] Define GIT_TEST_COMMIT_GRAPH for commit-graph test coverage

2018-08-28 Thread Derrick Stolee
On 8/28/2018 4:37 PM, Stefan Beller wrote: On Tue, Aug 28, 2018 at 1:33 PM Derrick Stolee via GitGitGadget wrote: The commit-graph (and multi-pack-index) features are optional data structures that can make Git operations faster. Since they are optional, we do not enable them in most Git tests.

Re: [PATCH 2/2] submodule.c: warn about missing submodule git directories

2018-08-28 Thread Stefan Beller
On Tue, Aug 28, 2018 at 11:56 AM Junio C Hamano wrote: > > Stefan Beller writes: > > > This is the continuation of f2d48994dc1 (submodule.c: submodule_move_head > > works with broken submodules, 2017-04-18), which tones down the case of > > "broken submodule" in case of a missing git directory

Re: [PATCH v4 10/11] rerere: teach rerere to handle nested conflicts

2018-08-28 Thread Thomas Gummerer
On 08/27, Junio C Hamano wrote: > Thomas Gummerer writes: > > > Agreed. I think it may be solvable if we'd actually get the > > information about what belongs to which side from the merge algorithm > > directly. > > The merge machinery may (eh, rather, "does") know, but we do not > have a way

Re: [PATCH 1/2] tests: fix non-portable "${var:-"str"}" construct

2018-08-28 Thread Junio C Hamano
Ævar Arnfjörð Bjarmason writes: > On both AIX 7200-00-01-1543 and FreeBSD 11.2-RELEASE-p2 the > "${var:-"str"}" syntax means something different than what it does > under the bash or dash shells. > > Both will consider the start of the new unescaped quotes to be a new > argument to

Re: Git clean removing tracked files semi-regularly

2018-08-28 Thread Derrick Stolee
On 8/28/2018 2:29 PM, Brennan Conroy wrote: I'm using windows. Have had it happen on 8.1 and 10. "git status" shows "nothing to commit, working tree clean". I am using git version 2.17.1.windows.2 I tested on my machine and could not reproduce this issue. Could you find a full repro (first

Re: [PATCH 1/1] commit-graph: define GIT_TEST_COMMIT_GRAPH

2018-08-28 Thread Stefan Beller
On Tue, Aug 28, 2018 at 1:33 PM Derrick Stolee via GitGitGadget wrote: > > From: Derrick Stolee > > The commit-graph feature is tested in isolation by > t5318-commit-graph.sh and t6600-test-reach.sh, but there are many > more interesting scenarios involving commit walks. Many of these >

Re: [PATCH v2 04/18] commit-reach: move commit_contains from ref-filter

2018-08-28 Thread Jonathan Nieder
Hi, Derrick Stolee wrote: > There are several commit walks in the codebase. Group them together into > a new commit-reach.c file and corresponding header. After we group these > walks into one place, we can reduce duplicate logic by calling > equivalent methods. > > All methods are direct moves,

Re: [PATCH v6] Implement --first-parent for git rev-list --bisect

2018-08-28 Thread Junio C Hamano
Junio C Hamano writes: > Something like the following, perhaps? Having said all that. > +# See the drawing near the top --- e4 is in the middle of the first parent > chain > +printf "%s\n" e4 | > +test_output_expect_success '--bisect --first-parent' ' > + git rev-list --bisect

[PATCH v3 06/11] t0064: make hash size independent

2018-08-28 Thread brian m. carlson
Compute test values of the appropriate size instead of hard-coding 40-character values. Rename the echo20 function to echoid, since the values may be of varying sizes. Signed-off-by: brian m. carlson --- t/t0064-sha1-array.sh | 49 --- 1 file changed, 27

[PATCH v3 02/11] t0000: use hash translation table

2018-08-28 Thread brian m. carlson
If the hash we're using is 32 bytes in size, attempting to insert a 20-byte object name won't work. Since these are synthesized objects that are almost all zeros, look them up in a translation table. Signed-off-by: brian m. carlson --- t/t-basic.sh | 13 +++-- 1 file changed, 7

[PATCH v3 08/11] t1400: switch hard-coded object ID to variable

2018-08-28 Thread brian m. carlson
Switch a hard-coded all-zeros object ID to use a variable instead. Signed-off-by: brian m. carlson --- t/t1400-update-ref.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh index 7c8df20955..6072650686 100755 ---

[PATCH v3 03/11] t0000: update tests for SHA-256

2018-08-28 Thread brian m. carlson
Test t tests the "basics of the basics" and as such, checks that we have various fixed hard-coded object IDs. The tests relying on these assertions have been marked with the SHA1 prerequisite, as they will obviously not function in their current form with SHA-256. Use the test_oid helper to

[PATCH v3 07/11] t1006: make hash size independent

2018-08-28 Thread brian m. carlson
Compute the size of the tree and commit objects we're creating by checking for the size of an object ID and computing the resulting sizes accordingly. Signed-off-by: brian m. carlson --- t/t1006-cat-file.sh | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git

[PATCH v3 09/11] t1405: make hash size independent

2018-08-28 Thread brian m. carlson
Instead of hard-coding a 40-based constant, split the output of for-each-ref and for-each-reflog by field. Signed-off-by: brian m. carlson --- t/t1405-main-ref-store.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/t1405-main-ref-store.sh b/t/t1405-main-ref-store.sh

[PATCH v3 11/11] t1407: make hash size independent

2018-08-28 Thread brian m. carlson
Instead of hard-coding a 40-based constant, split the output of for-each-ref and for-each-reflog by field. Signed-off-by: brian m. carlson --- t/t1407-worktree-ref-store.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/t1407-worktree-ref-store.sh

[RFC PATCH 00/12] Base SHA-256 algorithm implementation

2018-08-28 Thread brian m. carlson
This RFC series provides an actual SHA-256 implementation and wires it up, along with a few housekeeping patches to make it usable for testing. As discussed in some threads, this changes the algorithm name from "sha-1" to "sha1" (and also adds "sha256") because it's far easier to type. I

[RFC PATCH 03/12] hex: introduce functions to print arbitrary hashes

2018-08-28 Thread brian m. carlson
Currently, we have functions that turn an arbitrary SHA-1 value or an object ID into hex format, either using a static buffer or with a user-provided buffer. Add variants of these functions that can handle an arbitrary hash algorithm, specified by constant. Update the documentation as well.

[RFC PATCH 04/12] t: add basic tests for our SHA-1 implementation

2018-08-28 Thread brian m. carlson
We have in the past had some unfortunate endianness issues with some SHA-1 implementations we ship, especially on big-endian machines. Add an explicit test using the test helper to catch these issues and point them out prominently. This test can also be used as a staging ground for people

[RFC PATCH 05/12] t: make the sha1 test-tool helper generic

2018-08-28 Thread brian m. carlson
Since we're going to have multiple hash algorithms to test, it makes sense to share as much of the test code as possible. Convert the sha1 helper for the test-tool to be generic and move it out into its own module. This will allow us to share most of this code with our NewHash implementation.

[RFC PATCH 06/12] sha1-file: add a constant for hash block size

2018-08-28 Thread brian m. carlson
There is one place we need the hash algorithm block size: the HMAC code for push certs. Expose this constant in struct git_hash_algo and expose values for SHA-1 and for the largest value of any hash. Signed-off-by: brian m. carlson --- cache.h | 4 hash.h | 3 +++ sha1-file.c | 2

[RFC PATCH 09/12] Add a base implementation of SHA-256 support

2018-08-28 Thread brian m. carlson
SHA-1 is weak and we need to transition to a new hash function. For some time, we have referred to this new function as NewHash. The selection criteria for NewHash specify that it should (a) be 256 bits in length, (b) have high quality implementations available, (c) should match Git's needs in

[PATCH v3 01/11] t: add tool to translate hash-related values

2018-08-28 Thread brian m. carlson
Add a test function helper, test_oid, that produces output that varies depending on the hash in use. Add two additional helpers, test_oid_cache, which can be used to load data for test_oid from standard input, and test_oid_init, which can be used to load certain fixed values from lookup charts.

[PATCH v3 04/11] t0002: abstract away SHA-1 specific constants

2018-08-28 Thread brian m. carlson
Adjust the test so that it computes variables for object IDs instead of using hard-coded hashes. Signed-off-by: brian m. carlson --- t/t0002-gitfile.sh | 27 +++ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/t/t0002-gitfile.sh b/t/t0002-gitfile.sh index

[PATCH v3 05/11] t0027: make hash size independent

2018-08-28 Thread brian m. carlson
We transform various object IDs into all-zero object IDs for comparison. Adjust the length as well so that this works for all hash algorithms. Signed-off-by: brian m. carlson --- t/t0027-auto-crlf.sh | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/t/t0027-auto-crlf.sh

[RFC PATCH 07/12] t/helper: add a test helper to compute hash speed

2018-08-28 Thread brian m. carlson
Add a utility (which is less for the testsuite and more for developers) that can compute hash speeds for whatever hash algorithms are implemented. This allows developers to test their personal systems to determine the performance characteristics of various algorithms. Signed-off-by: brian m.

[RFC PATCH 08/12] commit-graph: convert to using the_hash_algo

2018-08-28 Thread brian m. carlson
Instead of using hard-coded constants for object sizes, use the_hash_algo to look them up. In addition, use a function call to look up the object ID version and produce the correct value. Signed-off-by: brian m. carlson --- commit-graph.c | 31 --- 1 file changed,

[RFC PATCH 10/12] sha256: add an SHA-256 implementation using libgcrypt

2018-08-28 Thread brian m. carlson
Generally, one gets better performance out of cryptographic routines written in assembly than C, and this is also true for SHA-256. In addition, most Linux distributions cannot distribute Git linked against OpenSSL for licensing reasons. Most systems with GnuPG will also have libgcrypt, since it

[RFC PATCH 12/12] commit-graph: specify OID version for SHA-256

2018-08-28 Thread brian m. carlson
Since the commit-graph code wants to serialize the hash algorithm into the data store, specify a version number for each supported algorithm. Note that we don't use the values of the constants themselves, as they are internal and could change in the future. Signed-off-by: brian m. carlson ---

[RFC PATCH 11/12] hash: add an SHA-256 implementation using OpenSSL

2018-08-28 Thread brian m. carlson
We already have OpenSSL routines available for SHA-1, so add routines for SHA-256 as well. Signed-off-by: brian m. carlson --- Makefile | 7 +++ hash.h | 2 ++ 2 files changed, 9 insertions(+) diff --git a/Makefile b/Makefile index 86867af083..8b7df4dfc5 100644 --- a/Makefile +++

[PATCH v3 10/11] t1406: make hash-size independent

2018-08-28 Thread brian m. carlson
Instead of hard-coding a 40-based constant, split the output of for-each-ref and for-each-reflog by field. Signed-off-by: brian m. carlson --- t/t1406-submodule-ref-store.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/t/t1406-submodule-ref-store.sh

avoid "Set preference list" during make test?

2018-08-28 Thread Tacitus Aedifex
While running `make test` on the git source tree I keep getting asked: Set preference list to: Cipher: ... Digest: ... etc... Is there any way to turn that prompt off so that `make test` completes without any keyboard input? //tæ

Re: [PATCH] read-cache.c: optimize reading index format v4

2018-08-28 Thread Ben Peart
On 8/28/2018 3:25 PM, Duy Nguyen wrote: On Mon, Aug 27, 2018 at 9:36 PM Junio C Hamano wrote: PS. I notice that v4 does not pad to align entries at 4 byte boundary like v2/v3. This could cause a slight slow down on x86 and segfault on some other platforms. Care to elaborate? Long time

Re: [PATCH 2/2] tests: fix non-portable iconv invocation

2018-08-28 Thread Jonathan Nieder
Hi, Ævar Arnfjörð Bjarmason wrote: > The iconv that comes with a FreeBSD 11.2-RELEASE-p2 box I have access > to doesn't support the SHIFT-JIS encoding. Guard a test added in > e92d62253 ("convert: add round trip check based on > 'core.checkRoundtripEncoding'", 2018-04-15) first released with Git

Re: avoid "Set preference list" during make test?

2018-08-28 Thread brian m. carlson
On Tue, Aug 28, 2018 at 11:53:55PM +, Tacitus Aedifex wrote: > While running `make test` on the git source tree I keep getting asked: > > Set preference list to: >Cipher: ... >Digest: ... >etc... > > Is there any way to turn that prompt off so that `make test` completes >

[PATCH v3 00/11] Hash-independent tests (part 3)

2018-08-28 Thread brian m. carlson
This is the next in the series of improvements to make tests hash-independent. A range-diff is below. Changes from v2: * Fix a typo in "zero_2". * Provide better matching of expected output. * Add and use test_oid_init instead of filename-based test_oid_cache. * Add test_set_hash. * Provide

[RFC PATCH 01/12] sha1-file: rename algorithm to "sha1"

2018-08-28 Thread brian m. carlson
The transition plan anticipates us using a syntax such as "^{sha1}" for disambiguation. Since this is a syntax some people will be typing a lot, it makes sense to provide a short, easy-to-type syntax. Omitting the dash doesn't create any ambiguity, but it does make it shorter and easier to type,

[RFC PATCH 02/12] sha1-file: provide functions to look up hash algorithms

2018-08-28 Thread brian m. carlson
There are several ways we might refer to a hash algorithm: by name, such as in the config file; by format ID, such as in a pack; or internally, by a pointer to the hash_algos array. Provide functions to look up hash algorithms based on these various forms and return the internal constant used for

Re: [PATCH v3 01/11] t: add tool to translate hash-related values

2018-08-28 Thread Jonathan Nieder
Hi, brian m. carlson wrote: > Add a test function helper, test_oid, that produces output that varies > depending on the hash in use. Cool! >Add two additional helpers, > test_oid_cache, which can be used to load data for test_oid from > standard input, and

Re: Contributor Summit planning

2018-08-28 Thread Jeff King
On Mon, Aug 27, 2018 at 03:22:39PM +0200, Johannes Schindelin wrote: > Having said that, I believe that we core contributors can learn to have a > fruitful online meeting. With 30+ participants, too. > > Learning from my past life in academia (it is hard for me to imagine a > less disciplined

Re: Contributor Summit planning

2018-08-28 Thread Jeff King
On Mon, Aug 27, 2018 at 03:34:16PM +0200, Johannes Schindelin wrote: > > - format > > > > For those who haven't attended before, it's basically 25-ish Git > > (and associated project) developers sitting in a room for a day > > chatting about the project. Topics go on a whiteboard

Re: Contributor Summit planning

2018-08-28 Thread Jeff King
On Tue, Aug 28, 2018 at 12:49:55AM +0200, Johannes Schindelin wrote: > On Mon, 13 Aug 2018, Ævar Arnfjörð Bjarmason wrote: > > > * Re the second half of "Not everyone can travel or can afford to do > >so" from Derrick, there's been travel sponsorships in past years. > > Just to make sure

Re: Thank you for public-inbox!

2018-08-28 Thread Jeff King
On Mon, Aug 27, 2018 at 04:25:13PM +0200, Johannes Schindelin wrote: > I would like to take five minutes to thank you for public-inbox. It is > invaluable for me in the meantime. And I think I will never be able to > thank you enough for it. Let me echo that appreciation. I have always kept my

Re: [PATCH 2/2] submodule: munge paths to submodule git directories

2018-08-28 Thread Jeff King
On Tue, Aug 28, 2018 at 02:35:25PM -0700, Stefan Beller wrote: > 3) (optional) instead of putting it all in modules/, use another >directory gitmodules/ for example. this will make sure we can tell >if a repository has been converted or is stuck with a setup of a >current git. I

Re: [PATCH 0/9] introducing oideq()

2018-08-28 Thread brian m. carlson
On Tue, Aug 28, 2018 at 05:21:27PM -0400, Jeff King wrote: > On Sun, Aug 26, 2018 at 08:56:21PM +, brian m. carlson wrote: > > I would quite like to see this series picked up for v2.20. If we want > > to minimize performance regressions with the SHA-256 work, I think it's > > important. > >

Re: [PATCH] t5310-pack-bitmaps: fix bogus 'pack-objects to file can use bitmap' test

2018-08-28 Thread Kirill Smelkov
On Mon, Aug 27, 2018 at 07:04:52PM -0400, Jeff King wrote: > On Mon, Aug 27, 2018 at 10:22:46AM +, Kirill Smelkov wrote: > > > A minor comment from outside observer: running tests under something > > like > > > > -e and -o pipefail > > > > would automatically catch the mistake in the

<    1   2