Re: reftable [v6]: new ref storage format

[Shawn Pearce]

6th iteration of the reftable storage format.

You can read a rendered version of this here:
https://googlers.googlesource.com/sop/jgit/+/reftable/Documentation/technical/reftable.md

Changes from v5:
- extensions.refStorage = reftable is used to select this format.

- Log records can be explicitly deleted (for refs/stash).
- Log records may use Michael Haggerty's chained idea to compress before zlib.
  This saved ~5.8% on one of my example repositories.


# reftable

[TOC]

## Overview

### Problem statement

Some repositories contain a lot of references (e.g.  android at 866k,
rails at 31k).  The existing packed-refs format takes up a lot of
space (e.g.  62M), and does not scale with additional references.
Lookup of a single reference requires linearly scanning the file.

Atomic pushes modifying multiple references require copying the
entire packed-refs file, which can be a considerable amount of data
moved (e.g. 62M in, 62M out) for even small transactions (2 refs
modified).

Repositories with many loose references occupy a large number of disk
blocks from the local file system, as each reference is its own file
storing 41 bytes (and another file for the corresponding reflog).
This negatively affects the number of inodes available when a large
number of repositories are stored on the same filesystem.  Readers can
be penalized due to the larger number of syscalls required to traverse
and read the `$GIT_DIR/refs` directory.

### Objectives

- Near constant time lookup for any single reference, even when the
  repository is cold and not in process or kernel cache.
- Near constant time verification a SHA-1 is referred to by at least
  one reference (for allow-tip-sha1-in-want).
- Efficient lookup of an entire namespace, such as `refs/tags/`.
- Support atomic push with `O(size_of_update)` operations.
- Combine reflog storage with ref storage for small transactions.
- Separate reflog storage for base refs and historical logs.

### Description

A reftable file is a portable binary file format customized for
reference storage. References are sorted, enabling linear scans,
binary search lookup, and range scans.

Storage in the file is organized into blocks.  Prefix compression
is used within a single block to reduce disk space.  Block size is
tunable by the writer.

### Performance

Space used, packed-refs vs. reftable:

repository | packed-refs | reftable | % original | avg ref  | avg obj
---|:|-:|---:|-:|:
android|  62.2 M |   34.4 M | 55.2%  | 33 bytes | 8 bytes
rails  |   1.8 M |1.1 M | 57.7%  | 29 bytes | 6 bytes
git|  78.7 K |   44.0 K | 60.0%  | 50 bytes | 6 bytes
git (heads)|   332 b |274 b | 83.1%  | 34 bytes | 0 bytes

Scan (read 866k refs), by reference name lookup (single ref from 866k
refs), and by SHA-1 lookup (refs with that SHA-1, from 866k refs):

format  | cache | scan| by name| by SHA-1
|--:|:|---:|---:
packed-refs | cold  |  402 ms | 409,660.1 usec | 412,535.8 usec
packed-refs | hot   | |   6,844.6 usec |  20,110.1 usec
reftable| cold  |  112 ms |  33.9 usec | 323.2 usec
reftable| hot   | |  20.2 usec | 320.8 usec

Space used for 149,932 log entries for 43,061 refs,
reflog vs. reftable:

format| size  | avg entry
--|--:|---:
$GIT_DIR/logs | 173 M | 1209 bytes
reftable  |   5 M |   37 bytes

## Details

### Peeling

References stored in a reftable are peeled, a record for an annotated
(or signed) tag records both the tag object, and the object it refers
to.

### Reference name encoding

Reference names are an uninterpreted sequence of bytes that must pass
[git-check-ref-format][ref-fmt] as a valid reference name.

[ref-fmt]: https://git-scm.com/docs/git-check-ref-format

### Network byte order

All multi-byte, fixed width fields are in network byte order.

### Ordering

Blocks are lexicographically ordered by their first reference.

### Directory/file conflicts

The reftable format accepts both `refs/heads/foo` and
`refs/heads/foo/bar` as distinct references.

This property is useful for retaining log records in reftable, but may
confuse versions of Git using `$GIT_DIR/refs` directory tree to
maintain references.  Users of reftable may choose to continue to
reject `foo` and `foo/bar` type conflicts to prevent problems for
peers.

## File format

### Structure

A reftable file has the following high-level structure:

first_block {
  header
  first_ref_block
}
ref_blocks*
ref_index?
obj_blocks*
obj_index?
log_blocks*
log_index?
footer

A log-only file omits the `ref_blocks`, `ref_index`, `obj_blocks` and
`obj_index` sections, containing only the file header and log blocks:

first_block {
  header
}
log_blocks*
log_index?
footer

in a log-only file the first log block immediately 

Re: [RFC] imap-send: escape backslash in password

[Junio C Hamano]

Jeff King  writes:

> That is fine by me. AFAIK, we already build the curl support by default
> when a sufficiently-advanced version of curl is available. So if there
> were feature-parity problems hopefully somebody would have reported it.
>
> I think the deprecation here can be relatively fast because we're not
> actually dropping support for any feature. We're just requiring that
> they install curl to get the same functionality (which might be
> inconvenient, but it's a heck of a lot less inconvenient than "there's
> no way to do what you want anymore").

Yeah, as long as imap-supporting libcurl is not too recent and are
available everywhere, we should be OK.

Thanks.

Re: [PATCH] tests: don't give unportable ">" to "test" built-in, use -gt

[Junio C Hamano]

Ævar Arnfjörð Bjarmason   writes:

> Change an argument to test_line_count (which'll ultimately be turned
> into a "test" expression) to use "-gt" instead of ">" for an
> arithmetic test.
>
> This broken on e.g. OpenBSD as of v2.13.0 with my commit
> ac3f5a3468 ("ref-filter: add --no-contains option to
> tag/branch/for-each-ref", 2017-03-24).
>
> Upstream just worked around it by patching git and didn't tell us
> about it, I discovered this when reading various Git packaging
> implementations: https://github.com/openbsd/ports/commit/7e48bf88a20

Thanks for finding and relaying this.

> Signed-off-by: Ævar Arnfjörð Bjarmason 
> ---
>
> David, it would be great to get a quick bug report to
> git@vger.kernel.org if you end up having to monkeypatch something
> we've done. We won't bite, promise :)

Yeah.  I was hoping that your new list for platform porters would
help communicate these issues easier for those who are a bit shy to
be on this general development list ;-)

> As shown in that linked Github commit OpenBSD has another recent
> workaround in turning on DIR_HAS_BSD_GROUP_SEMANTICS and skipping a
> related test, maybe René can make more sense of that?
>
> There's more patches in their ports which indicate possible bugs of
> ours: https://github.com/openbsd/ports/tree/master/devel/git/patches/
>
>  t/t7004-tag.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh
> index 0ef7b94394..0e2e57aa3d 100755
> --- a/t/t7004-tag.sh
> +++ b/t/t7004-tag.sh
> @@ -1887,7 +1887,7 @@ EOF"
>   run_with_limited_stack git tag --contains HEAD >actual &&
>   test_cmp expect actual &&
>   run_with_limited_stack git tag --no-contains HEAD >actual &&
> - test_line_count ">" 10 actual
> + test_line_count "-gt" 10 actual
>  '
>  
>  test_expect_success '--format should list tags as per format given' '

Re: [PATCH] Drop some dashes from built-in invocations in scripts

[Junio C Hamano]

Michael Forney  writes:

> Ah. Well, it looks like all but git-merge-resolve.sh run `.
> git-sh-setup`, so we know that GIT_EXEC_PATH must in their PATH (and
> at the front unless the script was invoked directly).
>
> git-merge-resolve.sh does not do this, so I suppose if the user ran
> $GIT_EXEC_PATH/git-merge-resolve directly, and also had a custom
> git-merge-index executable in their PATH, that would switch to running
> the git merge-index built-in instead.

We are good then.  Nobody other than "git merge" should be running
merge-resolve, so the original probably is OK.

Thanks for digging deeper.

Re: [PATCH] tests: don't give unportable ">" to "test" built-in, use -gt

[brian m. carlson]

On Sun, Aug 06, 2017 at 11:38:50PM +, Ævar Arnfjörð Bjarmason wrote:
> Change an argument to test_line_count (which'll ultimately be turned
> into a "test" expression) to use "-gt" instead of ">" for an
> arithmetic test.
> 
> This broken on e.g. OpenBSD as of v2.13.0 with my commit
> ac3f5a3468 ("ref-filter: add --no-contains option to
> tag/branch/for-each-ref", 2017-03-24).
> 
> Upstream just worked around it by patching git and didn't tell us
> about it, I discovered this when reading various Git packaging
> implementations: https://github.com/openbsd/ports/commit/7e48bf88a20
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason 
> ---
> 
> David, it would be great to get a quick bug report to
> git@vger.kernel.org if you end up having to monkeypatch something
> we've done. We won't bite, promise :)
> 
> As shown in that linked Github commit OpenBSD has another recent
> workaround in turning on DIR_HAS_BSD_GROUP_SEMANTICS and skipping a
> related test, maybe René can make more sense of that?

I've confirmed using the NetBSD 7.1 man pages that NetBSD will also want
DIR_HAS_BSD_GROUP_SEMANTICS.  MirBSD will also, according to its man
pages.

As I understand it, the only consequence of not setting this flag on BSD
systems is that some directories will be setgid, which, while ugly and
useless, should have no negative effect.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
https://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: https://keybase.io/bk2204


signature.asc
Description: PGP signature

[PATCH] tests: don't give unportable ">" to "test" built-in, use -gt

[Ævar Arnfjörð Bjarmason]

Change an argument to test_line_count (which'll ultimately be turned
into a "test" expression) to use "-gt" instead of ">" for an
arithmetic test.

This broken on e.g. OpenBSD as of v2.13.0 with my commit
ac3f5a3468 ("ref-filter: add --no-contains option to
tag/branch/for-each-ref", 2017-03-24).

Upstream just worked around it by patching git and didn't tell us
about it, I discovered this when reading various Git packaging
implementations: https://github.com/openbsd/ports/commit/7e48bf88a20

Signed-off-by: Ævar Arnfjörð Bjarmason 
---

David, it would be great to get a quick bug report to
git@vger.kernel.org if you end up having to monkeypatch something
we've done. We won't bite, promise :)

As shown in that linked Github commit OpenBSD has another recent
workaround in turning on DIR_HAS_BSD_GROUP_SEMANTICS and skipping a
related test, maybe René can make more sense of that?

There's more patches in their ports which indicate possible bugs of
ours: https://github.com/openbsd/ports/tree/master/devel/git/patches/

 t/t7004-tag.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh
index 0ef7b94394..0e2e57aa3d 100755
--- a/t/t7004-tag.sh
+++ b/t/t7004-tag.sh
@@ -1887,7 +1887,7 @@ EOF"
run_with_limited_stack git tag --contains HEAD >actual &&
test_cmp expect actual &&
run_with_limited_stack git tag --no-contains HEAD >actual &&
-   test_line_count ">" 10 actual
+   test_line_count "-gt" 10 actual
 '
 
 test_expect_success '--format should list tags as per format given' '
-- 
2.14.0.rc1.383.gd1ce394fe2

Re: reftable [v5]: new ref storage format

[Shawn Pearce]

On Sun, Aug 6, 2017 at 9:56 AM, Ævar Arnfjörð Bjarmason
 wrote:
> On Sun, Aug 06 2017, Shawn Pearce jotted:
>
>> 5th iteration of the reftable storage format.
>
> I haven't kept up with all of the discussion, sorry if these comments
> repeat something that's already mentioned.
>
>> ### Version 1
>>
>> A repository must set its `$GIT_DIR/config` to configure reftable:
>>
>> [core]
>> repositoryformatversion = 1
>> [extensions]
>> reftable = true
>
> David Turner's LMDB proposal specified a extensions.refStorage config
> variable instead. I think this is a much better idea, cf. the mistake we
> already made with grep.extendedRegexp & grep.patternType. I.e. to have
> 'extensions.refStorage = reftable' instead of 'extensions.reftable =
> true'.
>
> If we grow another storage backend this'll become messy, and it won't be
> obvious to the user that the configuration is mutually exclusive (which
> it surely will be), so we'll end up having to special case it similar to
> the grep.[extendedRegexp,patternType] (i.e. either make one override the
> other, or make specifying >1 an error, a hassle with the config API).

Good catch. I've fixed this to use extensions.refStorage. Thanks!


>> Performance testing indicates reftable is faster for lookups (51%
>> faster, 11.2 usec vs.  5.4 usec), although reftable produces a
>> slightly larger file (+ ~3.2%, 28.3M vs 29.2M):
>>
>> format|  size  | seek cold | seek hot  |
>> -:|---:|--:|--:|
>> mh-alt| 28.3 M | 23.4 usec | 11.2 usec |
>> reftable  | 29.2 M | 19.9 usec |  5.4 usec |
>>
>> [mh-alt]: 
>> https://public-inbox.org/git/camy9t_hcnyc1g8xwoowhe7nn0aefyybskv2aomb_fe+wgve...@mail.gmail.com/
>
> Might be worth noting "based on WIP Java implementation". I started
> searching for patches for this new format & found via
> 

HELLO,

[infob...@ono.com]

Liebste,
 
Wie gehts Ihnen heute, ich hoffe, dass meine Post Sie in gutem Zustand 
der Gesundheit erfüllt? Sehr geehrte Damen und Herren, ich habe mich 
entschlossen, mit Ihnen in Kontakt zu treten, wenn ich bedenkt habe, 
dass wir uns noch nicht getroffen haben, aber wegen eines Umstandes, 
der mich verpflichtet hat, habe ich mich entschlossen, Sie wegen der 
Dringlichkeit meiner gegenwärtigen Situation hier im Flüchtlingslager 
für Ihre Rettung zu kontaktieren Auch für ein Business-Venture / 
Projekt, das ich brauche Ihre Assistentin in diesem Business-
Niederlassung in Ihrem Land als mein ausländischer Partner sowie meine 
juristische ernannte Treuhänder.
 
Ich bin Aisha Muammar Gaddafi, die einzige Tochter des umkämpften 
Präsidenten von Libyen, Hon. Muammar Gaddafi Ich wohne derzeit in 
Burkina Faso leider als Flüchtling. Ich schreibe diese Post mit Tränen 
und Trauer aus meinem Herzen und frage nach deiner dringenden Hilfe. 
Ich bin seit dem Tod meines verstorbenen Vaters durch Schmerzen und 
traurigen Moment gegangen.
 
Mittlerweile ist meine Familie das Ziel der westlichen Nationen, die 
von Nato geführt werden, die meinen Vater um jeden Preis zerstören 
will. Unsere Investitionen und Bankkonten in mehreren Ländern sind ihre 
Ziele zum Einfrieren. Mein Vater des gesegneten Gedächtnisses stellte 
die Summe von $ 10.5M (Zehn Millionen, Fünfhundert Tausend Dollar) in 
(ECO) Bank Burkina Faso, die er benutzte meinen Namen als die nächste 
Angehörige. Ich habe von der (ECO) Bank beauftragt, einen 
interessierten ausländischen Investor / Partner zu präsentieren, der 
als Treuhänder stehen kann und den Fonds in seinem Konto für eine 
mögliche Investition in sein Land aufgrund meines Flüchtlingsstatus 
hier in Burkina Faso erhalten kann.
 
Ich bin auf der Suche nach einer ehrlichen und zuverlässigen Person, 
die mir helfen und als Treuhänder stehen wird, damit ich ihn der Bank 
für die Übertragung des Fonds auf sein Bankkonto in Übersee vorstellen 
werde. Ich habe beschlossen, dich nach meinen Gebeten zu kontaktieren 
und ich glaube, dass du mein Vertrauen nicht verraten wirst. Aber 
nehmen Sie mich als Ihre eigene Schwester oder Tochter. Wenn diese 
Transaktion Sie interessiert, müssen Sie es niemandem offenlegen, weil 
was mit meiner ganzen Familie los ist, wenn die vereinigte Nation 
dieses Konto kennt, werden sie es einfrieren, wenn sie andere 
einfrieren, also bitte diese Transaktion behalten Nur für dich, bis wir 
es fertig machen
 
Apologetic für meine Bilder werde ich es in meine nächste Mail und mehr 
über mich einschließen, wenn ich von dir höre okay. Bitte, ich möchte, 
dass Sie mich hier für weitere Konversationen kontaktieren. Danke und 
beste Grüße, Mit freundlichen Grüßen.
 
Aisha Gaddafi

FINANCING ASSISTANCE AT 2% INTEREST RATE

[Lucas Brown]

GOOD DAY,WE ARE OFFERING FINANCING ASSISTANCE AT 2% INTEREST RATE IF YOU NEED A 
LOAN CONTACT US 

Re: [RFC] imap-send: escape backslash in password

[Nicolas Morey-Chaisemartin]

Le 04/08/2017 à 23:22, Jeff King a écrit :
> On Fri, Aug 04, 2017 at 02:18:13PM -0700, Junio C Hamano wrote:
>
>>> I also think it might be reasonable to scrap all of this ad-hoc imap
>>> code in favor of curl, which already gets these cases right. We already
>>> have a curl-backed implementation. I think we just left the old code out
>>> of conservatism. But it seems somewhat buggy and unmaintained, and I
>>> wonder if we aren't better off to simply encourage people to install
>>> curl.
>> That is a very attractive direction to go in, especially in the mid
>> to longer term.  Perhaps we declare that the ad-hoc hardcoded imap
>> is deprecated in the next cycle and drop the support by the end of
>> this year?
> That is fine by me. AFAIK, we already build the curl support by default
> when a sufficiently-advanced version of curl is available. So if there
> were feature-parity problems hopefully somebody would have reported it.
>
> I think the deprecation here can be relatively fast because we're not
> actually dropping support for any feature. We're just requiring that
> they install curl to get the same functionality (which might be
> inconvenient, but it's a heck of a lot less inconvenient than "there's
> no way to do what you want anymore").
>
> -Peff

There is at least one difference right now:
When using --curl, the username/password are loaded from the gitconfig file 
only.
When using the legacy imap interface, it goes through credential_fill which 
prompts for a password.

I don't think everyone is ready to store his email account password in a 
gitconfig file (I know I'm not).
I don't see why it couldn't be fixed. I'll give it a try tomorrow.

Also it probably make sense to have at least one release where --curl is the 
default. Until your mail I had no idea this option existed so I never tried it 
out.
Making it the default will make sure almost everyone is using it and that there 
is feature-parity.

But I agree it's probably safer and cleaner to let curl handle everything and 
drop the legacy stuff once it fully works.

Nicolas

Re: What's cooking in git.git (Jul 2017, #09; Mon, 31)

[Ævar Arnfjörð Bjarmason]

On Sat, Aug 05 2017, Junio C. Hamano jotted:

> Ævar Arnfjörð Bjarmason  writes:
>
>> We've talked how this UX should look before on-list. Rather than
>> re-hashing the arguments I had before I thought it was useful to present
>> it as a table. Here's how the patch looks now:
>>
>> 
>> |+--+---+-+-+-|
>> | cmd| creates new? | moves | copies? | ...with 
>> config? | checks out? |
>> 
>> |+--+---+-+-+-|
>> | branch   | Y| N | N   | N
>>| N   |
>> | checkout | Y| N | N   | N
>>| Y   |
>> | checkout -b   | Y| N | Y   | N
>>| Y   |
>> | branch -m| Y| Y | N   | Y
>>| Y   |
>> | NEW: branch -c   | Y| N | Y   | Y
>>| Y   |
>> 
>> |+--+---+-+-+-|
>
> I actually consider "branch" to *never* invoking a checkout.  Even
> when "git branch -m A B" happens to be done when your checked out
> branch is A and you end up being on B.  That is not a "checkout".

I think we just have a different mental model of what "checkout"
means. In my mind any operation that updates the HEAD to point to a new
branch is a checkout of that branch.

In the interest of explaining what I mean with the above and not proving
you wrong or whatever, consider a basic implementation of git before
pack-refs or whatever.

You could then copy a branch name like:

$ cp .git/refs/heads/{master,master-copy}

Get your currently checked out branch with:

$ cat .git/HEAD
ref: refs/heads/master

And you could then implement "git checkout" like:

$ echo "ref: refs/heads/master-copy" >.git/HEAD

This still works today. Now if I make a commit it goes on the
.git/refs/heads/master-copy branch.

Now let's say I wanted to rename the 'master' branch. This would rename
it:

$ mv .git/refs/heads/{master,trunk}

But it wouldn't check it out, my HEAD would still point to master:

$ git log
fatal: your current branch 'master' does not have any commits yet
$ git log --oneline -1 trunk
8b18718 (trunk) moo

To check it out I need:

$ echo "ref: refs/heads/trunk" >.git/HEAD

Which yields:

$ git log --oneline -1 HEAD
8b18718 (HEAD -> trunk) moo

The mv sans the "echo" is what we'd get if we did:

diff --git a/builtin/branch.c b/builtin/branch.c
index 8a0595e115..62ed1a8e20 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -493,9 +493,6 @@ static void rename_branch(const char *oldname, const 
char *newname, int force)
if (recovery)
warning(_("Renamed a misnamed branch '%s' away"), 
oldref.buf + 11);

-   if (replace_each_worktree_head_symref(oldref.buf, newref.buf, 
logmsg.buf))
-   die(_("Branch renamed to %s, but HEAD is not updated!"), 
newname);
-
strbuf_release();

strbuf_addf(, "branch.%s", oldref.buf + 11);

I.e. the last step of "git branch -m" is checking out the branch by
updating the HEAD. Now let's compile git like that and rename "master"
to "trunk":

$ git symbolic-ref HEAD
refs/heads/master
$ ./git-branch -m trunk
$ git symbolic-ref HEAD
refs/heads/master
$ git show
fatal: your current branch 'master' does not have any commits yet

Now let's do the equivalent of that removed
replace_each_worktree_head_symref():

$ git checkout trunk
Switched to branch 'trunk'

And voila:

$ git symbolic-ref HEAD
refs/heads/trunk

This is why I'm saying that "git branch -m" involves a checkout. Because
it is literally exactly the case that it could be replaced by a
shellscript whose last step is a "git checkout" of the new branch.

Anyway, none of that changes what we /should/ be doing on the fact that
"git branch -m" not updating the HEAD would be rather useless. I just
wanted to explain what I was talking about and why I was talking about
"checkout" in that context.

> Really from the end-user's point of view that is not a checkout.
> The user renamed the branch A and the same conceptual entity, which
> is a branch, is now called B.  If that branch was what was checked
> out (IOW, if that branch was what would be grown by one commit if
> the user did "git commit"), then now that branch's name is B.  It is
> natural if you ask "symbolic-ref HEAD" what branch is checked out
> after renaming A to B (and A happened to be what was checked out),
> the answer chould be B.
>
> It's like the city you live in changed the name of the street your
> house is on.  You do not call movers, you do not do anything, but
> your 

Re: [PATCH for NEXT v3 2/2] sub-process: refactor handshake to common function

[Lars Schneider]

> On 26 Jul 2017, at 20:17, Jonathan Tan  wrote:
> 
> Refactor, into a common function, the version and capability negotiation
> done when invoking a long-running process as a clean or smudge filter.
> This will be useful for other Git code that needs to interact similarly
> with a long-running process.
> 
> As you can see in the change to t0021, this commit changes the error
> message reported when the long-running process does not introduce itself
> with the expected "server"-terminated line. Originally, the error
> message reports that the filter "does not support filter protocol
> version 2", differentiating between the old single-file filter protocol
> and the new multi-file filter protocol - I have updated it to something
> more generic and useful.
> 
> Signed-off-by: Jonathan Tan 
> ---
> convert.c |  75 
> pkt-line.c|  19 --
> pkt-line.h|   2 -
> sub-process.c | 103 ++
> sub-process.h |  26 +
> t/t0021-conversion.sh |   2 +-
> 6 files changed, 137 insertions(+), 90 deletions(-)
> 
> diff --git a/convert.c b/convert.c
> index dbdbb24e4..1012462e3 100644
> --- a/convert.c
> +++ b/convert.c
> @@ -513,78 +513,17 @@ static struct hashmap subprocess_map;
> 
> static int start_multi_file_filter_fn(struct subprocess_entry *subprocess)
> {
> - int err, i;
> - struct cmd2process *entry = (struct cmd2process *)subprocess;
> - struct string_list cap_list = STRING_LIST_INIT_NODUP;
> - char *cap_buf;
> - const char *cap_name;
> - struct child_process *process = >process;
> - const char *cmd = subprocess->cmd;
> -
> - static const struct {
> - const char *name;
> - unsigned int cap;
> - } known_caps[] = {
> + static int versions[] = {2, 0};
> + static struct subprocess_capability capabilities[] = {
>   { "clean",  CAP_CLEAN  },
>   { "smudge", CAP_SMUDGE },
>   { "delay",  CAP_DELAY  },
> + { NULL, 0 }
>   };
> -
> - sigchain_push(SIGPIPE, SIG_IGN);
> -
> - err = packet_writel(process->in, "git-filter-client", "version=2", 
> NULL);
> - if (err)
> - goto done;
> -
> - err = strcmp(packet_read_line(process->out, NULL), "git-filter-server");
> - if (err) {
> - error("external filter '%s' does not support filter protocol 
> version 2", cmd);
> - goto done;
> - }
> - err = strcmp(packet_read_line(process->out, NULL), "version=2");
> - if (err)
> - goto done;
> - err = packet_read_line(process->out, NULL) != NULL;
> - if (err)
> - goto done;
> -
> - for (i = 0; i < ARRAY_SIZE(known_caps); ++i) {
> - err = packet_write_fmt_gently(
> - process->in, "capability=%s\n", known_caps[i].name);
> - if (err)
> - goto done;
> - }
> - err = packet_flush_gently(process->in);
> - if (err)
> - goto done;
> -
> - for (;;) {
> - cap_buf = packet_read_line(process->out, NULL);
> - if (!cap_buf)
> - break;
> - string_list_split_in_place(_list, cap_buf, '=', 1);
> -
> - if (cap_list.nr != 2 || strcmp(cap_list.items[0].string, 
> "capability"))
> - continue;
> -
> - cap_name = cap_list.items[1].string;
> - i = ARRAY_SIZE(known_caps) - 1;
> - while (i >= 0 && strcmp(cap_name, known_caps[i].name))
> - i--;
> -
> - if (i >= 0)
> - entry->supported_capabilities |= known_caps[i].cap;
> - else
> - warning("external filter '%s' requested unsupported 
> filter capability '%s'",
> - cmd, cap_name);
> -
> - string_list_clear(_list, 0);
> - }
> -
> -done:
> - sigchain_pop(SIGPIPE);
> -
> - return err;
> + struct cmd2process *entry = (struct cmd2process *)subprocess;
> + return subprocess_handshake(subprocess, "git-filter", versions, NULL,
> + capabilities,
> + >supported_capabilities);

Wouldn't it make sense to add `supported_capabilities` to `struct 
subprocess_entry` ?


> }
> 
> static void handle_filter_error(const struct strbuf *filter_status,
> diff --git a/pkt-line.c b/pkt-line.c
> index 9d845ecc3..7db911957 100644
> --- a/pkt-line.c
> +++ b/pkt-line.c
> @@ -171,25 +171,6 @@ int packet_write_fmt_gently(int fd, const char *fmt, ...)
>   return status;
> }
> 
> -int packet_writel(int fd, const char *line, ...)
> -{
> - va_list args;
> - int err;
> - va_start(args, line);
> - for (;;) {
> - if (!line)
> - break;
> - if (strlen(line) > 

git checkout's stderr v stdout usage

[prateekrungta]

```
  git checkout -b new_branch 
```

Git does what's asked except that Git's success response ("Switched to a new
branch 'new_branch'"), goes to to stderr instead of stdout. 

This feels against standard unix conventions of what to output to stderr v
stdout. It's an extra jump that people scripting `Git` keep running into
when they encounter this behaviour. For e.g. other users have complained
about this as far back as 2010:
http://git.661346.n2.nabble.com/Bugreport-Git-responds-with-stderr-instead-of-stdout-td4959280.html

I'd like to suggest changing informational messages to stdout, and erroneous
ones to stderr. May I submit a patch to change this behaviour? 






--
View this message in context: 
http://git.661346.n2.nabble.com/git-checkout-s-stderr-v-stdout-usage-tp7657477.html
Sent from the git mailing list archive at Nabble.com.

Re: Contact with Latinamerica

[Christopher Díaz]

Hi all :) 

I just wanted to let you know how are we going :)

We can say now that we are international! New members are working with
us from Venezuela! 

By now we are on the 50% of our git translation project, we hope to
have it ready for the next release :)

This is our first draft of the website, it took me a while to finish
the legal stuff but hopefully in a couple of weeks we are going to have
 all done and our english version of the web for you to share it :)

https://codelabora.github.io/

Please feel free to add any comment or info that you want us to tell
here.

Thanks a lot for your participation in this project
Christopher Díaz Riveros

Re: reftable [v5]: new ref storage format

[Ævar Arnfjörð Bjarmason]

On Sun, Aug 06 2017, Shawn Pearce jotted:

> 5th iteration of the reftable storage format.

I haven't kept up with all of the discussion, sorry if these comments
repeat something that's already mentioned.

> ### Version 1
>
> A repository must set its `$GIT_DIR/config` to configure reftable:
>
> [core]
> repositoryformatversion = 1
> [extensions]
> reftable = true

David Turner's LMDB proposal specified a extensions.refStorage config
variable instead. I think this is a much better idea, cf. the mistake we
already made with grep.extendedRegexp & grep.patternType. I.e. to have
'extensions.refStorage = reftable' instead of 'extensions.reftable =
true'.

If we grow another storage backend this'll become messy, and it won't be
obvious to the user that the configuration is mutually exclusive (which
it surely will be), so we'll end up having to special case it similar to
the grep.[extendedRegexp,patternType] (i.e. either make one override the
other, or make specifying >1 an error, a hassle with the config API).

> Performance testing indicates reftable is faster for lookups (51%
> faster, 11.2 usec vs.  5.4 usec), although reftable produces a
> slightly larger file (+ ~3.2%, 28.3M vs 29.2M):
>
> format|  size  | seek cold | seek hot  |
> -:|---:|--:|--:|
> mh-alt| 28.3 M | 23.4 usec | 11.2 usec |
> reftable  | 29.2 M | 19.9 usec |  5.4 usec |
>
> [mh-alt]: 
> https://public-inbox.org/git/camy9t_hcnyc1g8xwoowhe7nn0aefyybskv2aomb_fe+wgve...@mail.gmail.com/

Might be worth noting "based on WIP Java implementation". I started
searching for patches for this new format & found via

Re: [git-for-windows] [ANNOUNCE] Git for Windows 2.14.0

[Lars Schneider]

> On 06 Aug 2017, at 01:00, Johannes Schindelin  
> wrote:
> 
> Dear Git users,
> 
> It is my pleasure to announce that Git for Windows 2.14.0 is available from:

Thank you :-)


>  * It is now possible to switch between Secure Channel and OpenSSL for
>Git's HTTPS transport by setting the http.sslBackend config
>variable to "openssl" or "schannel"; This is now also the method
>used by the installer (rather than copying libcurl-4.dll files
>around).

Does anyone have a pros/cons list for this option? AFAIK OpenSSL is still
the default in the GfW installer and I wonder why. My gut feeling would be to 
go with the SSL implementation shipped with the OS. However, I don't have 
enough 
knowledge of either implementation to make an assessment.

On macOS Git uses the Apple Security Framework instead of OpenSSL by default.

- Lars

Re: [git-for-windows] [ANNOUNCE] Git for Windows 2.14.0

[Johannes Sixt]

Am 06.08.2017 um 01:00 schrieb Johannes Schindelin:

Dear Git users,

It is my pleasure to announce that Git for Windows 2.14.0 is available from:

https://git-for-windows.github.io/

Changes since Git for Windows v2.13.3 (July 13th 2017)


Thank you so much! One question, though:


New Features
...
   * Comes with [BusyBox v1.28.0pre.15857.9480dca7c](https://github.com/
 git-for-windows/busybox-w32/commit/9480dca7c].


What is the implication of this addition? I guess it is not just for the 
fun of it. Does it mean that all POSIX command line tools invoked by Git 
including a POSIX shell are now routed through busybox instead of the 
MSYS2 variant?


-- Hannes