Re: difflame improvements

[Edmundo Carmona Antoranz]

On Fri, Feb 17, 2017 at 1:01 AM, Edmundo Carmona Antoranz
 wrote:
> On Thu, Feb 16, 2017 at 11:17 PM, Jeff King  wrote:
>
>> This isn't difflame's fault; that's what "git blame" tells you about
>> that line. But since I already told difflame "v2.6.5..HEAD", it would
>> probably make sense to similarly limit the blame to that range. That
>> turns up a boundary commit for the line. Which is _also_ not helpful,
>> but at least the tool is telling me that the line came from before
>> v2.6.5, and I don't really need to care much about it.
>
>
> I'm running my own tests on difflame and I have a theory about "when"
> it breaks at least one of the cases when it breaks:
>
> Analysis for deleted lines is being driven by git blame --reverse.
> What I have noticed is that it "breaks" when blame --reverse drives
> the analysis into revisions where "treeish1" is not part of their
> history (like, bringing analysis "to the sides" of treeish1 instead of
> keeping analysis in revisions in the history of treeish2 that have
> treeish1 as one of their ancestors which is definitely a valid
> case for analysis, anyway). In this case, blame --reverse stops being
> helpful.
>

At the cost of being slower, I just pushed to master the best results yet.

The workaround I developed for the case I described on the previous
mail ended up providing much better results overall so I ended up
replacing the whole merge-analysis logic with it.

Thanks for your kind help and comments, Peff. Let me know how it goes.

Re: [PATCH] git-check-ref-format: fix typo in man page

[Jeff King]

On Sun, Feb 19, 2017 at 12:20:33AM -, Philip Oakley wrote:

> >  Normalize 'refname' by removing any leading slash (`/`)
> >  characters and collapsing runs of adjacent slashes between
> > - name components into a single slash.  Iff the normalized
> > + name components into a single slash.  If the normalized
> >  refname is valid then print it to standard output and exit
> >  with a status of 0.  (`--print` is a deprecated way to spell
> >  `--normalize`.)
> > -- 
> 
> Could that be an 'iff' == 'If and only if' (which is common in mathematics)?
> Still could be spelling error though.

When we're not sure what the intent of a change is, a good first step is
to dig up the original commit via `git blame` or similar. In this case,
it comes from a40e6fb67 (Change check_refname_format() to reject
unnormalized refnames, 2011-09-15).

The commit message doesn't mention it (not that I really expected it
to), but it does tell you who the author is. And a good second step is
to cc them on the patch. :)

I suspect it _was_ intended as "iff" here. In my opinion, we probably
don't need to be so rigorous in this instance. However, I note that we
do not describe the "else" half of that "if". So maybe an overall
improvement would be something like:

  If the normalized refname is valid then print it to standard output
  and exit with a status of 0. Otherwise, exit with a non-zero status.

-Peff

[PATCH] fetch: print an error when declining to request an unadvertised object

[Matt McCutchen]

Currently "git fetch REMOTE SHA1" silently exits 1 if the server doesn't
allow requests for unadvertised objects by sha1.  The more common case
of requesting a nonexistent ref normally triggers a die() in
get_fetch_map, so "git fetch" wasn't bothering to check after the fetch
that it got all the refs it sought, like "git fetch-pack" does near the
end of cmd_fetch_pack.

Move the code from cmd_fetch_pack to a new function,
report_unmatched_refs, that is called by fetch_refs_via_pack as part of
"git fetch".  Also, change filter_refs (which checks whether a request
for an unadvertised object should be sent to the server) to set a new
match status on the "struct ref" when the request is not allowed, and
have report_unmatched_refs check for this status and print a special
error message, "Server does not allow request for unadvertised object".

Finally, add a simple test case for "git fetch REMOTE SHA1".

Signed-off-by: Matt McCutchen 
---
 builtin/fetch-pack.c  |  7 +--
 fetch-pack.c  | 51 ++-
 fetch-pack.h  |  9 +
 remote.h  |  9 +++--
 t/t5516-fetch-push.sh |  3 ++-
 transport.c   | 14 +-
 6 files changed, 66 insertions(+), 27 deletions(-)

diff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c
index cfe9e44..2a1c1c2 100644
--- a/builtin/fetch-pack.c
+++ b/builtin/fetch-pack.c
@@ -219,12 +219,7 @@ int cmd_fetch_pack(int argc, const char **argv, const char 
*prefix)
 * remote no-such-ref' would silently succeed without issuing
 * an error.
 */
-   for (i = 0; i < nr_sought; i++) {
-   if (!sought[i] || sought[i]->matched)
-   continue;
-   error("no such remote ref %s", sought[i]->name);
-   ret = 1;
-   }
+   ret |= report_unmatched_refs(sought, nr_sought);
 
while (ref) {
printf("%s %s\n",
diff --git a/fetch-pack.c b/fetch-pack.c
index 601f077..f12bfcd 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -578,7 +578,7 @@ static void filter_refs(struct fetch_pack_args *args,
break; /* definitely do not have it */
else if (cmp == 0) {
keep = 1; /* definitely have it */
-   sought[i]->matched = 1;
+   sought[i]->match_status = REF_MATCHED;
}
i++;
}
@@ -598,22 +598,24 @@ static void filter_refs(struct fetch_pack_args *args,
}
 
/* Append unmatched requests to the list */
-   if ((allow_unadvertised_object_request &
-   (ALLOW_TIP_SHA1 | ALLOW_REACHABLE_SHA1))) {
-   for (i = 0; i < nr_sought; i++) {
-   unsigned char sha1[20];
+   for (i = 0; i < nr_sought; i++) {
+   unsigned char sha1[20];
 
-   ref = sought[i];
-   if (ref->matched)
-   continue;
-   if (get_sha1_hex(ref->name, sha1) ||
-   ref->name[40] != '\0' ||
-   hashcmp(sha1, ref->old_oid.hash))
-   continue;
+   ref = sought[i];
+   if (ref->match_status != REF_NOT_MATCHED)
+   continue;
+   if (get_sha1_hex(ref->name, sha1) ||
+   ref->name[40] != '\0' ||
+   hashcmp(sha1, ref->old_oid.hash))
+   continue;
 
-   ref->matched = 1;
+   if ((allow_unadvertised_object_request &
+   (ALLOW_TIP_SHA1 | ALLOW_REACHABLE_SHA1))) {
+   ref->match_status = REF_MATCHED;
*newtail = copy_ref(ref);
newtail = &(*newtail)->next;
+   } else {
+   ref->match_status = REF_UNADVERTISED_NOT_ALLOWED;
}
}
*refs = newlist;
@@ -1094,3 +1096,26 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
clear_shallow_info();
return ref_cpy;
 }
+
+int report_unmatched_refs(struct ref **sought, int nr_sought)
+{
+   int i, ret = 0;
+
+   for (i = 0; i < nr_sought; i++) {
+   if (!sought[i])
+   continue;
+   switch (sought[i]->match_status) {
+   case REF_MATCHED:
+   continue;
+   case REF_NOT_MATCHED:
+   error(_("no such remote ref %s"), sought[i]->name);
+   break;
+   case REF_UNADVERTISED_NOT_ALLOWED:
+   error(_("Server does not allow request for unadvertised 
object %s"),
+ sought[i]->name);
+   break;
+   

Re: [PATCH] fetch: print an error when declining to request an unadvertised object

[Matt McCutchen]

On Sun, 2017-02-12 at 15:49 -0800, Junio C Hamano wrote:
> The fact that we have the above two choices tells me that a two-step
> approach may be an appropriate approach. [...]

> Even if you did only the first step, as long as the second step can
> be done without reverting what the first step did [*4*] by somebody
> who cares the "specific error" deeply enough, I am OK with that.  Of
> course if you did both steps, that is fine by me as well ;-)

I appreciate the flexibility, but now that I've spent the time to
understand all the code involved, it would be a pity not to go for the
complete solution.  New patch coming.

Matt

Re: [PATCH] fixup! bisect--helper: `bisect_next_check` & bisect_voc shell function in C

[Junio C Hamano]

Junio C Hamano  writes:

> ...
> So, let's give Pranit a concrete "here is what we want to see
> squashed in", while you guys discuss peculiarity with various
> platforms and their system headers, which admittedly is a more
> interesting tangent ;-)
>
> There are early returns with "goto finish" even before _syn
> variables are first assigned to, so they would need to be
> initialized to NULL.  The other two get their initial values
> right at the beginning, so they are OK.
>
>  builtin/bisect--helper.c | 14 +-
>  1 file changed, 5 insertions(+), 9 deletions(-)

While we are waiting for the topic to be fixed, I've tentatively
applied this on top to update 'pu', so Travis should now be happy
with 'pu' on Mac, too.

Re: [PATCH] git-check-ref-format: fix typo in man page

[Philip Oakley]

From: "Damien Regad" 

---
Documentation/git-check-ref-format.txt | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/git-check-ref-format.txt
b/Documentation/git-check-ref-format.txt
index 8611a99..377c85a 100644
--- a/Documentation/git-check-ref-format.txt
+++ b/Documentation/git-check-ref-format.txt
@@ -100,7 +100,7 @@ OPTIONS
--normalize::
 Normalize 'refname' by removing any leading slash (`/`)
 characters and collapsing runs of adjacent slashes between
- name components into a single slash.  Iff the normalized
+ name components into a single slash.  If the normalized
 refname is valid then print it to standard output and exit
 with a status of 0.  (`--print` is a deprecated way to spell
 `--normalize`.)
--


Could that be an 'iff' == 'If and only if' (which is common in mathematics)? 
Still could be spelling error though.

--
Philip 

RE: [PATCH 3/5] name-hash: precompute hash values during preload-index

[Jeff Hostetler]

From: Junio C Hamano [mailto:jch2...@gmail.com] On Behalf Of Junio C Hamano
> 
> The fact that each preload_thread() still walks the index in-order
> makes me wonder if it may allow us to further optimize the "dir"
> part of the hash by passing the previous ce for which we already
> precomputed hash values.  While the loop is iterating over the paths
> in the same directory, .dir component from the previous ce can be
> reused and .name component can "continue", no?
> 
> It's possible that you already tried such an optimization and
> rejected it after finding that the cost of comparison of pathnames
> to tell if ce and previous ce are still in the same directory is
> more than unconditionally memihash() the directory part, and I am in
> no way saying that I found a missed optimization opportunity you
> must pursue.  I am just being curious.

I looked at doing this, but I didn't think the complexity and overhead to
forward search for peers at the current level didn't warrant the limited gains.
(I was just looking at the complexity of clear_ce_flags_1() in unpack-trees.c
and how hard it has to look to find the end of the current directory and the
effect that that has on the recursion and it felt like too much work for the
potential gain.)

Whereas remembering the previous one was basically free.  Granted, it only
helps us for adjacent files in the index, so it's not perfect, but gives us the
best bang for the buck.

Jeff

Re: [PATCH] git-check-ref-format: fix typo in man page

[Jacob Keller]

On Sat, Feb 18, 2017 at 2:47 PM, Damien Regad  wrote:
>
> ---
>  Documentation/git-check-ref-format.txt | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/Documentation/git-check-ref-format.txt
> b/Documentation/git-check-ref-format.txt
> index 8611a99..377c85a 100644
> --- a/Documentation/git-check-ref-format.txt
> +++ b/Documentation/git-check-ref-format.txt
> @@ -100,7 +100,7 @@ OPTIONS
>  --normalize::
> Normalize 'refname' by removing any leading slash (`/`)
> characters and collapsing runs of adjacent slashes between
> -   name components into a single slash.  Iff the normalized
> +   name components into a single slash.  If the normalized

I think this is a good change, but I do know in some contexts, "Iff"
is used intentionally to mean "If and only if". It's somewhat unlikely
that's what was going on here, and I don't think we need to be that
pedantic in our help documentation anyway.

Thanks,
Jake

> refname is valid then print it to standard output and exit
> with a status of 0.  (`--print` is a deprecated way to spell
> `--normalize`.)
> --
> 2.7.4
>
>

RE: [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area

[Jeff Hostetler]

From: Junio C Hamano [mailto:jch2...@gmail.com] On Behalf Of Junio C Hamano
> Jeff King  writes:
>> On Wed, Feb 15, 2017 at 09:27:53AM -0500, Jeff Hostetler wrote:
>>
>>> I have some informal numbers in a spreadsheet.  I was seeing
>>> a 8-9% speed up on a status on my gigantic repo.
>>> 
>>> I'll try to put together a before/after perf-test to better
>>> demonstrate this.
>>
>> Thanks. What I'm mostly curious about is how much each individual step
>> buys. Sometimes when doing a long optimization series, I find that some
>> of the optimizations make other ones somewhat redundant (e.g., if patch
>> 2 causes us to call the optimized code from patch 3 less often).
>
> I am curious too.
>
> To me 1/5 (reduction of redundant calls), 4/5 (correctly size the
> hash that would grow to a known size anyway) and 5/5 (take advantage
> of the fact that adjacent cache entries are often in the same
> directory) look like no brainers to take, regardless of the others
> (including themselves). 

agreed.

> It is not clear to me if 3/5 (preload-index uses available cores to
> compute hashes) is an unconditional win (an operation that is
> pathspec limited may need hashes for only a small fraction of the
> index---would it still be a win to compute the hash for all entries
> upon loading the index, even if we are using otherwise-idel cores?).

I'm not sure about pathspec cases.  What I was seeing was that during
the call to lazy_name_init_hash() was taking 30% of the time in
"git status" and 40% in "git add ".  (Again this was on my
giant repo with a 450MB index).
 
> Of course 2/5 is a prerequisite step for 3/5 and 5/5, so if we want
> either of the latter two, we cannot avoid it.

jeff

RE: [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area

[Jeff Hostetler]

> Jeff King  writes:
> 
>> On Fri, Feb 17, 2017 at 09:58:21PM -0800, Junio C Hamano wrote:
>>
>>> Jeff Hostetler  writes:
>>> 
>>> > I'll try to put together a before/after perf-test to better
>>> > demonstrate this.
>>> 
>>> I didn't pick up the series while watching these exchanges, as I
>>> didn't know how quick your turnaround would be, but now a few days
>>> have passed.  Just to make sure we won't forget this topic, I'll
>>> pick up these 5 patches in the meantime.
>>
>> Yeah, to be clear my question was not an objection, but mostly
>> curiosity and interest.
>
> Yes, it was very clear that it wasn't an objection.
> 
> But the other Jeff sounded like a follow-up was to follow shortly if
> not imminent so I decided to allocate my time on other topics still
> only on the list first while waiting to see what happens.

Sorry, I was out of the office for a family emergency on Thursday
and Friday.  Add to that the long weekend, and I won't get back around
to this until Tuesday or Wednesday at the earliest.

Jeff



 

[PATCH] git-check-ref-format: fix typo in man page

[Damien Regad]

---
 Documentation/git-check-ref-format.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/git-check-ref-format.txt
b/Documentation/git-check-ref-format.txt
index 8611a99..377c85a 100644
--- a/Documentation/git-check-ref-format.txt
+++ b/Documentation/git-check-ref-format.txt
@@ -100,7 +100,7 @@ OPTIONS
 --normalize::
Normalize 'refname' by removing any leading slash (`/`)
characters and collapsing runs of adjacent slashes between
-   name components into a single slash.  Iff the normalized
+   name components into a single slash.  If the normalized
refname is valid then print it to standard output and exit
with a status of 0.  (`--print` is a deprecated way to spell
`--normalize`.)
-- 
2.7.4

Re: Git bisect does not find commit introducing the bug

[Johannes Sixt]

Am 18.02.2017 um 19:36 schrieb Alex Hoffman:

Let's consider your example with distinct names for nodes:

--o1--o2--o3--G--X1
\\
 x1--x2--x3--x4--X2--B--

It makes sense that git bisect is expecting a single transition, as
this is a precondition for a binary search to work. My definition of
"the transition" is a commit with at least one of its parents as a
good version, but the commit itself a bad version.


But that is not the definition of transition that lets you pin-point the 
breaking commit precisly. Assume x3 is the commit introducing the 
breakage in the graph above. Even though you only know initially that G 
is good and B is bad, would you not prefer to find x3 instead of X2? As 
Christian said, a transition is when a commit is bad and all its parents 
are good, and this definition lets you find x3.



I hope we agree
that git bisect's mission is to search for this transition (as I
suppose that most of people need such a functionality from git, if not
exactly from git bisect). How could be x1 or x3 be the transition, if
chances are that o1 is not a good version?


By declaring G as good, it is implied that o1 is also good. If it is in 
fact bad, the assumptions of git bisect are violated (because there 
would be a transition from bad to good at o2, o3, or G), and anything 
can happen.



If you consider that git bisect's mission is different from finding
the transition, could you please explain what exact commit git bisect
is supposed to return (ideally with terms from the graph theory) and
when it makes sense to return that? Because I do not see any sense in
looking in the path x1-x3 without knowing that those commits may be a
transition.


And I do not see a reason why git bisect should not look at those 
commits. If x3 is the commit that broke my program, I do prefer to be 
pointed at x3 rather than X2.


-- Hannes

Re: Git bisect does not find commit introducing the bug

[Hilco Wijbenga]

On 18 February 2017 at 10:36, Alex Hoffman  wrote:
> You definitely convinced me that git MUST search more than only in the
> paths between good and bad commits, as the good commit G does not have
> to be the first good commit (thank you for that). My problem/confusion
> is that it returns something that does not make sense to me, because
> it does not make sure it returns a transition.

If multiple transitions from GOOD to BAD happen, then I don't see how
binary search is useful/possible. The same is true for a simple list
of numbers, say, 1 5 6 2 3 4. You can't use binary search here because
you can't "throw away" all numbers to the left (or right) of your
pivot. Or am I missing your point?

Re: Git bisect does not find commit introducing the bug

[Philip Oakley]

From: "Alex Hoffman" 

But this is not how Git works. Git computes graph differences, i.e., it
subtracts from the commits reachable from v.bad those that are reachable
from v.good. This leaves more than just those on some path from v.good to
v.bad. And it should work this way. Consider this history:

--o--o--o--G--X
   \   \
x--x--x--x--X--B--

When you find B bad and G good, than any one of the x or X may have
introduced the breakage, not just one of the X.



Thank you for clarifying how git bisect works. How did you find that
out? Did you check the source code? If that is not documented in the
man page may be it worth documenting it in order to avoid future
confusion for other users.


Any suggestions for improving the documentation are always worthwhile. As 
someone who asked, what extra info would have helped?


Even beetter if it looks like a patch ;-)



Let's consider your example with distinct names for nodes:

--o1--o2--o3--G--X1
   \\
x1--x2--x3--x4--X1--B--

It makes sense that git bisect is expecting a single transition, as
this is a precondition for a binary search to work. My definition of
"the transition" is a commit with at least one of its parents as a
good version, but the commit itself a bad version. I hope we agree
that git bisect's mission is to search for this transition (as I
suppose that most of people need such a functionality from git, if not
exactly from git bisect). How could be x1 or x3 be the transition, if
chances are that o1 is not a good version? Of course it would make
sense to me if bisect would check o1 whether good and only then to
check also x1-x3, but this is not what git makes (at least not in my
initial example).

If you consider that git bisect's mission is different from finding
the transition, could you please explain what exact commit git bisect
is supposed to return (ideally with terms from the graph theory) and
when it makes sense to return that? Because I do not see any sense in
looking in the path x1-x3 without knowing that those commits may be a
transition.



Oh, IMO git bisect was well thought through. If it considered just paths
from good to bad, it would not given the correct answer. See the example
history above. Bisect authors would not have deemed that sufficiently 
good


You definitely convinced me that git MUST search more than only in the
paths between good and bad commits, as the good commit G does not have
to be the first good commit (thank you for that). My problem/confusion
is that it returns something that does not make sense to me, because
it does not make sure it returns a transition.

VG

PS: thank you for continuing this discussion.


--
Philip 

Re: [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area

[Junio C Hamano]

Jeff King  writes:

> On Fri, Feb 17, 2017 at 09:58:21PM -0800, Junio C Hamano wrote:
>
>> Jeff Hostetler  writes:
>> 
>> > I'll try to put together a before/after perf-test to better
>> > demonstrate this.
>> 
>> I didn't pick up the series while watching these exchanges, as I
>> didn't know how quick your turnaround would be, but now a few days
>> have passed.  Just to make sure we won't forget this topic, I'll
>> pick up these 5 patches in the meantime.
>
> Yeah, to be clear my question was not an objection, but mostly
> curiosity and interest.

Yes, it was very clear that it wasn't an objection.

But the other Jeff sounded like a follow-up was to follow shortly if
not imminent so I decided to allocate my time on other topics still
only on the list first while waiting to see what happens.

Re: [PATCH v3 02/19] builtin/diff-tree: convert to struct object_id

[Jeff King]

On Sat, Feb 18, 2017 at 07:12:18PM +, brian m. carlson wrote:

> On Fri, Feb 17, 2017 at 10:15:31PM -0500, Jeff King wrote:
> > So for this case, something like the patch below.
> > 
> > Incidentally, there's an off-by-one in the original loop of
> > stdin_diff_commit that reads past the end of the trailing NUL for the
> > final sha1 on the line. The problem is the:
> > 
> >   pos += GIT_SHA1_HEXSZ + 1;
> > 
> > which assumes we're slurping up the trailing space. This works in
> > practice because the caller will only permit a string which had a
> > newline (which it converted into a NUL).
> > 
> > I suspect that function could be more aggressive about complaining about
> > nonsense on the line, rather than silently ignoring it.
> 
> I'd come to basically the same patch, but I did pick up a few niceties
> from your patch, like avoiding the off-by-one issue you mentioned above.
> Can I place your sign-off on the resulting change?

Absolutely. Thanks for taking a look.

-Peff

Re: Git bisect does not find commit introducing the bug

[Christian Couder]

On Sat, Feb 18, 2017 at 7:36 PM, Alex Hoffman  wrote:
>> But this is not how Git works. Git computes graph differences, i.e., it
>> subtracts from the commits reachable from v.bad those that are reachable
>> from v.good. This leaves more than just those on some path from v.good to
>> v.bad. And it should work this way. Consider this history:
>>
>> --o--o--o--G--X
>>\   \
>> x--x--x--x--X--B--
>>
>> When you find B bad and G good, than any one of the x or X may have
>> introduced the breakage, not just one of the X.
>>
>
> Thank you for clarifying how git bisect works. How did you find that
> out? Did you check the source code? If that is not documented in the
> man page may be it worth documenting it in order to avoid future
> confusion for other users.

At the end of the git-bisect man page (in the SEE ALSO section) there
is a link to 
https://github.com/git/git/blob/master/Documentation/git-bisect-lk2009.txt
which has a lot of details about how bisect works.

> Let's consider your example with distinct names for nodes:
>
> --o1--o2--o3--G--X1
> \\
>  x1--x2--x3--x4--X1--B--
>
> It makes sense that git bisect is expecting a single transition, as
> this is a precondition for a binary search to work. My definition of
> "the transition" is a commit with at least one of its parents as a
> good version, but the commit itself a bad version.

What if a commit C has one good parent A and one bad parent B?
Isn't it more likely that the first bad commit is B instead of C?

> I hope we agree
> that git bisect's mission is to search for this transition (as I
> suppose that most of people need such a functionality from git, if not
> exactly from git bisect).

The goal is to find the first bad commit, which is a commit that has
only good parents.

> How could be x1 or x3 be the transition, if
> chances are that o1 is not a good version?

As o1 is an ancestor of G, then o1 is considered good by the bisect algorithm.
If it was bad, it would means that there is a transition from bad to
good between o1 and G.
But when a good commit is an ancestor of the bad commit, git bisect
makes the assumption that there is no transition from bad to good in
the graph.

> Of course it would make
> sense to me if bisect would check o1 whether good and only then to
> check also x1-x3, but this is not what git makes (at least not in my
> initial example).
>
> If you consider that git bisect's mission is different from finding
> the transition, could you please explain what exact commit git bisect
> is supposed to return (ideally with terms from the graph theory) and
> when it makes sense to return that? Because I do not see any sense in
> looking in the path x1-x3 without knowing that those commits may be a
> transition.

I hope it makes sense with the above explanations.

>> Oh, IMO git bisect was well thought through. If it considered just paths
>> from good to bad, it would not given the correct answer. See the example
>> history above. Bisect authors would not have deemed that sufficiently good
>
> You definitely convinced me that git MUST search more than only in the
> paths between good and bad commits, as the good commit G does not have
> to be the first good commit (thank you for that). My problem/confusion
> is that it returns something that does not make sense to me, because
> it does not make sure it returns a transition.

git bisect makes some assumptions that are true most of the time, so
in practice it works well most of the time.

Re: [PATCH v3 02/19] builtin/diff-tree: convert to struct object_id

[brian m. carlson]

On Fri, Feb 17, 2017 at 10:15:31PM -0500, Jeff King wrote:
> So for this case, something like the patch below.
> 
> Incidentally, there's an off-by-one in the original loop of
> stdin_diff_commit that reads past the end of the trailing NUL for the
> final sha1 on the line. The problem is the:
> 
>   pos += GIT_SHA1_HEXSZ + 1;
> 
> which assumes we're slurping up the trailing space. This works in
> practice because the caller will only permit a string which had a
> newline (which it converted into a NUL).
> 
> I suspect that function could be more aggressive about complaining about
> nonsense on the line, rather than silently ignoring it.

I'd come to basically the same patch, but I did pick up a few niceties
from your patch, like avoiding the off-by-one issue you mentioned above.
Can I place your sign-off on the resulting change?
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | https://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: https://keybase.io/bk2204


signature.asc
Description: PGP signature

Re: Git bisect does not find commit introducing the bug

[Alex Hoffman]

> But this is not how Git works. Git computes graph differences, i.e., it
> subtracts from the commits reachable from v.bad those that are reachable
> from v.good. This leaves more than just those on some path from v.good to
> v.bad. And it should work this way. Consider this history:
>
> --o--o--o--G--X
>\   \
> x--x--x--x--X--B--
>
> When you find B bad and G good, than any one of the x or X may have
> introduced the breakage, not just one of the X.
>

Thank you for clarifying how git bisect works. How did you find that
out? Did you check the source code? If that is not documented in the
man page may be it worth documenting it in order to avoid future
confusion for other users.

Let's consider your example with distinct names for nodes:

--o1--o2--o3--G--X1
\\
 x1--x2--x3--x4--X1--B--

It makes sense that git bisect is expecting a single transition, as
this is a precondition for a binary search to work. My definition of
"the transition" is a commit with at least one of its parents as a
good version, but the commit itself a bad version. I hope we agree
that git bisect's mission is to search for this transition (as I
suppose that most of people need such a functionality from git, if not
exactly from git bisect). How could be x1 or x3 be the transition, if
chances are that o1 is not a good version? Of course it would make
sense to me if bisect would check o1 whether good and only then to
check also x1-x3, but this is not what git makes (at least not in my
initial example).

If you consider that git bisect's mission is different from finding
the transition, could you please explain what exact commit git bisect
is supposed to return (ideally with terms from the graph theory) and
when it makes sense to return that? Because I do not see any sense in
looking in the path x1-x3 without knowing that those commits may be a
transition.


> Oh, IMO git bisect was well thought through. If it considered just paths
> from good to bad, it would not given the correct answer. See the example
> history above. Bisect authors would not have deemed that sufficiently good

You definitely convinced me that git MUST search more than only in the
paths between good and bad commits, as the good commit G does not have
to be the first good commit (thank you for that). My problem/confusion
is that it returns something that does not make sense to me, because
it does not make sure it returns a transition.

VG

PS: thank you for continuing this discussion.

Re: Git bisect does not find commit introducing the bug

[Johannes Sixt]

Am 18.02.2017 um 12:15 schrieb Alex Hoffman:

No one commented the fact, that I find this very confusing. Don't you
find this confusing? I will underline, that 'git bisect good v.good'
will fail if the commit 'v.good' is not a parent of the bad commit,
meaning there MUST be at least a path between 'v.good' and 'v.bad',
thus I would expect it looks on this path ONLY. Beside that, this is
what I understand by 'binary search' (to search on this commit path).


But this is not how Git works. Git computes graph differences, i.e., it 
subtracts from the commits reachable from v.bad those that are reachable 
from v.good. This leaves more than just those on some path from v.good 
to v.bad. And it should work this way. Consider this history:


--o--o--o--G--X
   \   \
x--x--x--x--X--B--

When you find B bad and G good, than any one of the x or X may have 
introduced the breakage, not just one of the X.



Correct. The assumption of bisection is that there is only one
transition between GOOD and BAD. By violating that assumption,
anything  can happen.


I did not find that in the manpage or did I miss it? Why would someone
assume that the commit graph looks in a certain way?


There is no restriction in the commit graph. The only restriction, 
actually assumption, is that there is *one* transition from good to bad 
and no transition from bad to good. Otherwise, bisection cannot work.



I assume, that
'git bisect' was not thought through and that it considers the first
directed path between v.good and v.bad, instead of all paths (in my
example graph there are two such paths). I will also underline that
git bisect was designed to work with multiple good commits and one bad
commit (also multiple paths), but probably NOT with multiple paths
between the same pair of good and bad commits.


Oh, IMO git bisect was well thought through. If it considered just paths 
from good to bad, it would not given the correct answer. See the example 
history above. Bisect authors would not have deemed that sufficiently 
good ;)


-- Hannes

[PATCH v4 15/15] refs: rename get_ref_store() to get_submodule_ref_store() and make it public

[Nguyễn Thái Ngọc Duy]

This function is intended to replace *_submodule() refs API. It provides
a ref store for a specific submodule, which can be operated on by a new
set of refs API.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs.c   | 12 
 refs.h   | 11 +++
 refs/files-backend.c |  2 +-
 refs/refs-internal.h | 12 
 4 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/refs.c b/refs.c
index 8035c3ea7..bc2247b59 100644
--- a/refs.c
+++ b/refs.c
@@ -1165,7 +1165,7 @@ int head_ref(each_ref_fn fn, void *cb_data)
 static int do_for_each_ref(const char *submodule, const char *prefix,
   each_ref_fn fn, int trim, int flags, void *cb_data)
 {
-   struct ref_store *refs = get_ref_store(submodule);
+   struct ref_store *refs = get_submodule_ref_store(submodule);
struct ref_iterator *iter;
 
if (!refs)
@@ -1338,10 +1338,10 @@ int resolve_gitlink_ref(const char *submodule, const 
char *refname,
/* We need to strip off one or more trailing slashes */
char *stripped = xmemdupz(submodule, len);
 
-   refs = get_ref_store(stripped);
+   refs = get_submodule_ref_store(stripped);
free(stripped);
} else {
-   refs = get_ref_store(submodule);
+   refs = get_submodule_ref_store(submodule);
}
 
if (!refs)
@@ -1457,13 +1457,17 @@ struct ref_store *get_main_ref_store(void)
return refs;
 }
 
-struct ref_store *get_ref_store(const char *submodule)
+struct ref_store *get_submodule_ref_store(const char *submodule)
 {
struct strbuf submodule_sb = STRBUF_INIT;
struct ref_store *refs;
int ret;
 
if (!submodule || !*submodule) {
+   /*
+* FIXME: This case is ideally not allowed. But that
+* can't happen until we clean up all the callers.
+*/
return get_main_ref_store();
}
 
diff --git a/refs.h b/refs.h
index a35b687d7..5bd065f12 100644
--- a/refs.h
+++ b/refs.h
@@ -556,5 +556,16 @@ int reflog_expire(const char *refname, const unsigned char 
*sha1,
 int ref_storage_backend_exists(const char *name);
 
 struct ref_store *get_main_ref_store(void);
+/*
+ * Return the ref_store instance for the specified submodule. For the
+ * main repository, use submodule==NULL; such a call cannot fail. For
+ * a submodule, the submodule must exist and be a nonbare repository,
+ * otherwise return NULL. If the requested reference store has not yet
+ * been initialized, initialize it first.
+ *
+ * For backwards compatibility, submodule=="" is treated the same as
+ * submodule==NULL.
+ */
+struct ref_store *get_submodule_ref_store(const char *submodule);
 
 #endif /* REFS_H */
diff --git a/refs/files-backend.c b/refs/files-backend.c
index b8ccf4e47..603601f1b 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3125,7 +3125,7 @@ int set_worktree_head_symref(const char *gitdir, const 
char *target)
 * backends. This function needs to die.
 */
struct files_ref_store *refs =
-   files_downcast(get_ref_store(NULL), "set_head_symref");
+   files_downcast(get_main_ref_store(), "set_head_symref");
static struct lock_file head_lock;
struct ref_lock *lock;
struct strbuf head_path = STRBUF_INIT;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index 3ef020acf..ed6527ce1 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -640,18 +640,6 @@ struct ref_store {
 void base_ref_store_init(struct ref_store *refs,
 const struct ref_storage_be *be);
 
-/*
- * Return the ref_store instance for the specified submodule. For the
- * main repository, use submodule==NULL; such a call cannot fail. For
- * a submodule, the submodule must exist and be a nonbare repository,
- * otherwise return NULL. If the requested reference store has not yet
- * been initialized, initialize it first.
- *
- * For backwards compatibility, submodule=="" is treated the same as
- * submodule==NULL.
- */
-struct ref_store *get_ref_store(const char *submodule);
-
 const char *resolve_ref_recursively(struct ref_store *refs,
const char *refname,
int resolve_flags,
-- 
2.11.0.157.gd943d85

[PATCH v4 13/15] refs: move submodule code out of files-backend.c

[Nguyễn Thái Ngọc Duy]

files-backend is now initialized with a $GIT_DIR. Converting a submodule
path to where real submodule gitdir is located is done in get_ref_store().

The new code in init_submodule_ref_store() is basically a copy of
strbuf_git_path_submodule().

This gives a slight performance improvement for submodules since we
don't convert submodule path to gitdir at every backend call like
before. We pay that once at ref-store creation.

More cleanup in files_downcast() follows shortly. It's separate to keep
noises from this patch.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs.c   | 18 +-
 refs/files-backend.c | 24 +++-
 refs/refs-internal.h |  6 +++---
 3 files changed, 23 insertions(+), 25 deletions(-)

diff --git a/refs.c b/refs.c
index 1b60e6180..8035c3ea7 100644
--- a/refs.c
+++ b/refs.c
@@ -9,6 +9,7 @@
 #include "refs/refs-internal.h"
 #include "object.h"
 #include "tag.h"
+#include "submodule.h"
 
 /*
  * List of all available backends
@@ -1424,9 +1425,9 @@ static void register_submodule_ref_store(struct ref_store 
*refs,
 
 /*
  * Create, record, and return a ref_store instance for the specified
- * submodule (or the main repository if submodule is NULL).
+ * gitdir (or the main repository if gitdir is NULL).
  */
-static struct ref_store *ref_store_init(const char *submodule)
+static struct ref_store *ref_store_init(const char *gitdir)
 {
const char *be_name = "files";
struct ref_storage_be *be = find_ref_storage_backend(be_name);
@@ -1435,7 +1436,7 @@ static struct ref_store *ref_store_init(const char 
*submodule)
if (!be)
die("BUG: reference backend %s is unknown", be_name);
 
-   refs = be->init(submodule);
+   refs = be->init(gitdir);
return refs;
 }
 
@@ -1460,6 +1461,7 @@ struct ref_store *get_ref_store(const char *submodule)
 {
struct strbuf submodule_sb = STRBUF_INIT;
struct ref_store *refs;
+   int ret;
 
if (!submodule || !*submodule) {
return get_main_ref_store();
@@ -1470,8 +1472,14 @@ struct ref_store *get_ref_store(const char *submodule)
return refs;
 
strbuf_addstr(_sb, submodule);
-   if (is_nonbare_repository_dir(_sb))
-   refs = ref_store_init(submodule);
+   ret = is_nonbare_repository_dir(_sb);
+   strbuf_release(_sb);
+   if (!ret)
+   return refs;
+
+   ret = submodule_to_gitdir(_sb, submodule);
+   if (!ret)
+   refs = ref_store_init(submodule_sb.buf);
strbuf_release(_sb);
 
if (refs)
diff --git a/refs/files-backend.c b/refs/files-backend.c
index dbcaf9bda..529c80af1 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -917,13 +917,6 @@ struct packed_ref_cache {
 struct files_ref_store {
struct ref_store base;
 
-   /*
-* The name of the submodule represented by this object, or
-* NULL if it represents the main repository's reference
-* store:
-*/
-   const char *submodule;
-
struct strbuf gitdir;
struct strbuf gitcommondir;
 
@@ -945,11 +938,8 @@ static void files_path(struct files_ref_store *refs, 
struct strbuf *sb,
va_start(vap, fmt);
strbuf_vaddf(, fmt, vap);
va_end(vap);
-   if (refs->submodule) {
-   strbuf_git_path_submodule(sb, refs->submodule,
- "%s", tmp.buf);
-   } else if (!strcmp(tmp.buf, "packed-refs") ||
-  !strcmp(tmp.buf, "logs")) { /* non refname path */
+   if (!strcmp(tmp.buf, "packed-refs") || !strcmp(tmp.buf, "logs")) {
+   /* non refname path */
strbuf_addf(sb, "%s/%s", refs->gitcommondir.buf, tmp.buf);
} else if (skip_prefix(tmp.buf, "logs/", )) { /* reflog */
if (is_per_worktree_ref(ref))
@@ -1018,7 +1008,7 @@ static void clear_loose_ref_cache(struct files_ref_store 
*refs)
  * Create a new submodule ref cache and add it to the internal
  * set of caches.
  */
-static struct ref_store *files_ref_store_create(const char *submodule)
+static struct ref_store *files_ref_store_create(const char *gitdir)
 {
struct files_ref_store *refs = xcalloc(1, sizeof(*refs));
struct ref_store *ref_store = (struct ref_store *)refs;
@@ -1028,8 +1018,9 @@ static struct ref_store *files_ref_store_create(const 
char *submodule)
strbuf_init(>gitdir, 0);
strbuf_init(>gitcommondir, 0);
 
-   if (submodule) {
-   refs->submodule = xstrdup(submodule);
+   if (gitdir) {
+   strbuf_addstr(>gitdir, gitdir);
+   get_common_dir_noenv(>gitcommondir, gitdir);
} else {
strbuf_addstr(>gitdir, get_git_dir());
strbuf_addstr(>gitcommondir, get_git_common_dir());
@@ -1045,8 +1036,7 @@ static struct ref_store *files_ref_store_create(const 
char *submodule)
 static void 

[PATCH v4 12/15] path.c: move some code out of strbuf_git_path_submodule()

[Nguyễn Thái Ngọc Duy]

refs is learning to avoid path rewriting that is done by
strbuf_git_path_submodule(). Factor out this code so it could be reused
by refs*

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 path.c  | 34 +++---
 submodule.c | 31 +++
 submodule.h |  1 +
 3 files changed, 39 insertions(+), 27 deletions(-)

diff --git a/path.c b/path.c
index efcedafba..3451d2916 100644
--- a/path.c
+++ b/path.c
@@ -475,35 +475,16 @@ const char *worktree_git_path(const struct worktree *wt, 
const char *fmt, ...)
 static int do_submodule_path(struct strbuf *buf, const char *path,
 const char *fmt, va_list args)
 {
-   const char *git_dir;
struct strbuf git_submodule_common_dir = STRBUF_INIT;
struct strbuf git_submodule_dir = STRBUF_INIT;
-   const struct submodule *sub;
-   int err = 0;
+   int ret;
 
-   strbuf_addstr(buf, path);
-   strbuf_complete(buf, '/');
-   strbuf_addstr(buf, ".git");
-
-   git_dir = read_gitfile(buf->buf);
-   if (git_dir) {
-   strbuf_reset(buf);
-   strbuf_addstr(buf, git_dir);
-   }
-   if (!is_git_directory(buf->buf)) {
-   gitmodules_config();
-   sub = submodule_from_path(null_sha1, path);
-   if (!sub) {
-   err = SUBMODULE_PATH_ERR_NOT_CONFIGURED;
-   goto cleanup;
-   }
-   strbuf_reset(buf);
-   strbuf_git_path(buf, "%s/%s", "modules", sub->name);
-   }
-
-   strbuf_addch(buf, '/');
-   strbuf_addbuf(_submodule_dir, buf);
+   ret = submodule_to_gitdir(_submodule_dir, path);
+   if (ret)
+   goto cleanup;
 
+   strbuf_complete(_submodule_dir, '/');
+   strbuf_addbuf(buf, _submodule_dir);
strbuf_vaddf(buf, fmt, args);
 
if (get_common_dir_noenv(_submodule_common_dir, 
git_submodule_dir.buf))
@@ -514,8 +495,7 @@ static int do_submodule_path(struct strbuf *buf, const char 
*path,
 cleanup:
strbuf_release(_submodule_dir);
strbuf_release(_submodule_common_dir);
-
-   return err;
+   return ret;
 }
 
 char *git_pathdup_submodule(const char *path, const char *fmt, ...)
diff --git a/submodule.c b/submodule.c
index 3b98766a6..36b8d1d11 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1514,3 +1514,34 @@ void absorb_git_dir_into_superproject(const char *prefix,
strbuf_release();
}
 }
+
+int submodule_to_gitdir(struct strbuf *buf, const char *submodule)
+{
+   const struct submodule *sub;
+   const char *git_dir;
+   int ret = 0;
+
+   strbuf_reset(buf);
+   strbuf_addstr(buf, submodule);
+   strbuf_complete(buf, '/');
+   strbuf_addstr(buf, ".git");
+
+   git_dir = read_gitfile(buf->buf);
+   if (git_dir) {
+   strbuf_reset(buf);
+   strbuf_addstr(buf, git_dir);
+   }
+   if (!is_git_directory(buf->buf)) {
+   gitmodules_config();
+   sub = submodule_from_path(null_sha1, submodule);
+   if (!sub) {
+   ret = -1;
+   goto cleanup;
+   }
+   strbuf_reset(buf);
+   strbuf_git_path(buf, "%s/%s", "modules", sub->name);
+   }
+
+cleanup:
+   return ret;
+}
diff --git a/submodule.h b/submodule.h
index 05ab674f0..fc3d0303e 100644
--- a/submodule.h
+++ b/submodule.h
@@ -81,6 +81,7 @@ extern int push_unpushed_submodules(struct sha1_array 
*commits,
int dry_run);
 extern void connect_work_tree_and_git_dir(const char *work_tree, const char 
*git_dir);
 extern int parallel_submodules(void);
+int submodule_to_gitdir(struct strbuf *buf, const char *submodule);
 
 /*
  * Prepare the "env_array" parameter of a "struct child_process" for executing
-- 
2.11.0.157.gd943d85

[PATCH v4 14/15] files-backend: remove submodule_allowed from files_downcast()

[Nguyễn Thái Ngọc Duy]

Since submodule or not is irrelevant to files-backend after the last
patch, remove the parameter from files_downcast() and kill
files_assert_main_repository().

PS. This implies that all ref operations are allowed for submodules. But
we may need to look more closely to see if that's really true...

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs/files-backend.c | 70 
 1 file changed, 21 insertions(+), 49 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 529c80af1..b8ccf4e47 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1030,24 +1030,13 @@ static struct ref_store *files_ref_store_create(const 
char *gitdir)
 }
 
 /*
- * Die if refs is for a submodule (i.e., not for the main repository).
- * caller is used in any necessary error messages.
- */
-static void files_assert_main_repository(struct files_ref_store *refs,
-const char *caller)
-{
-   /* This function is to be deleted in the next patch */
-}
-
-/*
  * Downcast ref_store to files_ref_store. Die if ref_store is not a
  * files_ref_store. If submodule_allowed is not true, then also die if
  * files_ref_store is for a submodule (i.e., not for the main
  * repository). caller is used in any necessary error messages.
  */
-static struct files_ref_store *files_downcast(
-   struct ref_store *ref_store, int submodule_allowed,
-   const char *caller)
+static struct files_ref_store *files_downcast(struct ref_store *ref_store,
+ const char *caller)
 {
struct files_ref_store *refs;
 
@@ -1057,9 +1046,6 @@ static struct files_ref_store *files_downcast(
 
refs = (struct files_ref_store *)ref_store;
 
-   if (!submodule_allowed)
-   files_assert_main_repository(refs, caller);
-
return refs;
 }
 
@@ -1395,7 +1381,7 @@ static int files_read_raw_ref(struct ref_store *ref_store,
  struct strbuf *referent, unsigned int *type)
 {
struct files_ref_store *refs =
-   files_downcast(ref_store, 1, "read_raw_ref");
+   files_downcast(ref_store, "read_raw_ref");
struct strbuf sb_contents = STRBUF_INIT;
struct strbuf sb_path = STRBUF_INIT;
const char *path;
@@ -1588,7 +1574,6 @@ static int lock_raw_ref(struct files_ref_store *refs,
int ret = TRANSACTION_GENERIC_ERROR;
 
assert(err);
-   files_assert_main_repository(refs, "lock_raw_ref");
 
*type = 0;
 
@@ -1812,7 +1797,7 @@ static enum peel_status peel_entry(struct ref_entry 
*entry, int repeel)
 static int files_peel_ref(struct ref_store *ref_store,
  const char *refname, unsigned char *sha1)
 {
-   struct files_ref_store *refs = files_downcast(ref_store, 0, "peel_ref");
+   struct files_ref_store *refs = files_downcast(ref_store, "peel_ref");
int flag;
unsigned char base[20];
 
@@ -1921,7 +1906,7 @@ static struct ref_iterator *files_ref_iterator_begin(
const char *prefix, unsigned int flags)
 {
struct files_ref_store *refs =
-   files_downcast(ref_store, 1, "ref_iterator_begin");
+   files_downcast(ref_store, "ref_iterator_begin");
struct ref_dir *loose_dir, *packed_dir;
struct ref_iterator *loose_iter, *packed_iter;
struct files_ref_iterator *iter;
@@ -2059,7 +2044,6 @@ static struct ref_lock *lock_ref_sha1_basic(struct 
files_ref_store *refs,
int resolve_flags = RESOLVE_REF_NO_RECURSE;
int resolved;
 
-   files_assert_main_repository(refs, "lock_ref_sha1_basic");
assert(err);
 
lock = xcalloc(1, sizeof(struct ref_lock));
@@ -2184,8 +2168,6 @@ static int lock_packed_refs(struct files_ref_store *refs, 
int flags)
struct strbuf sb = STRBUF_INIT;
int ret;
 
-   files_assert_main_repository(refs, "lock_packed_refs");
-
if (!timeout_configured) {
git_config_get_int("core.packedrefstimeout", _value);
timeout_configured = 1;
@@ -2225,8 +2207,6 @@ static int commit_packed_refs(struct files_ref_store 
*refs)
int save_errno = 0;
FILE *out;
 
-   files_assert_main_repository(refs, "commit_packed_refs");
-
if (!packed_ref_cache->lock)
die("internal error: packed-refs not locked");
 
@@ -2258,8 +2238,6 @@ static void rollback_packed_refs(struct files_ref_store 
*refs)
struct packed_ref_cache *packed_ref_cache =
get_packed_ref_cache(refs);
 
-   files_assert_main_repository(refs, "rollback_packed_refs");
-
if (!packed_ref_cache->lock)
die("internal error: packed-refs not locked");
rollback_lock_file(packed_ref_cache->lock);
@@ -2420,7 +2398,7 @@ static void prune_refs(struct files_ref_store *refs, 
struct ref_to_prune *r)
 static int 

[PATCH v4 11/15] refs.c: make get_main_ref_store() public and use it

[Nguyễn Thái Ngọc Duy]

get_ref_store() will soon be renamed to get_submodule_ref_store().
Together with future get_worktree_ref_store(), the three functions
provide an appropriate ref store for different operation modes. New APIs
will be added to operate directly on ref stores.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs.c | 36 ++--
 refs.h |  2 ++
 2 files changed, 20 insertions(+), 18 deletions(-)

diff --git a/refs.c b/refs.c
index 8b640bd05..1b60e6180 100644
--- a/refs.c
+++ b/refs.c
@@ -1308,7 +1308,7 @@ const char *resolve_ref_recursively(struct ref_store 
*refs,
 /* backend functions */
 int refs_init_db(struct strbuf *err)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->init_db(refs, err);
 }
@@ -1316,7 +1316,7 @@ int refs_init_db(struct strbuf *err)
 const char *resolve_ref_unsafe(const char *refname, int resolve_flags,
   unsigned char *sha1, int *flags)
 {
-   return resolve_ref_recursively(get_ref_store(NULL), refname,
+   return resolve_ref_recursively(get_main_ref_store(), refname,
   resolve_flags, sha1, flags);
 }
 
@@ -1439,7 +1439,7 @@ static struct ref_store *ref_store_init(const char 
*submodule)
return refs;
 }
 
-static struct ref_store *get_main_ref_store(void)
+struct ref_store *get_main_ref_store(void)
 {
struct ref_store *refs;
 
@@ -1488,14 +1488,14 @@ void base_ref_store_init(struct ref_store *refs,
 /* backend functions */
 int pack_refs(unsigned int flags)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->pack_refs(refs, flags);
 }
 
 int peel_ref(const char *refname, unsigned char *sha1)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->peel_ref(refs, refname, sha1);
 }
@@ -1503,7 +1503,7 @@ int peel_ref(const char *refname, unsigned char *sha1)
 int create_symref(const char *ref_target, const char *refs_heads_master,
  const char *logmsg)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->create_symref(refs, ref_target, refs_heads_master,
   logmsg);
@@ -1512,7 +1512,7 @@ int create_symref(const char *ref_target, const char 
*refs_heads_master,
 int ref_transaction_commit(struct ref_transaction *transaction,
   struct strbuf *err)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->transaction_commit(refs, transaction, err);
 }
@@ -1522,14 +1522,14 @@ int verify_refname_available(const char *refname,
 const struct string_list *skip,
 struct strbuf *err)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->verify_refname_available(refs, refname, extra, skip, 
err);
 }
 
 int for_each_reflog(each_ref_fn fn, void *cb_data)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
struct ref_iterator *iter;
 
iter = refs->be->reflog_iterator_begin(refs);
@@ -1540,7 +1540,7 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 int for_each_reflog_ent_reverse(const char *refname, each_reflog_ent_fn fn,
void *cb_data)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->for_each_reflog_ent_reverse(refs, refname,
 fn, cb_data);
@@ -1549,14 +1549,14 @@ int for_each_reflog_ent_reverse(const char *refname, 
each_reflog_ent_fn fn,
 int for_each_reflog_ent(const char *refname, each_reflog_ent_fn fn,
void *cb_data)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->for_each_reflog_ent(refs, refname, fn, cb_data);
 }
 
 int reflog_exists(const char *refname)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->reflog_exists(refs, refname);
 }
@@ -1564,14 +1564,14 @@ int reflog_exists(const char *refname)
 int safe_create_reflog(const char *refname, int force_create,
   struct strbuf *err)
 {
-   struct ref_store *refs = get_ref_store(NULL);
+   struct ref_store *refs = get_main_ref_store();
 
return refs->be->create_reflog(refs, refname, force_create, err);
 }
 
 int delete_reflog(const char *refname)
 {
-   struct ref_store *refs = 

[PATCH v4 05/15] refs.c: share is_per_worktree_ref() to files-backend.c

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs.c   | 6 --
 refs/refs-internal.h | 6 ++
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index 81b64b4ed..c6af84357 100644
--- a/refs.c
+++ b/refs.c
@@ -489,12 +489,6 @@ int dwim_log(const char *str, int len, unsigned char 
*sha1, char **log)
return logs_found;
 }
 
-static int is_per_worktree_ref(const char *refname)
-{
-   return !strcmp(refname, "HEAD") ||
-   starts_with(refname, "refs/bisect/");
-}
-
 static int is_pseudoref_syntax(const char *refname)
 {
const char *c;
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index f732473e1..8c5febf54 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -657,4 +657,10 @@ const char *resolve_ref_recursively(struct ref_store *refs,
int resolve_flags,
unsigned char *sha1, int *flags);
 
+static inline int is_per_worktree_ref(const char *refname)
+{
+   return !strcmp(refname, "HEAD") ||
+   starts_with(refname, "refs/bisect/");
+}
+
 #endif /* REFS_REFS_INTERNAL_H */
-- 
2.11.0.157.gd943d85

[PATCH v4 10/15] refs.c: kill register_ref_store(), add register_submodule_ref_store()

[Nguyễn Thái Ngọc Duy]

This is the last function in this code (besides public API) that takes
submodule argument and handles both main/submodule cases. Break it down,
move main store registration in get_main_ref_store() and keep the rest
in register_submodule_ref_store().

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs.c | 36 +++-
 1 file changed, 19 insertions(+), 17 deletions(-)

diff --git a/refs.c b/refs.c
index 549eeccb4..8b640bd05 100644
--- a/refs.c
+++ b/refs.c
@@ -1407,25 +1407,19 @@ static struct ref_store 
*lookup_submodule_ref_store(const char *submodule)
 
 /*
  * Register the specified ref_store to be the one that should be used
- * for submodule (or the main repository if submodule is NULL). It is
- * a fatal error to call this function twice for the same submodule.
+ * for submodule. It is a fatal error to call this function twice for
+ * the same submodule.
  */
-static void register_ref_store(struct ref_store *refs, const char *submodule)
+static void register_submodule_ref_store(struct ref_store *refs,
+const char *submodule)
 {
-   if (!submodule) {
-   if (main_ref_store)
-   die("BUG: main_ref_store initialized twice");
-
-   main_ref_store = refs;
-   } else {
-   if (!submodule_ref_stores.tablesize)
-   hashmap_init(_ref_stores, submodule_hash_cmp, 
0);
+   if (!submodule_ref_stores.tablesize)
+   hashmap_init(_ref_stores, submodule_hash_cmp, 0);
 
-   if (hashmap_put(_ref_stores,
-   alloc_submodule_hash_entry(submodule, refs)))
-   die("BUG: ref_store for submodule '%s' initialized 
twice",
-   submodule);
-   }
+   if (hashmap_put(_ref_stores,
+   alloc_submodule_hash_entry(submodule, refs)))
+   die("BUG: ref_store for submodule '%s' initialized twice",
+   submodule);
 }
 
 /*
@@ -1442,7 +1436,6 @@ static struct ref_store *ref_store_init(const char 
*submodule)
die("BUG: reference backend %s is unknown", be_name);
 
refs = be->init(submodule);
-   register_ref_store(refs, submodule);
return refs;
 }
 
@@ -1454,6 +1447,12 @@ static struct ref_store *get_main_ref_store(void)
return main_ref_store;
 
refs = ref_store_init(NULL);
+   if (refs) {
+   if (main_ref_store)
+   die("BUG: main_ref_store initialized twice");
+
+   main_ref_store = refs;
+   }
return refs;
 }
 
@@ -1474,6 +1473,9 @@ struct ref_store *get_ref_store(const char *submodule)
if (is_nonbare_repository_dir(_sb))
refs = ref_store_init(submodule);
strbuf_release(_sb);
+
+   if (refs)
+   register_submodule_ref_store(refs, submodule);
return refs;
 }
 
-- 
2.11.0.157.gd943d85

[PATCH v4 06/15] files-backend: remove the use of git_path()

[Nguyễn Thái Ngọc Duy]

Given $GIT_DIR and $GIT_COMMON_DIR, files-backend is now in charge of
deciding what goes where. The end goal is to pass $GIT_DIR only. A
refs "view" of a linked worktree is a logical ref store that combines
two files backends together.

(*) Not entirely true since strbuf_git_path_submodule() still does path
translation underneath. But that's for another patch.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs/files-backend.c | 37 +
 1 file changed, 33 insertions(+), 4 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index b599ddf92..dbcaf9bda 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -924,6 +924,9 @@ struct files_ref_store {
 */
const char *submodule;
 
+   struct strbuf gitdir;
+   struct strbuf gitcommondir;
+
struct ref_entry *loose;
struct packed_ref_cache *packed;
 };
@@ -937,15 +940,33 @@ static void files_path(struct files_ref_store *refs, 
struct strbuf *sb,
 {
struct strbuf tmp = STRBUF_INIT;
va_list vap;
+   const char *ref;
 
va_start(vap, fmt);
strbuf_vaddf(, fmt, vap);
va_end(vap);
-   if (refs->submodule)
+   if (refs->submodule) {
strbuf_git_path_submodule(sb, refs->submodule,
  "%s", tmp.buf);
-   else
-   strbuf_git_path(sb, "%s", tmp.buf);
+   } else if (!strcmp(tmp.buf, "packed-refs") ||
+  !strcmp(tmp.buf, "logs")) { /* non refname path */
+   strbuf_addf(sb, "%s/%s", refs->gitcommondir.buf, tmp.buf);
+   } else if (skip_prefix(tmp.buf, "logs/", )) { /* reflog */
+   if (is_per_worktree_ref(ref))
+   strbuf_addf(sb, "%s/%s", refs->gitdir.buf, tmp.buf);
+   else
+   strbuf_addf(sb, "%s/%s", refs->gitcommondir.buf, 
tmp.buf);
+   } else {/* refname */
+   switch (ref_type(tmp.buf)) {
+   case REF_TYPE_PER_WORKTREE:
+   case REF_TYPE_PSEUDOREF:
+   strbuf_addf(sb, "%s/%s", refs->gitdir.buf, tmp.buf);
+   break;
+   case REF_TYPE_NORMAL:
+   strbuf_addf(sb, "%s/%s", refs->gitcommondir.buf, 
tmp.buf);
+   break;
+   }
+   }
strbuf_release();
 }
 
@@ -1004,7 +1025,15 @@ static struct ref_store *files_ref_store_create(const 
char *submodule)
 
base_ref_store_init(ref_store, _be_files);
 
-   refs->submodule = xstrdup_or_null(submodule);
+   strbuf_init(>gitdir, 0);
+   strbuf_init(>gitcommondir, 0);
+
+   if (submodule) {
+   refs->submodule = xstrdup(submodule);
+   } else {
+   strbuf_addstr(>gitdir, get_git_dir());
+   strbuf_addstr(>gitcommondir, get_git_common_dir());
+   }
 
return ref_store;
 }
-- 
2.11.0.157.gd943d85

[PATCH v4 07/15] refs.c: introduce get_main_ref_store()

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs.c | 16 
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/refs.c b/refs.c
index c6af84357..c3bdc99d8 100644
--- a/refs.c
+++ b/refs.c
@@ -1450,15 +1450,23 @@ static struct ref_store *ref_store_init(const char 
*submodule)
return refs;
 }
 
+static struct ref_store *get_main_ref_store(void)
+{
+   struct ref_store *refs;
+
+   if (main_ref_store)
+   return main_ref_store;
+
+   refs = ref_store_init(NULL);
+   return refs;
+}
+
 struct ref_store *get_ref_store(const char *submodule)
 {
struct ref_store *refs;
 
if (!submodule || !*submodule) {
-   refs = lookup_ref_store(NULL);
-
-   if (!refs)
-   refs = ref_store_init(NULL);
+   return get_main_ref_store();
} else {
refs = lookup_ref_store(submodule);
 
-- 
2.11.0.157.gd943d85

[PATCH v4 08/15] refs: rename lookup_ref_store() to lookup_submodule_ref_store()

[Nguyễn Thái Ngọc Duy]

With get_main_ref_store() being used inside get_ref_store(),
lookup_ref_store() is only used for submodule code path. Rename to
reflect that and delete dead code.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs.c | 13 -
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index c3bdc99d8..6aab5e4ac 100644
--- a/refs.c
+++ b/refs.c
@@ -1389,17 +1389,13 @@ static struct ref_store *main_ref_store;
 static struct hashmap submodule_ref_stores;
 
 /*
- * Return the ref_store instance for the specified submodule (or the
- * main repository if submodule is NULL). If that ref_store hasn't
- * been initialized yet, return NULL.
+ * Return the ref_store instance for the specified submodule. If that
+ * ref_store hasn't been initialized yet, return NULL.
  */
-static struct ref_store *lookup_ref_store(const char *submodule)
+static struct ref_store *lookup_submodule_ref_store(const char *submodule)
 {
struct submodule_hash_entry *entry;
 
-   if (!submodule)
-   return main_ref_store;
-
if (!submodule_ref_stores.tablesize)
/* It's initialized on demand in register_ref_store(). */
return NULL;
@@ -1468,7 +1464,7 @@ struct ref_store *get_ref_store(const char *submodule)
if (!submodule || !*submodule) {
return get_main_ref_store();
} else {
-   refs = lookup_ref_store(submodule);
+   refs = lookup_submodule_ref_store(submodule);
 
if (!refs) {
struct strbuf submodule_sb = STRBUF_INIT;
@@ -1479,7 +1475,6 @@ struct ref_store *get_ref_store(const char *submodule)
strbuf_release(_sb);
}
}
-
return refs;
 }
 
-- 
2.11.0.157.gd943d85

[PATCH v4 09/15] refs.c: flatten get_ref_store() a bit

[Nguyễn Thái Ngọc Duy]

This helps the future changes in this code. And because get_ref_store()
is destined to become get_submodule_ref_store(), the "get main store"
code path will be removed eventually. After this the patch to delete
that code will be cleaner.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs.c | 19 +--
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/refs.c b/refs.c
index 6aab5e4ac..549eeccb4 100644
--- a/refs.c
+++ b/refs.c
@@ -1459,22 +1459,21 @@ static struct ref_store *get_main_ref_store(void)
 
 struct ref_store *get_ref_store(const char *submodule)
 {
+   struct strbuf submodule_sb = STRBUF_INIT;
struct ref_store *refs;
 
if (!submodule || !*submodule) {
return get_main_ref_store();
-   } else {
-   refs = lookup_submodule_ref_store(submodule);
+   }
 
-   if (!refs) {
-   struct strbuf submodule_sb = STRBUF_INIT;
+   refs = lookup_submodule_ref_store(submodule);
+   if (refs)
+   return refs;
 
-   strbuf_addstr(_sb, submodule);
-   if (is_nonbare_repository_dir(_sb))
-   refs = ref_store_init(submodule);
-   strbuf_release(_sb);
-   }
-   }
+   strbuf_addstr(_sb, submodule);
+   if (is_nonbare_repository_dir(_sb))
+   refs = ref_store_init(submodule);
+   strbuf_release(_sb);
return refs;
 }
 
-- 
2.11.0.157.gd943d85

[PATCH v4 04/15] files-backend: replace *git_path*() with files_path()

[Nguyễn Thái Ngọc Duy]

This centralizes all path rewriting of files-backend.c in one place so
we have easier time removing the path rewriting later. There could be
some hidden indirect git_path() though, I didn't audit the code to the
bottom.

Side note: set_worktree_head_symref() is a bad boy and should not be in
files-backend.c (probably should not exist in the first place). But
we'll leave it there until we have better multi-worktree support in refs
before we update it.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs/files-backend.c | 208 +++
 1 file changed, 111 insertions(+), 97 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 7eaf28bd6..b599ddf92 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -165,7 +165,8 @@ static struct ref_entry *create_dir_entry(struct 
files_ref_store *ref_store,
  const char *dirname, size_t len,
  int incomplete);
 static void add_entry_to_dir(struct ref_dir *dir, struct ref_entry *entry);
-static int files_log_ref_write(const char *refname, const unsigned char 
*old_sha1,
+static int files_log_ref_write(struct files_ref_store *refs,
+  const char *refname, const unsigned char 
*old_sha1,
   const unsigned char *new_sha1, const char *msg,
   int flags, struct strbuf *err);
 
@@ -1178,12 +1179,10 @@ static void read_packed_refs(FILE *f, struct ref_dir 
*dir)
 static struct packed_ref_cache *get_packed_ref_cache(struct files_ref_store 
*refs)
 {
char *packed_refs_file;
+   struct strbuf sb = STRBUF_INIT;
 
-   if (refs->submodule)
-   packed_refs_file = git_pathdup_submodule(refs->submodule,
-"packed-refs");
-   else
-   packed_refs_file = git_pathdup("packed-refs");
+   files_path(refs, , "packed-refs");
+   packed_refs_file = strbuf_detach(, NULL);
 
if (refs->packed &&
!stat_validity_check(>packed->validity, packed_refs_file))
@@ -1249,10 +1248,7 @@ static void read_loose_refs(const char *dirname, struct 
ref_dir *dir)
size_t path_baselen;
int err = 0;
 
-   if (refs->submodule)
-   err = strbuf_git_path_submodule(, refs->submodule, "%s", 
dirname);
-   else
-   strbuf_git_path(, "%s", dirname);
+   files_path(refs, , "%s", dirname);
path_baselen = path.len;
 
if (err) {
@@ -1394,10 +1390,7 @@ static int files_read_raw_ref(struct ref_store 
*ref_store,
*type = 0;
strbuf_reset(_path);
 
-   if (refs->submodule)
-   strbuf_git_path_submodule(_path, refs->submodule, "%s", 
refname);
-   else
-   strbuf_git_path(_path, "%s", refname);
+   files_path(refs, _path, "%s", refname);
 
path = sb_path.buf;
 
@@ -1585,7 +1578,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
*lock_p = lock = xcalloc(1, sizeof(*lock));
 
lock->ref_name = xstrdup(refname);
-   strbuf_git_path(_file, "%s", refname);
+   files_path(refs, _file, "%s", refname);
 
 retry:
switch (safe_create_leading_directories(ref_file.buf)) {
@@ -2057,7 +2050,7 @@ static struct ref_lock *lock_ref_sha1_basic(struct 
files_ref_store *refs,
if (flags & REF_DELETING)
resolve_flags |= RESOLVE_REF_ALLOW_BAD_NAME;
 
-   strbuf_git_path(_file, "%s", refname);
+   files_path(refs, _file, "%s", refname);
resolved = !!resolve_ref_unsafe(refname, resolve_flags,
lock->old_oid.hash, type);
if (!resolved && errno == EISDIR) {
@@ -2179,7 +2172,7 @@ static int lock_packed_refs(struct files_ref_store *refs, 
int flags)
timeout_configured = 1;
}
 
-   strbuf_git_path(, "packed-refs");
+   files_path(refs, , "packed-refs");
ret = hold_lock_file_for_update_timeout(, sb.buf,
flags, timeout_value);
strbuf_release();
@@ -2332,7 +2325,9 @@ enum {
  * subdirs. flags is a combination of REMOVE_EMPTY_PARENTS_REF and/or
  * REMOVE_EMPTY_PARENTS_REFLOG.
  */
-static void try_remove_empty_parents(const char *refname, unsigned int flags)
+static void try_remove_empty_parents(struct files_ref_store *refs,
+const char *refname,
+unsigned int flags)
 {
struct strbuf buf = STRBUF_INIT;
struct strbuf sb = STRBUF_INIT;
@@ -2359,12 +2354,12 @@ static void try_remove_empty_parents(const char 
*refname, unsigned int flags)
strbuf_setlen(, q - buf.buf);
 
strbuf_reset();
-   strbuf_git_path(, "%s", buf.buf);
+   files_path(refs, , "%s", buf.buf);
if ((flags & REMOVE_EMPTY_PARENTS_REF) && 

[PATCH v4 03/15] files-backend: add files_path()

[Nguyễn Thái Ngọc Duy]

This will be the replacement for both git_path() and git_path_submodule()
in this file. The idea is backend takes a git path and use that,
oblivious of submodule, linked worktrees and such.

This is the middle step towards that. Eventually the "submodule" field
in 'struct files_ref_store' should be replaced by "gitdir". And a
compound ref_store is created to combine two files backends together,
one represents the shared refs in $GIT_COMMON_DIR, one per-worktree. At
that point, files_path() becomes a wrapper of strbuf_vaddf().

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs/files-backend.c | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index c6c86f850..7eaf28bd6 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -930,6 +930,24 @@ struct files_ref_store {
 /* Lock used for the main packed-refs file: */
 static struct lock_file packlock;
 
+__attribute__((format (printf, 3, 4)))
+static void files_path(struct files_ref_store *refs, struct strbuf *sb,
+  const char *fmt, ...)
+{
+   struct strbuf tmp = STRBUF_INIT;
+   va_list vap;
+
+   va_start(vap, fmt);
+   strbuf_vaddf(, fmt, vap);
+   va_end(vap);
+   if (refs->submodule)
+   strbuf_git_path_submodule(sb, refs->submodule,
+ "%s", tmp.buf);
+   else
+   strbuf_git_path(sb, "%s", tmp.buf);
+   strbuf_release();
+}
+
 /*
  * Increment the reference count of *packed_refs.
  */
-- 
2.11.0.157.gd943d85

[PATCH v4 02/15] files-backend: convert git_path() to strbuf_git_path()

[Nguyễn Thái Ngọc Duy]

git_path() and friends are going to be killed in files-backend.c in near
future. And because there's a risk with overwriting buffer in
git_path(), let's convert them all to strbuf_git_path(). We'll have
easier time killing/converting strbuf_git_path() then because we won't
have to worry about memory management again.

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs/files-backend.c | 146 ---
 1 file changed, 115 insertions(+), 31 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 1ebd59ec0..c6c86f850 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -2151,6 +2151,8 @@ static int lock_packed_refs(struct files_ref_store *refs, 
int flags)
static int timeout_configured = 0;
static int timeout_value = 1000;
struct packed_ref_cache *packed_ref_cache;
+   struct strbuf sb = STRBUF_INIT;
+   int ret;
 
files_assert_main_repository(refs, "lock_packed_refs");
 
@@ -2159,10 +2161,13 @@ static int lock_packed_refs(struct files_ref_store 
*refs, int flags)
timeout_configured = 1;
}
 
-   if (hold_lock_file_for_update_timeout(
-   , git_path("packed-refs"),
-   flags, timeout_value) < 0)
+   strbuf_git_path(, "packed-refs");
+   ret = hold_lock_file_for_update_timeout(, sb.buf,
+   flags, timeout_value);
+   strbuf_release();
+   if (ret < 0)
return -1;
+
/*
 * Get the current packed-refs while holding the lock.  If the
 * packed-refs file has been modified since we last read it,
@@ -2312,6 +2317,7 @@ enum {
 static void try_remove_empty_parents(const char *refname, unsigned int flags)
 {
struct strbuf buf = STRBUF_INIT;
+   struct strbuf sb = STRBUF_INIT;
char *p, *q;
int i;
 
@@ -2333,14 +2339,19 @@ static void try_remove_empty_parents(const char 
*refname, unsigned int flags)
if (q == p)
break;
strbuf_setlen(, q - buf.buf);
-   if ((flags & REMOVE_EMPTY_PARENTS_REF) &&
-   rmdir(git_path("%s", buf.buf)))
+
+   strbuf_reset();
+   strbuf_git_path(, "%s", buf.buf);
+   if ((flags & REMOVE_EMPTY_PARENTS_REF) && rmdir(sb.buf))
flags &= ~REMOVE_EMPTY_PARENTS_REF;
-   if ((flags & REMOVE_EMPTY_PARENTS_REFLOG) &&
-   rmdir(git_path("logs/%s", buf.buf)))
+
+   strbuf_reset();
+   strbuf_git_path(, "logs/%s", buf.buf);
+   if ((flags & REMOVE_EMPTY_PARENTS_REFLOG) && rmdir(sb.buf))
flags &= ~REMOVE_EMPTY_PARENTS_REFLOG;
}
strbuf_release();
+   strbuf_release();
 }
 
 /* make sure nobody touched the ref, and unlink */
@@ -2426,7 +2437,11 @@ static int repack_without_refs(struct files_ref_store 
*refs,
return 0; /* no refname exists in packed refs */
 
if (lock_packed_refs(refs, 0)) {
-   unable_to_lock_message(git_path("packed-refs"), errno, err);
+   struct strbuf sb = STRBUF_INIT;
+
+   strbuf_git_path(, "packed-refs");
+   unable_to_lock_message(sb.buf, errno, err);
+   strbuf_release();
return -1;
}
packed = get_packed_refs(refs);
@@ -2504,9 +2519,11 @@ static int files_delete_refs(struct ref_store *ref_store,
 
 static int rename_tmp_log_callback(const char *path, void *cb)
 {
+   struct strbuf sb = STRBUF_INIT;
int *true_errno = cb;
 
-   if (rename(git_path(TMP_RENAMED_LOG), path)) {
+   strbuf_git_path(, TMP_RENAMED_LOG);
+   if (rename(sb.buf, path)) {
/*
 * rename(a, b) when b is an existing directory ought
 * to result in ISDIR, but Solaris 5.8 gives ENOTDIR.
@@ -2515,30 +2532,40 @@ static int rename_tmp_log_callback(const char *path, 
void *cb)
 * that it knows to retry.
 */
*true_errno = errno;
+   strbuf_release();
if (errno == ENOTDIR)
errno = EISDIR;
return -1;
} else {
+   strbuf_release();
return 0;
}
 }
 
 static int rename_tmp_log(const char *newrefname)
 {
-   char *path = git_pathdup("logs/%s", newrefname);
+   struct strbuf sb = STRBUF_INIT;
+   const char *path;
int ret, true_errno;
 
+   strbuf_git_path(, "logs/%s", newrefname);
+   path = sb.buf;
ret = raceproof_create_file(path, rename_tmp_log_callback, _errno);
if (ret) {
if (errno == EISDIR)
error("directory not empty: %s", path);
-   else
+   else {
+   struct strbuf tmp_renamed = 

[PATCH v4 00/15] Remove submodule from files-backend.c

[Nguyễn Thái Ngọc Duy]

v4:

 - is based on master+mh/ref-remove-empty-directory+mh/submodule-hash
   instead of mh/submodule-hash alone. It could merge with pu cleanly.

 - 06/16 from v5, "correct is_per_worktree_ref", is removed. The
   "remove the use of git_path" patch is updated to handle it
   correctly even without that patch. More on this later.

 - update "replace" typo in a commit message

The story behind the 06/16 removal is, in my simple view, there are
per-repo and per-worktree refs. But refs.c and backends see them as
three types: per-worktree, pseudo and normal/per-repo. Refs like
ORIG_HEAD are _both_ per-worktree and pseudo.

06/16 reclassifies ORIG_HEAD from pseudo to per-worktree, which
triggers a failure in t1400 because as pseudo refs, reflog will not be
created while it will be for per-worktree. This new test is on master,
not maint. I now fall back to treating both per-worktree and pseudo as
per-worktree in files_path() only and keep classification unchanged.

Side note, my other two series seem to apply cleanly on top of this
v3, so no resend.

Nguyễn Thái Ngọc Duy (15):
  refs-internal.c: make files_log_ref_write() static
  files-backend: convert git_path() to strbuf_git_path()
  files-backend: add files_path()
  files-backend: replace *git_path*() with files_path()
  refs.c: share is_per_worktree_ref() to files-backend.c
  files-backend: remove the use of git_path()
  refs.c: introduce get_main_ref_store()
  refs: rename lookup_ref_store() to lookup_submodule_ref_store()
  refs.c: flatten get_ref_store() a bit
  refs.c: kill register_ref_store(), add register_submodule_ref_store()
  refs.c: make get_main_ref_store() public and use it
  path.c: move some code out of strbuf_git_path_submodule()
  refs: move submodule code out of files-backend.c
  files-backend: remove submodule_allowed from files_downcast()
  refs: rename get_ref_store() to get_submodule_ref_store() and make it public

 path.c   |  34 +
 refs.c   | 144 +-
 refs.h   |  13 ++
 refs/files-backend.c | 406 ---
 refs/refs-internal.h |  28 ++--
 submodule.c  |  31 
 submodule.h  |   1 +
 7 files changed, 396 insertions(+), 261 deletions(-)

-- 
2.11.0.157.gd943d85

[PATCH v4 01/15] refs-internal.c: make files_log_ref_write() static

[Nguyễn Thái Ngọc Duy]

Created in 5f3c3a4e6f (files_log_ref_write: new function - 2015-11-10)
but probably never used outside refs-internal.c

Signed-off-by: Nguyễn Thái Ngọc Duy 
---
 refs/files-backend.c | 3 +++
 refs/refs-internal.h | 4 
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index db3bd42a9..1ebd59ec0 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -165,6 +165,9 @@ static struct ref_entry *create_dir_entry(struct 
files_ref_store *ref_store,
  const char *dirname, size_t len,
  int incomplete);
 static void add_entry_to_dir(struct ref_dir *dir, struct ref_entry *entry);
+static int files_log_ref_write(const char *refname, const unsigned char 
*old_sha1,
+  const unsigned char *new_sha1, const char *msg,
+  int flags, struct strbuf *err);
 
 static struct ref_dir *get_ref_dir(struct ref_entry *entry)
 {
diff --git a/refs/refs-internal.h b/refs/refs-internal.h
index fa93c9a32..f732473e1 100644
--- a/refs/refs-internal.h
+++ b/refs/refs-internal.h
@@ -228,10 +228,6 @@ struct ref_transaction {
enum ref_transaction_state state;
 };
 
-int files_log_ref_write(const char *refname, const unsigned char *old_sha1,
-   const unsigned char *new_sha1, const char *msg,
-   int flags, struct strbuf *err);
-
 /*
  * Check for entries in extras that are within the specified
  * directory, where dirname is a reference directory name including
-- 
2.11.0.157.gd943d85

Re: [PATCH v2 00/16] Remove submodule from files-backend.c

[Duy Nguyen]

On Sat, Feb 18, 2017 at 1:35 AM, Junio C Hamano  wrote:
> Nguyễn Thái Ngọc Duy   writes:
>
>> I'll be sending two more follow-up series, if you are interested, soon:
>>
>> 1) v2 of nd/worktree-gc-protection
>>
>> which kills parse_ref() in worktree.c _and_ set_worktree_head_symref()
>> in files-backend.c. Both are bad things that should not have happened.
>> (PS. The topic name is misleading as this is mostly about eliminating
>> warts, unless Junio intended to combine my second series as well)
>
> Your description sounded that these two are just preparatory step
> for the main one that would soon follow, and that was why these two
> patches landed on a topic named as such without any of its friends
> (which were yet to come).  If you prefer to keep these a separate
> preparatory step from the remainder and have them graduate sooner,
> let's do so, as that is my preference as well.
>
> Rename it "nd/worktree-kill-parse-ref" perhaps?

That's exactly my branch name (minus the nd/)
-- 
Duy

Re: [git-for-windows] Re: Continuous Testing of Git on Windows

[Philip Oakley]

From: "Junio C Hamano" 
Sent: Thursday, February 16, 2017 12:20 AM

"Philip Oakley"  writes:


It may even be worth 'splitting' the pu branch sequence into the
existing pu (with merges from series that are selected as reasonable),
and then a pr branch (public review?) on top of that holding the rest
of the series that have been submitted, so that the CI can do a full
test on the tips of them to support those devs with limited test
capability.


I won't stop you from publishing such a pr branch yourself.


But would others see it?... (rhetorical, better thoughts below))



For patches whose merit is not clear because the problem they try to
solve is under-explained, whose solution is ill-designed, etc., IOW,
with issues that makes me judge that they are not interesting enough
for 'pu',


It is reasonble that that a project's integrator is able to make these 
decisions. For some projects they may have a layered approach of descisions 
which does allow the gradations for the submitted feature series.


Some of this does fall into dscho's differentiation between those patch 
series that should pass the CI (continuous integration) testing, and those 
that are there for CT (continuous testing) feedback. This could either be an 
extra branch marking the transition, or a named commit similar to the 
"pu^{/^### match next}", etc. In some ways it is similar to my 'pr' 
suggestion, without the inclusion of the 'all and sundry' series.


For for integrators who are willing/want to recieve any/all contributions 
for public view (usually those for projects of a more lenient and less 
critical variety), then even the CT grouping could then have those 
additional pr submissions. For Git, you provide that that 'voice of reason' 
for gatekeeping the pu branch.




it is not worth my time to deal with whitespace brekages
in them to make them not even apply, to figure out what base the
patches are meant to apply to, or to resolve conflicts caused by
them with topics already in flight, etc.

If a centralised CT service was available to the project, maybe via the 
GitHub PR process (which isn't curently used by the project) then is may be 
a way of allowing the 'all and sundry' contributors to get their ideas upto 
a basic level before even bothering yourself (because PRs do not trouble 
you).


It may need an extra gatekeeper between the passing patches (PRs) and auto 
submission  (the Heroku script thingy) which could flood the list with with 
inane changes - one only has to look at the 
http://stackoverflow.com/questions/tagged/git stream to see that.


At least if there was a break point within pu that allowed differentiation 
between the series that should fit a CI view, and those that are still at 
the CT stage, then that may help.


Thanks

Philip.

Re: Git bisect does not find commit introducing the bug

[Alex Hoffman]

>> First of all this is confusing, as this commit cannot be reached
>> starting from "v.good".

> Hm, IMHO it shows that your example is pretty artificial (although you
> might have come across it in a real-world scenario): you introduced a
> new feature in f4154e9 (and it worked) and you broke that feature by
> making the merge 671cec2. However, the feature (that broke in 671cec2)
> did not even exist in 04c6f4b; so a test on the feature would not fail
> (leading to "bisect bad" as in the example), it would not exist (leading
> to "bisect skip").

No one commented the fact, that I find this very confusing. Don't you
find this confusing? I will underline, that 'git bisect good v.good'
will fail if the commit 'v.good' is not a parent of the bad commit,
meaning there MUST be at least a path between 'v.good' and 'v.bad',
thus I would expect it looks on this path ONLY. Beside that, this is
what I understand by 'binary search' (to search on this commit path).
You might find this example artificial, but I doubt git is/was
intentionally designed to work with  'natural' examples only (no
matter how you define 'natural' and 'artificial').



>> In other words: bisect assumes that your repo is usually in a good state
>> and you have a commit that changes it to a bad state. In your case you
>> have a repo that is in a bad state and you have a commit that switches
>> it to a good state and later you merge a bad-state branch and you have a
>> bad state again. It is not made for that use-case, I think.

> Correct. The assumption of bisection is that there is only one transition 
> between GOOD and BAD. By violating that assumption, anything can happen.

I did not find that in the manpage or did I miss it? Why would someone
assume that the commit graph looks in a certain way? I assume, that
'git bisect' was not thought through and that it considers the first
directed path between v.good and v.bad, instead of all paths (in my
example graph there are two such paths). I will also underline that
git bisect was designed to work with multiple good commits and one bad
commit (also multiple paths), but probably NOT with multiple paths
between the same pair of good and bad commits.

VG


2017-02-18 10:12 GMT+01:00 Johannes Sixt :
> Am 18.02.2017 um 00:21 schrieb Stephan Beyer:
>>
>> On 02/17/2017 11:29 PM, Alex Hoffman wrote:
>> *   7a9e952 (bisect bad) 
>> |\
>> | *   671cec2  <--- expected
>> | |\
>> | * | 04c6f4b  <--- found
>> * | |   3915157 
>> |\ \ \
>> | | |/
>> | |/|
>> | * | f4154e9 (bisect good) 
>> | * | 85855bf 
>> | |/
>> * | f1a36f5 
>> |/
>> * 1b7fb88 
>>
>> The  and  markers are set by your definition of what good and
>> what bad commits are.
>>
>> [...]
>> In other words: bisect assumes that your repo is usually in a good state
>> and you have a commit that changes it to a bad state. In your case you
>> have a repo that is in a bad state and you have a commit that switches
>> it to a good state and later you merge a bad-state branch and you have a
>> bad state again. It is not made for that use-case, I think.
>
>
> Correct. The assumption of bisection is that there is only one transition
> between GOOD and BAD. By violating that assumption, anything can happen.
>
> -- Hannes
>

Please I want you to patiently read this offer.?

[Mr.Hassan Habib]

Hello.

I know this means of communication may not be morally right to you as a person 
but I also have had a great thought about it and I have come to this conclusion 
which I am about to share with you.

INTRODUCTION:I am the Credit Manager U. B. A Bank of Burkina Faso Ouagadougou
and in one way or the other was hoping you will cooperate with me as a partner 
in a project of transferring an abandoned fund of a late customer of the bank 
worth of $18,000,000 (Eighteen Million Dollars US).

This will be disbursed or shared between the both of us in these percentages, 
55% to me and 35% to you while 10% will be for expenses both parties might have 
incurred during the process of transferring.
I
await for your response so that we can commence on this project as soon as 
possible.

Reply to this Email:mr_habib2...@yahoo.com

Regards,
Mr.Hassan Habib.

Credit Manager U.B.A Bank of
Burkina Faso Ouagadougou

Re: Git bisect does not find commit introducing the bug

[Johannes Sixt]

Am 18.02.2017 um 00:21 schrieb Stephan Beyer:

On 02/17/2017 11:29 PM, Alex Hoffman wrote:
*   7a9e952 (bisect bad) 
|\
| *   671cec2  <--- expected
| |\
| * | 04c6f4b  <--- found
* | |   3915157 
|\ \ \
| | |/
| |/|
| * | f4154e9 (bisect good) 
| * | 85855bf 
| |/
* | f1a36f5 
|/
* 1b7fb88 

The  and  markers are set by your definition of what good and
what bad commits are.

[...]
In other words: bisect assumes that your repo is usually in a good state
and you have a commit that changes it to a bad state. In your case you
have a repo that is in a bad state and you have a commit that switches
it to a good state and later you merge a bad-state branch and you have a
bad state again. It is not made for that use-case, I think.


Correct. The assumption of bisection is that there is only one 
transition between GOOD and BAD. By violating that assumption, anything 
can happen.


-- Hannes