Re: Retrieving a file in git that was deleted and committed

[Bryan Turner]

On Thu, Dec 6, 2018 at 11:26 PM Jeff King  wrote:
>
> On Thu, Dec 06, 2018 at 11:07:00PM -0800, biswaranjan panda wrote:
>
> > Thanks! Strangely git log --follow does work.
>
> I suspect it would work even without --follow. When you limit a log
> traversal with a pathspec, like:
>
>   git log foo
>
> that is not about following some continuous stream of content, but
> rather just applying that pathspec to the diff of each commit, and
> pruning ones where it did not change. So even if there are gaps where
> the file did not exist, we continue to apply the pathspec to the older
> commits.

Ah, of course. Thanks for the clarification, Jeff. And my apologies to
Biswaranjan Panda for the incorrect information.

>
> Tools like git-blame will _not_ work, though, as they really are trying
> to track the content as they walk back through history. And Once all of
> the content seems to appear from nowhere in your new commit, that seems
> like a dead end.
>
> In theory there could be some machine-readable annotation in the commit
> object (or in a note created after the fact) to say "even though 'foo'
> is a new file here, it came from $commit:foo".  And then git-blame could
> keep following the content there. But such a feature does not yet exist.
>
> -Peff

Re: Retrieving a file in git that was deleted and committed

[Bryan Turner]

On Thu, Dec 6, 2018 at 10:49 PM biswaranjan panda
 wrote:
>
> I have the following scenario:
>
> On a branch A, I deleted a file foo.txt and committed the change. Then
> I did a bunch of other changes.
> Now I want to undelete foo.txt.
>
> One way is to checkout a separate branch B where the file is present.
> Then checkout A. Then do
> git checkout B -- path_to_file

It doesn't change anything, but note that you don't need to checkout B
first, to restore the file. If you know a commit SHA where the file is
present, "git checkout SHA -- path_to_file" will pull back the file as
it existed at that commit.

>
> While this does gets the file back, the file shows up as a new file to
> be committed. Once I commit it, git blame doesn't show the old history
> for the file.
>
> I would appreciate if anyone knows how to preserve git blame history

It's not possible, as far as I'm aware. While the new file has the
same name as the old file, to Git they are two unrelated entries that
happen to reside at the same path. Even things like "git log --follow"
won't consider the file to be related to its previous history.

Bryan

Re: Parsing a git HTTP protocol response

[Bryan Turner]

On Fri, Nov 30, 2018 at 6:58 PM Bryan Turner  wrote:
>
> Here's a (very ugly) patch I threw together on top of your code:
...snip

Gmail butchered my patch, so here it is as an attachment.

Bryan


short-size-reads.patch
Description: Binary data

Re: Parsing a git HTTP protocol response

[Bryan Turner]

On Fri, Nov 30, 2018 at 5:05 PM Farhan Khan  wrote:
>
> Hi all,
>
> I am writing an implementation of the git HTTP pack protocol in C. It
> just does a request to clone a repository. It works pretty well for
> small repositories, but seems to fail on larger repositories and I do
> not understand why.
>
> All that my code does is send a hard-coded "want" request. As I
> understand it, responses come with a four-byte size prefix, then the
> data of the size - 4. The first four bytes are the size of the object
> in ASCII and after that size, a new object should begin by specifying
> its size. The final "" string should signify the end.
>
> On small repositories my code works fine. But when use a large git
> repository, I hit an object size that cannot be interpreted by ASCII
> into a number. In particular, right before the crash I am pretty
> consistently getting a size starting with 0x32,0x00 or 0x32,0x30,0x00
> (note, it starts with 0x32 and has 0x00 in there). This is not a
> readable four byte ascii value, likely an erroneous size value, which
> causes the next object size calculation to land incorrectly and
> subsequently the program to malfunction.
>
> My questions:
> 1. Am I misunderstanding the protocol?
> 2. Does 0x32 signify something?
> 3. Also, where can I see in the source code git parses the data it
> receives from a "want" request?
>
> If you would like to see my code, it is located here:
> http://git.farhan.codes/farhan/post
> To compile to Linux run: gcc post.c main.c -lcurl -o post
> To compile on FreeBSD you can use the Makefile or: cc -L
> /usr/local/lib -I /usr/local/include -lcurl main.c post.c -o post
> In both cases you need to have libcurl installed.
>
> To run do: ./post [a git http url] [a commit value]
> ie: ./post http://git.farhan.codes/farhan/openbsd
> 373dd5ba116d42cddf1fdcb58b578a4274f6d589
>
> I have a lot of debugging printf() messages, but it eventually hits a
> point where you see this:
>
> Start=
> Current stats: Current state: 999 Received: 1448
> We have enough bytes, moving forward
> == New Object
> No Error, object is 32 bytes
> Size bytes: 32 30 00 00

I modified your debugging output a little bit, and right before the
failure happens I see this in my output:

Start=
Current stats: Current state: 999 Received: 1298
Read complete; still missing 1296 bytes (6901 read of 8197)
Offset: 1298

Start=
Current stats: Current state: 999 Received: 1298
Read complete; 2 more bytes in buffer
== New Object
Size bytes: 32 30 00 2b

Based on that, it appears what's happening is you're not handling the
case where you end up with fewer than 4 bytes in the buffer. As a
result, memcpy reads past the end of the response buffer and gets
whatever it gets, resulting in an incorrect parsed size.

The while loop in pack_protocol_line is checking +1, but I think it
should be checking +3 to ensure there's at least 4 bytes so it can get
a complete size. The "parseread" struct will need to grow a couple
more fields to allow buffering some additional bytes between
pack_protocol_line calls (because curl requires the write function to
always consume the full buffer) and, at the start of the loop, when
reading/parsing a size, the code will need to consider any buffered
bytes from the previous function call. That requires some knock-on
changes to how the offset is handled, as well as the the initial
"psize" set when starting a new packet.

Once you start accounting for reads that cut off in 1, 2 or 3 bytes
into the 4 required to parse a size, your program should be able to
run to completion.

Here's a (very ugly) patch I threw together on top of your code:

diff --git a/main.c b/main.c
index 956362b..8873fd5 100644
--- a/main.c
+++ b/main.c
@@ -71,7 +71,7 @@ int main(int argc, char *argv[]) {
  curl_easy_setopt(curl, CURLOPT_HTTPHEADER, list);
  curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long)content_length);
  curl_easy_setopt(curl, CURLOPT_WRITEDATA, );
- curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, pack_protocol_line);
+ curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, _protocol_line);

  res = curl_easy_perform(curl);
  if (curl != CURLE_OK)
diff --git a/post.c b/post.c
index cfffd5c..4f8512c 100644
--- a/post.c
+++ b/post.c
@@ -36,89 +36,83 @@ size_t pack_protocol_line(void *buffer, size_t
size, size_t nmemb, void *userp)
 {
  unsigned char *reply = buffer;
  struct parseread *parseread = userp;
- int offset = 0;
+ int offset = 0, pending = 0, remaining = 0;
  char tmp[5];
- int check;
-
- int pool = size * nmemb;

  printf("\n\nStart=\n");
- printf("Current stats: Current state: %d Received: %ld\n",
parseread->state, size*nmemb);
+ printf("Current stats: Current state: %d Received: %ld\n",
parseread->state, nmemb);

- while(offset + 1 < size + nmemb) {
+ while (offset + 3 < nmemb) {
  if (parseread->state == STATE_NEWLINE) {
  printf("== New Object\n");
  bzero(tmp, 5);
- memcpy(tmp, buffer+offset, 4); tmp[4] = '\0';
- check 

Re: Forcing GC to always fail

[Bryan Turner]

On Wed, Nov 28, 2018 at 5:19 PM Junio C Hamano  wrote:
>
> > Another issue with the canned steps for "git gc" is that it means it
> > can't be used to do specific types of cleanup on a different schedule
> > from others. For example, we use "git pack-refs" directly to
> > frequently pack the refs in our repositories, separate from "git
> > repack" + "git prune" for repacking objects. That allows us to keep
> > our refs packed better without incurring the full overhead of
> > constantly building new packs.
>
> I am not sure if the above is an example of things that are good.
> We keep individual "pack-refs" and "rev-list | pack-objects"
> available exactly to give finer grained control to repository
> owners, and "gc" is meant to be one-size-fits-all easy to run
> by end users.  Adding options to "git gc --no-reflog --pack-refs"
> to complicate it sounds somewhat backwards.

I think we're in agreement there. I was citing the fact that GC isn't
good for targeted maintenance as a reason why we use "pack-refs"
directly, which sounds like what you're saying as well. I don't think
that inflating GC with options to skip specific steps is a good idea,
but that does mean that, for those targeted operations, we need to use
the lower-level commands directly, rather than GC.

Bryan

Re: Forcing GC to always fail

[Bryan Turner]

On Tue, Nov 27, 2018 at 5:55 PM Elijah Newren  wrote:
>
> On Tue, Nov 27, 2018 at 4:16 PM Ævar Arnfjörð Bjarmason
>  wrote:
> >
> > On Wed, Nov 28 2018, Bryan Turner wrote:
> >
> > > On Tue, Nov 27, 2018 at 3:47 PM Ævar Arnfjörð Bjarmason
> > >  wrote:
> > >>
> > >> On Tue, Nov 27 2018, Bryan Turner wrote:
> > >>
> > >> >
> > >> > Is there anything I can set, perhaps some invalid configuration
> > >> > option/value, that will make "git gc" (most important) and "git
> > >> > reflog" (ideal, but less important) fail when they're run in our
> > >> > repositories? Hopefully at that point customers will reach out to us
> > >> > for help _before_ they corrupt their repositories.
> > >>
> > >> $ stahp='Bryan.Turner.will.hunt.you.down.if.you.manually.run.gc' && 
> > >> git -c gc.pruneExpire=$stahp gc; git -c gc.reflogExpire=$stahp reflog 
> > >> expire
> > >> error: Invalid gc.pruneexpire: 
> > >> 'Bryan.Turner.will.hunt.you.down.if.you.manually.run.gc'
> > >> fatal: unable to parse 'gc.pruneexpire' from command-line config
> > >> error: 'Bryan.Turner.will.hunt.you.down.if.you.manually.run.gc' for 
> > >> 'gc.reflogexpire' is not a valid timestamp
> > >> fatal: unable to parse 'gc.reflogexpire' from command-line config
> > >
> > > Thanks for that! It looks like that does block both "git gc" and "git
> > > reflog expire" without blocking "git pack-refs", "git repack" or "git
> > > prune". Fantastic! The fact that it shows the invalid value means it
> > > might also be possible to at least provide a useful hint that manual
> > > GC is not safe.
> > >
> > > I appreciate your help, Ævar.
> >
> > No problem. I was going to add that you can set
> > e.g. pack.windowMemory='some.message' to make this go for git-repack
> > too, but it sounds like you don't want that.
> >
> > Is there a reason for why BitBucket isn't using 'git-gc'? Some other
> > hosting providers use it, and if you don't run it with "expire now" or
> > similarly aggressive non-default values on an active repository it won't
> > corrupt anything.

We did use "git gc" (sans options; its configuration was applied via
"config") for several years, but there were some rough edges.

The biggest one was that "git gc" has a canned set of steps it runs
that can't be disabled, even when they add no value (or are actively
detrimental).

For us, the biggest issue was "git gc"'s insistence on trying to run
"git reflog expire". That triggers locking behaviors that resulted in
very frequent GC failures--and the only reflogs Bitbucket Server (by
default) creates are all configured to never ex[ire or be pruned, so
the effort is all wasted anyway.

Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 270 bytes | 0 bytes/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: error: cannot lock ref 'stash-refs/pull-requests/13996/merge':
ref 'stash-refs/pull-requests/13996/merge' is at
2ec6a74b453d76f7a5247baa9f396361027ffdf but expected
1678f303202010c6d7e6201226df08dc8fc49ae3
remote: error: cannot lock ref 'stash-refs/pull-requests/13996/to':
ref 'stash-refs/pull-requests/13996/to' is at
bd32d53e4fe63b15be029085f1b6d795d526adbc but expected
f55ec06e89a11b8bdbcd97680a900361307d28c4
remote: error: cannot lock ref 'stash-refs/pull-requests/14006/merge':
ref 'stash-refs/pull-requests/14006/merge' is at
1cc907e2a7082033d70a164f222d3cce17a453a9 but expected
ae057003d7ed7d096b5b952191d784113f25b982
remote: error: cannot lock ref 'stash-refs/pull-requests/14006/to':
ref 'stash-refs/pull-requests/14006/to' is at
bd32d53e4fe63b15be029085f1b6d795d526adbc but expected
f55ec06e89a11b8bdbcd97680a900361307d28c4
remote: error: cannot lock ref 'stash-refs/pull-requests/14043/merge':
ref 'stash-refs/pull-requests/14043/merge' is at
a2e510b1b2b583b273f6d6d28e13151619e8d143 but expected
7735a5bde21815d307c68244e8fd2d67a09d5a39
remote: error: cannot lock ref 'stash-refs/pull-requests/14043/to':
ref 'stash-refs/pull-requests/14043/to' is at
bd32d53e4fe63b15be029085f1b6d795d526adbc but expected
f55ec06e89a11b8bdbcd97680a900361307d28c4
remote: error: cannot lock ref 'stash-refs/pull-requests/14047/merge':
ref 'stash-refs/pull-requests/14047/merge' is at
bd4f0e9bcbed34fd9befa65763f5aee6c9ebd8ce but expected
649dea948e8e6b54506615e5d61c6779c242d5af
remote: error: cannot lock ref 'stash-refs/pull-requests/14047/to':
ref 'stash-refs/pull-requests

Re: Forcing GC to always fail

[Bryan Turner]

On Tue, Nov 27, 2018 at 3:47 PM Ævar Arnfjörð Bjarmason
 wrote:
>
>
> On Tue, Nov 27 2018, Bryan Turner wrote:
>
> >
> > Is there anything I can set, perhaps some invalid configuration
> > option/value, that will make "git gc" (most important) and "git
> > reflog" (ideal, but less important) fail when they're run in our
> > repositories? Hopefully at that point customers will reach out to us
> > for help _before_ they corrupt their repositories.
>
> You could fix this and so many other issues by just hanging up a "Is
> This Good For The Company?" banner up in Atlassian offices .

Not sure I understand what this means, or what your goal was in saying
it. No one inside Atlassian is running these commands. I'm trying to
help save administrators from themselves, which reduces real world
end-user pain that comes from decisions made without fully
understanding the consequences. It feels like this comment is mocking
my earnest desire to help, and my genuine question looking for any
insight people more familiar with the code might be able to offer.
Perhaps I'm just missing the joke, but if it's an Office Space
reference it feels like it's in pretty poor taste.

>
> But more seriously:
>
> $ stahp='Bryan.Turner.will.hunt.you.down.if.you.manually.run.gc' && git 
> -c gc.pruneExpire=$stahp gc; git -c gc.reflogExpire=$stahp reflog expire
> error: Invalid gc.pruneexpire: 
> 'Bryan.Turner.will.hunt.you.down.if.you.manually.run.gc'
> fatal: unable to parse 'gc.pruneexpire' from command-line config
> error: 'Bryan.Turner.will.hunt.you.down.if.you.manually.run.gc' for 
> 'gc.reflogexpire' is not a valid timestamp
> fatal: unable to parse 'gc.reflogexpire' from command-line config

Thanks for that! It looks like that does block both "git gc" and "git
reflog expire" without blocking "git pack-refs", "git repack" or "git
prune". Fantastic! The fact that it shows the invalid value means it
might also be possible to at least provide a useful hint that manual
GC is not safe.

I appreciate your help, Ævar.

Bryan

Forcing GC to always fail

[Bryan Turner]

Something of an odd question, but is there something I can do in the
configuration for a repository that forces any "git gc" run in that
repository to always fail without doing anything? (Ideally I'd like to
make "git reflog expire" _also_ fail.)

Background: For Bitbucket Server, we have a fairly recurrent issue
where administrators decide they know how to manage garbage collection
for our repositories better than we do, so they jump on the server and
start running things like this:

git reflog expire --expire=now –all
git gc --prune=now
git repack -adf --window=200 --depth=200

They then come running to us with their corrupted repository expecting
and/or hoping that we can fix it (often without proper backups).

Bitbucket Server itself never runs "git gc" (or "git reflog expire").
We've configured how reflog expiry should be handled, but of course
that's overridden by explicit command line options like
"--expire=now". We _do_ run "git pack-refs", "git repack" and "git
prune" (with various options), so those commands need to continue to
work.

Is there anything I can set, perhaps some invalid configuration
option/value, that will make "git gc" (most important) and "git
reflog" (ideal, but less important) fail when they're run in our
repositories? Hopefully at that point customers will reach out to us
for help _before_ they corrupt their repositories.

Bryan

Re: Git for Windows v2.20.0-rc0, was Re: [ANNOUNCE] Git v2.20.0-rc0

[Bryan Turner]

On Wed, Nov 21, 2018 at 6:20 AM Jeff King  wrote:
>
> On Tue, Nov 20, 2018 at 03:17:07PM -0800, Bryan Turner wrote:
>
> > I've run 2.20.0-rc0 through the test matrix for Bitbucket Server on
> > both Linux and Windows, and the only failures were related to this
> > change:
> >
> > * "git branch -l " used to be a way to ask a reflog to be
> >created while creating a new branch, but that is no longer the
> >case.  It is a short-hand for "git branch --list " now.
> >
> > Since this is an intentional change I suspect there's nothing to do
> > for it but patch Bitbucket Server and move on, but I'll confess it's a
> > little frustrating that the option was deprecated in 2.19 and then
> > immediately removed in the next minor release. Such a
> > backwards-incompatible change seems far more apt for a major release,
> > a perspective that's reinforced by having the change follow such a
> > brief deprecation period--2.19.0 was only tagged September 10th (in my
> > timezone), so it's been less than 3 months. (Looking at the git branch
> > documentation for 2.18.0 [1] shows nothing about this deprecation; the
> > messaging first appears in 2.19.0 [2]. To be honest, I didn't even
> > realize it was deprecated until now, when it's gone--our test suite is
> > automated, so the deprecation warning was not visible.)
>
> We did go a bit faster than usual, under the assumption that nobody's
> really using "-l". It has been the default since 2006.
>
> Can you tell us a little more about your invocation?

Our invocation is... A little difficult to nail down, if I'm honest.
Bitbucket Server code does not use "git branch -l" anywhere in its
_shipping_ code, only in its _test_ code.

But that test code exists because Bitbucket Server provides a Java API
[1][2] which allows third-party developers to easily build arbitrary
Git commands to invoke for their own functionality. Setting
`GitBranchCreateBuilder.reflog(true)` will trigger adding "-l" to the
assembled "git branch" command. I've changed the code now so that it
will use "--create-reflog" instead; however, because many of the
Bitbucket Server add-ons on Marketplace [3], whether free or paid, are
not open source, and because there are a significant number of
in-house plugins that are not listed there, it's difficult to know how
many might be affected. If I were to hazard a guess it would be
_none_, but I've been surprised before. The end result is that the net
impact is hard to predict--especially because Git on the server would
need to be upgraded to 2.20.

(In case you're curious why we used shorthand options, it's because of
our Windows support. While "git branch" commands rarely, if ever, get
very long, as a general rule we use shorthand options where they exist
to keep our command lines shorter, to allow passing more options
without hitting the hard limit (generally 32K) imposed by
Windows--something we _have_ had issues with on other commands. For
commands like "git diff", where it's not possible to pass in paths via
stdin, every character matters.)

To try and protect against the unexpected, we have a Supported
Platforms [4] page which lists Git versions that we've _explicitly
tested_ with Bitbucket Server. 2.20 won't be marked tested until a
future release, so the majority of installs will not use it with older
versions of the system--but there's always that subset who ignore the
documentation. (Since we do text parsing on Git output, subtle
breakages do happen from time to time.)

I would say it's _unlikely_ that we'll hear of any installations where
all the conditions are met for this to come up:
- Git 2.20
- Bitbucket Server (without fixes)
- Third-party add-on using `reflog(true)`

It's really just that a) I was caught off guard by the change (my own
fault for not reading the 2.19 announcement more carefully) and b)
it's impossible for me to say with _certainty_ that it won't be an
issue. I'd imagine that latter point is true of the change in general,
though (it's not really possible to know what scripts it might break,
and that's going to be true regardless of when the change actually
gets released), and I'd agree that that shouldn't hold Git back from
making useful improvements.

Thanks for your time!

Bryan

[1] 
https://docs.atlassian.com/bitbucket-server/javadoc/5.16.0/git-api/reference/com/atlassian/bitbucket/scm/git/command/GitScmCommandBuilder.html
[2] 
https://docs.atlassian.com/bitbucket-server/javadoc/5.16.0/git-api/reference/com/atlassian/bitbucket/scm/git/command/branch/GitBranchCreateBuilder.html
[3] https://marketplace.atlassian.com/addons/app/bitbucket
[4] 
https://confluence.atlassian.com/bitbucketserver/supported-platforms-776640981.html#Supportedplatforms-dvcsDVCS

>
> We still have time to avoid the change for this release. And this early
> testing of master and release candidates is wonderful exactly to get
> this kind of feedback before it's too late.
>
> -Peff

Re: Git for Windows v2.20.0-rc0, was Re: [ANNOUNCE] Git v2.20.0-rc0

[Bryan Turner]

On Tue, Nov 20, 2018 at 12:57 PM Johannes Schindelin
 wrote:
>
> Team,
>
> On Sun, 18 Nov 2018, Junio C Hamano wrote:
>
> > An early preview release Git v2.20.0-rc0 is now available for
> > testing at the usual places.  It is comprised of 887 non-merge
> > commits since v2.19.0, contributed by 71 people, 23 of which are
> > new faces.
>
> The "for Windows" flavor of Git v2.20.0-rc0 is available here:
>

Thanks again for publishing these release candidates! I greatly
appreciate the effort that's involved, and the opportunity it affords
to test new versions prior to their final release.

I've run 2.20.0-rc0 through the test matrix for Bitbucket Server on
both Linux and Windows, and the only failures were related to this
change:

* "git branch -l " used to be a way to ask a reflog to be
   created while creating a new branch, but that is no longer the
   case.  It is a short-hand for "git branch --list " now.

Since this is an intentional change I suspect there's nothing to do
for it but patch Bitbucket Server and move on, but I'll confess it's a
little frustrating that the option was deprecated in 2.19 and then
immediately removed in the next minor release. Such a
backwards-incompatible change seems far more apt for a major release,
a perspective that's reinforced by having the change follow such a
brief deprecation period--2.19.0 was only tagged September 10th (in my
timezone), so it's been less than 3 months. (Looking at the git branch
documentation for 2.18.0 [1] shows nothing about this deprecation; the
messaging first appears in 2.19.0 [2]. To be honest, I didn't even
realize it was deprecated until now, when it's gone--our test suite is
automated, so the deprecation warning was not visible.)

For what it's worth, I think having -l mean --list is a _good change_,
and one that will likely be more natural for both new and existing
users. It's the rapid changeover that hurts.

Best regards,
Bryan Turner

[1] https://git-scm.com/docs/git-branch/2.18.0
[2] https://git-scm.com/docs/git-branch/2.19.0

Re: Fwd: Git credentials not working

[Bryan Turner]

On Wed, Oct 3, 2018 at 12:34 PM Dimitri Kopriwa  wrote:
>
> I have replaced the way I fill the git credentials store, I have verify
> ~/.git-credentials and information are there, the ~/.gitconfig look fine
> too.
>
> I still have 401 error when reading from that file.
>
> This is the paste log : https://paste.gnome.org/pmntlkdw0
>
> Now that I use git approve, I dont think that I need a custom helper.
>
> Any idea why I still can't log in using git-credential?

I'm pretty sure Peff touched on this in his reply. When it works,
you're either sending a "Private-Token" header or including it in the
URL, but, as Peff said, Git will never do either of those things. It
sends an "Authorization" header, and, based on their documentation, it
doesn't appear Gitlab accepts access tokens in that header.

It looks like you're either going to need to include it in the URL
(like what happens earlier in the posted trace), or adjust your git
config with a "http.extraHeader" set to "Private-Token: " to
include the "Private-Token" header (or you could pass it on the
command line, like `git -c http.extraHeader="Private-Token: "
clone ...`.

Hope this helps!
Bryan

>
> Thanks in advance,
>
> On 10/4/18 1:24 AM, Jeff King wrote:
> > On Thu, Oct 04, 2018 at 01:12:11AM +0700, Dimitri Kopriwa wrote:
> >
> >> Thanks for your reply. I have activated GIT_TRACE_CURL=1 and I can see that
> >> the request is failing 401.
> >>
> >> I can't see which token is used and using what header ?
> >>
> >> The log say:
> >>
> >> 17:50:26.414654 http.c:657  => Send header: Authorization: 
> >> Basic 
> > Yeah, we redact the auth information so people don't accidentally share
> > it publicly. If you use the older GIT_CURL_VERBOSE=1, it will include
> > the credential (I think it may be base64 encoded, though, so you'll have
> > to decipher it).
> >
> >> I have retested the token locally and it work when used in the url or using
> >> `Private-Token: ` as stated in the Gitlab documentation
> >> https://docs.gitlab.com/ee/api/README.html#personal-access-tokens
> > I don't think Git will ever send your token in either of those ways. It
> > will always some as an Authorization header.
> >
> >> Peff, what would be the appropriate way to input my git credential in a 
> >> 100%
> >> success way in a CI?
> > I don't know the details of what GitLab would want, but...
> >
> >> Is this good:
> >>
> >> git credential approve < >> protocol=https
> >> host=example.com
> >> username=bob
> >> password=secr3t
> >> OEF
> > Yes, that would work to preload a token into any configured helpers.
> >
> > -Peff

Re: [Possible GIT Bug]

[Bryan Turner]

On Sun, Sep 9, 2018 at 6:31 AM Dylan Young  wrote:
>
> Works:
>
> git show -C --find-copies-harder  055f6c89fa4506037d1621761f13430f469b8029
>
> git show -C --find-copies-harder
> 055f6c89fa4506037d1621761f13430f469b8029 --name-status

Here, because you didn't provide _any_ paths, Git is allowed to
consider all of the paths modified in the commit and, because you
specified --find-copies-harder, it's allowed to consider paths that
_weren't_ modified too. That means it can "see" both the source and
destination for the copy, and it detects the copy as you're expecting.

>
> Doesn’t Work:
>
> git show -C --find-copies-harder
> 055f6c89fa4506037d1621761f13430f469b8029  --  PATH_TO_MY_COPIED_FILE

Here, though, you've _explicitly limited_ Git to only the copied file.
It's not allowed to consider any others, which means it can't "see"
the source path anymore. As a result, the copy is detected as a
straight add. Note that --find-copies-harder means the diff machinery
is allowed to consider files that weren't modified in the commit as
possible sources for copies, but that's still subject to your explicit
filtering. In other words, if PATH_TO_SOURCE_FILE wasn't modified,
running this would _not_ see a copy:

git show -C 055f6c89fa4506037d1621761f13430f469b8029  --
PATH_TO_MY_COPIED_FILE PATH_TO_SOURCE_FILE

But running this would:

git show -C -C 055f6c89fa4506037d1621761f13430f469b8029  --
PATH_TO_MY_COPIED_FILE PATH_TO_SOURCE_FILE

No bugs here. Everything is working as intended, if not, perhaps, as
you expected.

Hope this helps,
Bryan

>
> i.e.
>
> --- /dev/null
>
> +++ b/ PATH_TO_MY_COPIED_FILE
>
> Hope that’s self-explanatory!!!
>
> Best,
> Casey Meijer

Re: Get "Your branch is ahead of 'origin/master'" message when explicitly passing origin url at push command

[Bryan Turner]

On Sun, Aug 26, 2018 at 4:39 AM Bentzy Sagiv  wrote:
>
> git version 2.7.4
> __
>
> First try: NOT passing origin url explicitly
>
> ubuntu@ci-bentzy:/var/lib/jenkins$ sudo git push

Since you didn't specify a remote here, Git assumes origin. It uses
your configured "push.default" behavior ("simple" by default) to
determine which refs to push and pushes your master branch to the
origin's master branch. Since it _knows_ it pushed to origin, it
updates your local "refs/remotes/origin/master" ref with the same
commit it just pushed, resulting in an "up-to-date" message.

>
> ubuntu@ci-bentzy:/var/lib/jenkins$ git status
> On branch master
> Your branch is up-to-date with 'origin/master'.
> nothing to commit, working directory clean
> ubuntu@ci-bentzy:/var/lib/jenkins$
>
> __
>
> Second try: passing origin url explicitily
>
> ubuntu@ci-bentzy:/var/lib/jenkins$ sudo git push 
> https://bitbucket.org/OWNER/jenkinsconf-repo.git

This, on the other hand, _is not pushing to a configured remote_. It's
pushing to an explicit URL, which happens to match the URL of a
configured remote. But it's still not a configured remote. It's using
origin's URL, but you didn't push to origin. As a result,
"refs/remotes/origin/master" is not updated, and you get an "ahead"
message.

>
> ubuntu@ci-bentzy:/var/lib/jenkins$ git status
> On branch master
> Your branch is ahead of 'origin/master' by 1 commit.
>   (use "git push" to publish your local commits)
> nothing to commit, working directory clean
>
> __
>
> An additional " sudo git push" (without explicit origin) solves the issue

Everything here is working as intended. If you want to push to a
_remote_, you either need to:
- Name the remote ("git push origin"), or
- Leave it off, so Git will assume origin ("git push")

Pushing to a URL that matches a remote's URL is _not_ pushing to a
remote. It's pushing to an explicit URL.

Hope this helps,
Bryan Turner
>
>
>

Re: Request for testing v2.19.0-rc0 *with builtin stash/rebase*

[Bryan Turner]

On Fri, Aug 24, 2018 at 5:14 AM Johannes Schindelin
 wrote:
>
> For that reason, I was delighted to see that our Google Summer of Code
> pushed pretty hard in that direction. And I could not help myself so I had
> to test how much faster things got. Here is the result of my first, really
> quick and dirty test:
>
> without builtin stash/rebasewith builtin stash/rebase
> t3400 (rebase)  1m27s   32s
> t3404 (rebase -i)   13m15s  3m59s
> t3903 (stash)   8m37s   1m18s
>
> What can I say? Even if the numbers are off by as much as 10%, these are
> impressive improvements: keep in mind that there is a lot of churn going
> on in the test suite because it is itself implemented in Unix shell
> scripts (and hence I won't even bother to try more correct performance
> benchmarking because that is simply not possible when Unix shell scripts
> are in the equation). So the speed improvements of the stash/rebase
> commands are *even higher* than this.

Thanks for taking the time to make these pre-releases available. I
appreciate the effort. And the same to Junio, for always posting
release candidates. We rely on them heavily to find changes that might
cause issues before admins start upgrading in the wild and find them
for us.

I downloaded both the rc0.1 and rc0.2 builds, as well as 2.18.0, and
ran them all through Bitbucket Server's test suite a few times (to
ensure warm disk for comparable numbers). I added support for some
"simple" rebase cases a few releases ago, so we have a set of tests
that verify the rebase behaviors we use. (We don't use git stash, so
we don't have any tests in our suite for that.)

Running our entire Git test suite (~1,600 tests) against Git for
Windows 2.18.0 takes ~5 minutes, and 2.19.0-rc0.1 produced an almost
identical duration. Running our tests against rc0.2 cut the duration
down to 4 minutes. There were no test failures on either pre-release
build.

To try and get a better sense of the rebase performance improvement
specifically, I filtered down to a set of 14 specific tests which use
it and ran those. On Git 2.18, those 14 tests take just over 19
seconds. On 2.19.0-rc0.2, they take just over 8 seconds.

When they do ship, whether it's in 2.19 (by default or optional) or
later, the changes definitely offer some significant performance wins.

Thanks again, to everyone involved, for all the effort that went into
designing, implementing, reviewing and releasing these improvements.
As someone who develops under Windows most of the time, they make a
big difference in my day to day work. And that's not to mention all
the Bitbucket Server and Bitbucket Data Center users who will enjoy a
snappier experience as these changes make their way out into the wild.

Best regards,
Bryan Turner

Re: "Changes not staged for commit" after cloning a repo on macOS

[Bryan Turner]

On Wed, Aug 15, 2018 at 1:50 PM Hadi Safari  wrote:
>
> Hi everyone!
>
> I encountered a strange situation on OS X recently. I cloned a
> repository (https://github.com/kevinxucs/Sublime-Gitignore.git), went to
> folder, and saw "Changes not staged for commit" message for four
> specific files. It happened every time I repeated the procedure. I even
> was able to commit those to the repo.
> At first I thought there's something wrong with repo, but I cloned it on
> Ubuntu 16.04 and everything was OK; no "Changes not staged for commit"
> message.
>
> Does anyone have any idea?
>
>  modified:   boilerplates/Nanoc.gitignore
>  modified:   boilerplates/OpenCart.gitignore
>  modified:   boilerplates/SASS.gitignore
>  modified:   boilerplates/WordPress.gitignore

Taking a look at the repository's file list on GitHub[1], it shows
that this is because HFS and APFS by default are case-insensitive.

The file listing shows that there is a "nanoc.gitignore" and
"Nanoc.gitignore". On APFS and HFS, those are the same file. As a
result, one overwrites the other. This is discussed pretty regularly
on the list[2], so you can find more details there.

[1]: https://github.com/kevinxucs/Sublime-Gitignore/tree/master/boilerplates
[2]: 
https://public-inbox.org/git/24a09b73-b4d4-4c22-bc1b-41b22cb59...@gmail.com/
is a fairly recent (fairly long) thread about this behavior.

Hope this helps!
Bryan

Re: Unexpected ignorecase=false behavior on Windows

[Bryan Turner]

On Mon, Jun 25, 2018 at 9:34 AM Junio C Hamano  wrote:
>
> Bryan Turner  writes:
>
> > Git on Windows is not designed to run with anything other than
> > core.ignoreCase=true, and attempting to do so will cause unexpected
> > behavior.
>
> Even though I fully agree with your conclusion that the document
> must make it crystal clear that core.ignoreCase must be set to
> reflect the reality, I found the above statement misleading and do
> not want it to be used as the basis of a documentation update.  But
> it is possible that I am misunderstanding the current state of
> affairs.
>
> Is the case insensitivity that deeply ingrained in the Git for
> Windows code?
>
> IOW, even if the code used to build Git for Windows were executed on
> a case sensitive filesystem, is there a case-smashing code on _our_
> side that kicks in to cause unexpected behaviour, _even_ when
> core.ignorecase is set to false to match (hypothetical) reality?
>
> To put it yet another way, if a case sensitive filesystem were
> available, wouldn't running "git init" from Git for Windows in a
> directory on such a filesytem set core.ignoreCase to false in the
> resulting repository and from then on wouldn't everything work fine?
>
> If my suspicion (i.e. the code for Git for Windows is perfectly
> fine---it is just the users are not running with case sensitive
> filesystems and flipping core.ignoreCase to true does not make case
> incapable filesystems suddenly capable) is correct, then it is not
> "Git on Windows is not designed to run" from two angles.  (1) it is
> not just Git for Windows---Git running on UNIX that mounts VFAT, or
> Git running on macOS with default HFS+, would exhibit the same
> symptom, and (2) it is not "Git is not designed to run"---it is
> core.ignoreCase that is not designed to be a way to make case
> incapable filesystems suddenly capable of distinguishing cases in
> filesystems.

Apologies for the unclear word choice. Given Git was designed first to
work with case-sensitive filesystems, certainly the obvious (and
correct) conclusion is that Git itself is fine in a case-sensitive
environment. It wasn't my intention to suggest otherwise.

Note that my word choice was not "Git _for_ Windows", however; it was
"Git _on_ Windows". (This still doesn't change the correctness of your
clarification, let me be quick to add. I'm only pointing it out
because it's relevant to what I intended the comment to say.) On
Windows, the default filesystem is NTFS, and NTFS is not case
sensitive. Hence, Git on Windows (by which I'm implying Git on NTFS),
is not designed to run with anything other than core.ignoreCase=true,
because that setting aligns Git's expectations with how the underlying
filesystem actually works. In other words, Git on a case-insensitive
filesystem (APFS, HFS+, FAT32, exFAT, vFAT, NTFS, etc.) is not
designed to be run with anything other than core.ignoreCase=true.

Bryan

>
> Thanks.

Re: Unexpected ignorecase=false behavior on Windows

[Bryan Turner]

On Fri, Jun 22, 2018 at 1:45 PM Marc Strapetz  wrote:
>
> On 22.06.2018 19:36, Johannes Sixt wrote:
> > Am 22.06.2018 um 14:04 schrieb Marc Strapetz:
> >> On Windows, when creating following repository:
> >>
> >> $ git init
> >> $ echo "1" > file.txt
> >> $ git add .
> >> $ git commit -m "initial import"
> >> $ ren file.txt File.txt
> >> $ git config core.ignorecase false
> >
> > This is a user error. core.ignorecase is *not* an instruction as in
> > "hey, Git, do not ignore the case of file names". It is better regarded
> > as an internal value, with which Git remembers how it should treat the
> > names of files that it receives when it traverses the directories on the
> > disk.
> >
> > Git could probe the file system capabilities each time it runs. But that
> > would be wasteful. Hence, this probe happens only once when the
> > repository is initialized, and the result is recorded in this
> > configuration value. You should not change it.
>
> Sorry, it looks like my example was misleading. I'm actually questioning
> current behavior in case of Windows repositories with core.ignorecase
> initialized to false, like in following setup:
>
> $ git init
> $ git config core.ignorecase false
>
> The repository is now set up to be case-sensitive on Windows. From this
> point on, core.ignorecase won't change anymore and the repository will
> be filled:

I don't think Hannes's point was _when_ you changed it; it was that
you changed it _at all_.

Git on Windows is not designed to run with anything other than
core.ignoreCase=true, and attempting to do so will cause unexpected
behavior. In other words, it's not a behavior toggle so user's can
request the functionality to work one way or the other; it's an
implementation detail that `git init` and `git clone` set when a
repository is created purely so they don't have to probe the file
system each time you run a `git` command.

NTFS is case-preserving-but-case-insensitive by default[1]. So long as
that's the case, the only mode for running Git on Windows is
core.ignoreCase=true.

Hopefully this clarifies things!

Bryan

[1] Windows 10 1803 introduced the ability to set a folder as
case-sensitive[2], but, since it's not inherited automatically by
subdirectories, it still doesn't work well for Git.
[2] 
https://blogs.msdn.microsoft.com/commandline/2018/02/28/per-directory-case-sensitivity-and-wsl/

>
> $ echo "1" > file.txt
> $ git add .
> $ git commit -m "initial import"
> $ ren file.txt File.txt
>
> Still, status results are:
>
> $ git status --porcelain
> ?? File.txt
>
> With the same setup sequence on Unix, it's:
>
> $ git status --porcelain
>D file.txt
> ?? File.txt
>
> Is this difference, which is depending on the platform, intended? Why
> not report missing file.txt as well?
>
> The drawback of the current behavior is that a subsequent "git add ."
> will result in two file names in the .git/index which are only differing
> in case. This will break the repository on Windows, because only one of
> both files can be checked out in the working tree. Also, it makes
> case-only renames harder to be performed.
>
> -Marc

Re: git question from a newbie

[Bryan Turner]

On Tue, Jun 5, 2018 at 2:33 PM Heinz, Steve  wrote:
>
> Hi.
>
> I am new to Git and have read quite a few articles on it.
> I am planning on setting up a remote repository on a windows 2012 R2 server 
> and will access it via HTTPS.
> I am setting up a local repository on my desk top (others in my group will do 
> the same).
> On "server1":  I install Git and create a repository "repos".
> On "server1":  I create a dummy webpage "default.htm" and place it in the 
> repo folder.
> On "server1":  I create a web application in IIS pointing to Git
> On Server1":   change permissions so IIS_User  has access to the folders.
> On "server1":  inside the "repos" folder and right click and choose "bash 
> here"
> On "server1":   $ git init  -bare(it's really 2 hyphens)


This might create a _repository_, but it's not going to set up any Git
hosting processing for it. You might be able to clone using the
fallback to the "dumb" HTTP protocol (though I doubt it, with the
steps you've shown) , but you won't be able to push.

You need handlers for git-http-backend which handle info/refs and
other requests that are related to the Git HTTP wire protocol.[1]

Documentation for setting up Git's HTTP protocol via Apache are pretty
easy to find[2], but IIS instructions are a bit more sparse. I don't
know of any good ones off the top of my head. But that's your issue;
your IIS setup isn't really a valid Git remote; it's just a Git
repository with contents visible via HTTP.

[1] 
https://github.com/git/git/blob/master/Documentation/technical/http-protocol.txt
[2] 
https://github.com/git/git/blob/master/Documentation/howto/setup-git-server-over-http.txt

Bryan

Re: Git case-sensitivity bug

[Bryan Turner]

Hey Cliff,

On Thu, May 10, 2018 at 12:46 PM Cliff <miliw...@gmail.com> wrote:

> I believe I have discovered a bug with git tools.

> If you create a git branch, you can refer to that branch with
> case-insensitive alterations and it will track as the same branch.

This comes up on the list fairly often on the list. The most recent thread
is here:
https://public-inbox.org/git/d4d8d8208b6a41c380ecf20807763...@xch15-05-02.nw.nos.boeing.com/

The replies to that thread explain the behavior, why it does what it does,
and why it hasn't been changed yet. It happens on case-insensitive macOS
filesystems and on NTFS on Windows.

Hope this helps!
Bryan Turner

Re: [Best Practices Request] clean/smudge configuration

[Bryan Turner]

On Wed, May 9, 2018 at 3:09 PM Randall S. Becker 
wrote:

> The question: what is the best practice for versioning the parts of
> clean/smudge filters that are in .git/config given that only some users in
> my environment will be cloning the repository in question and that I
really
> can't put the entries in /etc/gitconfig or ~/.gitconfig because of
potential
> conflicts with other repositories that might also have clean/smudge
> definitions.

Depending on level of trust, one approach might be to use an [include] in
.git/config to include a file that's in the repository. Something like:

[include]
 path = ../path/to/config

That way that part of the configuration can be derived from the
repository's contents. If you checkout a revision where the file doesn't
exist, then Git just ignores the include.

You're trusting the repository's contents, though, at that point; whatever
is in that file will be included in your overall config and honored by Git.

Bryan

Re: Fetching tags overwrites existing tags

[Bryan Turner]

On Fri, Apr 27, 2018 at 12:08 PM, Wink Saville  wrote:
>
> The other change was rather than using 
> ""+refs/tags/*:refs/remote-tags/$name/*"
> I've changed it to "+refs/tags/*:refs/remote/tags/$name/*" which seems 
> cleaner.
> Again, if remote-tags is preferred I'll change it back.


>From looking at the code, it looks like you mean
"+refs/tags/*:refs/remotes/tags/$name/*".

The issue with that approach is that it collides with a remote named
"tags". "refs/remote-tags", on the other hand, represents a new-to-Git
path, one that won't already be in use by any other standard
functionality. That seems like a better approach than hoping no one
out there will call one of their remotes "tags".

Bryan

Re: Bug Report - Pull remote branch does not retrieve new tags

[Bryan Turner]

Andrew,

On Thu, Apr 19, 2018 at 6:55 AM, Andrew Ducker
 wrote:
>
> What happens:
> When I create a new tag on the remote (changing nothing else)
> "git pull origin master" produces the following:
>   From git.internal.company.com:team/testrepo
>* branchmaster -> FETCH_HEAD
>   Already up-to-date.
>
> If I instead do a "git pull" I get:
>   From git.internal.company.com:team/testrepo
>* [new tag] Testing11  -> Testing11
>   Already up-to-date.
>
> What I think should happen:
> The "git pull origin master" should retrieve the tag.
>
> This is with 2.16.2.windows.1, but also occurred on my previously installed 
> version (2.12.2.windows.2)
>
> My understanding is that "git pull" and "git pull $repo $currentbranch" 
> should function identically.
>
> Is this a bug, or am I misunderstanding the manual page?

Looks like a misunderstanding, to me. Perhaps I can help clarify.

"git pull" without arguments fetches from the "origin" repository
using the configured "fetch" refspecs, which typically looks something
like "fetch = +refs/heads/*:refs/remotes/origin/*". It _doesn't_
actually fetch all tags, but any tag referencing any object/commit
included in the branches is brought along for the ride. This is
documented on "git pull":

--no-tags

By default, tags that point at objects that are downloaded from
the remote repository are fetched and stored locally. This option
disables this automatic tag following. The default behavior for a
remote may be specified with the remote..tagOpt setting. See
git-config(1).

By comparison, on your "git pull $repo $currentBranch", what you're
calling "$currentBranch" is actually "[...]" from the
documentation. In other words, by passing "master", you've told "git
pull" to fetch _nothing but "master"_, ignoring the configured
refspec(s). Additionally, since you haven't told "git pull" where to
_put_ "master" once it's fetched, it writes it to "FETCH_HEAD". If you
have a tracking branch setup, "git pull origin master" will also
update the tracking branch. For example, the same command for me
produces:

$ git pull origin master
>From ...
 * branchmaster -> FETCH_HEAD
   aca5eb0fef5..ad484477508  master -> origin/master

As you can see, both FETCH_HEAD and origin/master were updated, since
my local "master" tracks "origin"'s "master":

[branch "master"]
remote = origin
merge = refs/heads/master

Hope this helps!
Bryan

Re: Self-inflicted "abort" in a newbie attempt at read-only exploration of a cloned repository?

[Bryan Turner]

On Thu, Apr 5, 2018 at 4:18 PM, Bryan Turner <btur...@atlassian.com> wrote:
> On Thu, Apr 5, 2018 at 12:42 PM, Thierry Moreau
> <thierry.mor...@connotech.com> wrote:
>> Dear GIT enthusiasts!
>>
>> This ends up with a "git checkout" command aborting. A bit frustrating at
>> the early stage of GIT learning curve.
>>
>> My first goal is to clone repositories locally in order to explore the
>> various linux kernel versions, with the rich GIT metadata.
>>
>> Thus, I clone:
>>
>> $  git clone --branch linux-4.16.y
>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
>> linux-stable
>> $  git -C linux-stable/ branch
>> * linux-4.16.y
>>
>> So far so good. Then, I want to extract an earlier kernel version into a tmp
>> dir:
>>
>> $  mkdir tmp
>> $  git -C linux-stable/ --work-tree $PWD/tmp/ checkout linux-4.15.y
>> $  git -C linux-stable/ branch
>> * linux-4.15.y
>>   linux-4.16.y
>
> The documentation for --work-tree says:
>
> --work-tree=
>
> Set the path to the working tree. It can be an absolute path or a path
> relative to the current working directory. This can also be controlled
> by setting the GIT_WORK_TREE environment variable and the
> core.worktree configuration variable (see core.worktree in
> git-config(1) for a more detailed discussion).
>
> So passing --work-tree tells Git where to store your _files_, but it's
> still using the same .git directory.
>
> If your goal is to have worktrees for various versions, that implies
> the git worktree [1] command might be more along the lines of what
> you're looking for. An invocation based on above might look like this:
> $ git -C linux-stable/ worktree add $PWD/tmp/ checkout linux-4.15.y

Apologies, I didn't mean to have the "checkout" in that.
$ git -C linux-stable/ worktree add $PWD/tmp/ linux-4.15.y

>
> That should leave linux-4.16.y checked out in linux-stable, while
> creating a full work tree in $PWD/tmp that has 4.15.y checked out.
>
> Note that worktree is a newer git command. 2.17 has it, but old
> versions like 2.1 won't.
>
> [1] https://git-scm.com/docs/git-worktree
>
> Hope this helps!
> Bryan

Re: Self-inflicted "abort" in a newbie attempt at read-only exploration of a cloned repository?

[Bryan Turner]

On Thu, Apr 5, 2018 at 12:42 PM, Thierry Moreau
 wrote:
> Dear GIT enthusiasts!
>
> This ends up with a "git checkout" command aborting. A bit frustrating at
> the early stage of GIT learning curve.
>
> My first goal is to clone repositories locally in order to explore the
> various linux kernel versions, with the rich GIT metadata.
>
> Thus, I clone:
>
> $  git clone --branch linux-4.16.y
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
> linux-stable
> $  git -C linux-stable/ branch
> * linux-4.16.y
>
> So far so good. Then, I want to extract an earlier kernel version into a tmp
> dir:
>
> $  mkdir tmp
> $  git -C linux-stable/ --work-tree $PWD/tmp/ checkout linux-4.15.y
> $  git -C linux-stable/ branch
> * linux-4.15.y
>   linux-4.16.y

The documentation for --work-tree says:

--work-tree=

Set the path to the working tree. It can be an absolute path or a path
relative to the current working directory. This can also be controlled
by setting the GIT_WORK_TREE environment variable and the
core.worktree configuration variable (see core.worktree in
git-config(1) for a more detailed discussion).

So passing --work-tree tells Git where to store your _files_, but it's
still using the same .git directory.

If your goal is to have worktrees for various versions, that implies
the git worktree [1] command might be more along the lines of what
you're looking for. An invocation based on above might look like this:
$ git -C linux-stable/ worktree add $PWD/tmp/ checkout linux-4.15.y

That should leave linux-4.16.y checked out in linux-stable, while
creating a full work tree in $PWD/tmp that has 4.15.y checked out.

Note that worktree is a newer git command. 2.17 has it, but old
versions like 2.1 won't.

[1] https://git-scm.com/docs/git-worktree

Hope this helps!
Bryan

Re: [ANNOUNCE] Git v2.17.0-rc1

[Bryan Turner]

On Fri, Mar 23, 2018 at 10:47 AM, Johannes Schindelin
 wrote:
> Hi team,
>
> On Wed, 21 Mar 2018, Junio C Hamano wrote:
>
>> A release candidate Git v2.17.0-rc1 is now available for testing
>> at the usual places.  It is comprised of 493 non-merge commits
>> since v2.16.0, contributed by 62 people, 19 of which are new faces.
>>
>> The tarballs are found at:
>>
>> 
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.kernel.org_pub_software_scm_git_testing_=DwIBAg=wBUwXtM9sKhff6UeHOQgvw=uBedA6EFFVX1HiLgmpdrBrv8bIDAScKjk1yk9LOASBM=yXNBIWf9n-gxAIgQyCzXfuKaFkHQaMmwUdtiNBNE8XI=E_Z2M418iwz-HyJg5D0VyTCvyMMd4kGIvYccgJkyTwA=
>
> And Git for Windows v2.17.0-rc1 can be found here:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_git-2Dfor-2Dwindows_git_releases_tag_v2.17.0-2Drc1.windows.1=DwIBAg=wBUwXtM9sKhff6UeHOQgvw=uBedA6EFFVX1HiLgmpdrBrv8bIDAScKjk1yk9LOASBM=yXNBIWf9n-gxAIgQyCzXfuKaFkHQaMmwUdtiNBNE8XI=7ePu15Fwlwuxo8JGcqj-pBNh1wSZYAfYmboqBvJOyA0=
>
> Please test so that we can hammer out a robust v2.17.0!

I've added 2.16.3 and 2.17.0-rc1, for both Linux and Windows, to the
test matrix for Bitbucket Server. All ~1500 tests have passed for all
4 versions.

Bryan

Re: getting fatal error trying to open git gui

[Bryan Turner]

On Fri, Mar 16, 2018 at 2:01 PM, Briggs, John  wrote:
> No, it was a fresh install.  Plus file search reveals only one copy of the 
> file.
>
> I also noticed that I cannot use the file properties to run as administrator. 
> I must right-click on Git GUI and select "More >> Run as administrator" in 
> the start menu. Even though I have "run as administrator" checked on both 
> shortcut and the program (or any combo of).

You definitely shouldn't need to run it as an administrator at all. I
have the same version installed:

$ git version
git version 2.16.2.windows.1

Running Git GUI from the Start menu, it starts without a UAC prompt
(which means it's not running as Administrator) and runs without
issue. Starting it from a non-escalated command prompt via "git gui"
also does not have a UAC prompt and runs without issue.

Seems like there must be something more going on. What options did you
select during installation? What installer did you use?

Bryan

Re: Please test Git for Windows' latest snapshot

[Bryan Turner]

On Fri, Feb 16, 2018 at 3:30 PM, Johannes Schindelin
 wrote:
> Hi team,
>
> I am unwilling to release Git for Windows v2.16.2 on a Friday night, but I
> have something almost as good. There is a snapshot available here:
>
> 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wingit.blob.core.windows.net_files_index.html=DwIBAg=wBUwXtM9sKhff6UeHOQgvw=uBedA6EFFVX1HiLgmpdrBrv8bIDAScKjk1yk9LOASBM=xZghHWteeNbJ2bu5ySDq9WwqnfX8X7FZ_CWsV9gAyJU=NzSYCFSWWokPP9A9FA_EmJO5yu8qtRKw5M-Ep_qooUc=
>
> That snapshot brings quite a few updated components apart from Git proper
> (such as an updated MSYS2 runtime), and I would love to ask y'all to give
> this snapshot a proper "tire kicking".

I've run Bitbucket Server's full Git test suite (~1,500 tests) against
the Portable Git snapshot (e1848984d1), no failures to report.

Bryan

Re: repository history?

[Bryan Turner]

On Tue, Feb 6, 2018 at 1:41 AM, Zsolt SZALAI  wrote:
> Hi,
>
> I wonder if there is a feature with which the history of the
> repository can be listed?
> i am interested in especially the usecases of pull and push, i.e. to
> query when the a branch was refreshed with remote changes and which
> commits were pulled, things like that.
> Is this possible?

Have you looked at the "git reflog"[1] command? That can show the old
and new SHAs when a ref changes, and you can use "git log" to
determine which commits were added. That said, it's most useful for
fetch/pull; push information won't be as easily derived from your
local reflogs.

[1] https://git-scm.com/docs/git-reflog

Hope this helps!
Bryan

2 conflicts referencing the same path?

[Bryan Turner]

While investigating an issue with rendering conflicts on a pull
request, I noticed that the merge was producing this output (sanitized
for paths)

$ git merge --no-ff --log -m "Test" 190a25b6e0f32c7b8ccddf8c31e054149dece8b7
CONFLICT (rename/add): Rename A->B in HEAD. B added in
190a25b6e0f32c7b8ccddf8c31e054149dece8b7
Adding as B~190a25b6e0f32c7b8ccddf8c31e054149dece8b7 instead
...
Auto-merging B
CONFLICT (content): Merge conflicts in B

(There are several other conflicts listed "between" the two I'm
showing here, various rename/add, add/add and content conflicts, but
I'm trimming those out to focus on the lines that I think are relevant
to my question.)

This merge produces 2 (or is it 3?) conflicts for the same B path:
- Rename A to B in HEAD, add B in 190a25b
- Content conflicts in B

The version of B left in place _does_ contain conflict markers, after
the merge processes completes. The version added as B~190a25b has no
conflict markers; it just shows a small number of inserted lines.

The merge in question produces 23 different CONFLICT lines, but aside
from "composite" conflicts (rename/add, add/add) that can include the
same path multiple times in their output, only this one path is
mentioned in multiple CONFLICT lines.

I'm still trying to produce a set of steps that will allow a minimal
reproduction, but I thought I'd post to the list just to see if anyone
had any thoughts on how it could happen. Is it a "normal" (albeit
rare) case? Or could it represent some sort of issue in Git's 3-way
merge algorithm (in its behavior or perhaps in how the merge conflicts
are logged)?

Any insights appreciated!
Bryan Turner

Re: Location limits on development, staging and production environments

[Bryan Turner]

On Mon, Jan 29, 2018 at 11:08 AM, H  wrote:
> I am a newcomer to git looking to set up a web development environment where 
> individual computers are used for development, the development.git, 
> staging.git and production.git repositories are stored on an external server 
> reachable by password-less ssh and the staging and production websites are on 
> yet another server, also reachable by password-less ssh from the git-server 
> (and the development machines).
>
> Locating the three git repositories on an external server works fine but I 
> have not been able to have the staging and production deployment files on 
> another server. I believe this is what is referred by GIT_WORK_TREE and based 
> on what I found on the web I created a post-receive hook of staging.git with 
> the two lines:
>
> #!/bin/sh
> GIT_WORK_TREE=user@1.2.3.4:/var/www/html/dev.whatever git checkout -f master
>
> I believe this should deploy the files from the development work tree.
>
> The above, however, fails. Should it work? I am running git 1.7.1 on CentOS 6.

No, I wouldn't expect that to work. GIT_WORK_TREE is not remote-aware
in that way. It's expected to be a normal-ish filesystem path.

Based on your description, and the hook you've written, it seems like
your intention is for the source to automatically be fetched and
checked out on the staging environment after each push. (This is
dangerous, and likely _not_ what you actually want, but I'll get to
that in a moment.)

One option would be to setup something like NFS, so the git-server can
mount the filesystems from the staging and production nodes.

A different, likely better, option would be to have the post-receive
script on the git-server use straight ssh to trigger a checkout script
on the staging server, e.g.:
#!/bin/sh
ssh example@staging-server -C /opt/deploy-staging.sh

Your deploy-staging script would then do something like:
#!/bin/sh
GIT_WORK_TREE=/var/www/html/dev.whatever git pull origin

That said, though, having such a simple script is dangerous because
Git is fully capable of having receiving multiple pushes concurrently,
and they can all succeed as long as they're updating different
branches. Since your script isn't considering what branches were
changed by the push, it could end up triggering simultaneous git
processes on the staging server all attempting to deploy concurrently.

The stdin for the post-receive hook receives details about which refs
were changed, and you'll likely want to update your script to parse
stdin and only try to deploy staging if a specific, relevant branch
(master in your example) has changed.

Lastly, I'll note that using post-receive will make the pushing
(remote) user wait while the staging server is deployed. If that
process is likely to take very long, you might want to decouple the
two somehow.

Hope this helps!

Re: git 2.16, Jenkins git client plugin, and ""

[Bryan Turner]

On Wed, Jan 24, 2018 at 3:55 PM, Mark Waite  wrote:
> It appears that git 2.16.0 and 2.16.1 have introduced a change which
> surprises the Jenkins git client plugin.
>
> Git 2.16.0 and 2.16.1 on Linux both report "fatal: ssh variant
> 'simple' does not support setting port" when used in the context of
> the Jenkins git client plugin.

We noticed a similar issue in Bitbucket Server's tests, related to us
using a wrapper script with the GIT_SSH variable. Some further
discussion is available in [1].

>
> The solution we've accepted into the git client plugin source code is
> to set the environment variable "GIT_SSH_VARIANT=ssh".  That allows
> the failing case to pass, and does not seem to harm older versions of
> git.
>
> The documentation at https://git-scm.com/docs/git says that
> GIT_SSH_VARIANT "overrides Git’s autodetection whether
> GIT_SSH/GIT_SSH_COMMAND/core.sshCommand refer to OpenSSH, plink or
> tortoiseplink."
>
> I haven't seen the same message from Git 2.16.1 for Windows.

Our Windows builds didn't seem to fail for this either, which was interesting.

>
> The Jenkins bug report
> (https://issues.jenkins-ci.org/browse/JENKINS-49111) describes the
> user experience.
>
> Is the GIT_SSH_VARIANT solution expected to be used by git consumers
> like the Jenkins git client plugin when passing credential information
> through environment variables like SSH_ASKPASS?

The expectation seems to be that if you use GIT_SSH or
GIT_SSH_COMMAND, you'll also use GIT_SSH_VARIANT to tell Git what to
expect, because it doesn't (shouldn't?) make assumptions, starting in
2.16, that your custom SSH uses OpenSSH syntax.

>
> I see "ssh", "plink", and "tortoiseplink" as values of GIT_SSH_VARIANT
> used in tests, and "auto", "putty", and "simple" used in the source
> code in addition to "ssh", "plink", and "tortoiseplink".  What are the
> allowed values for GIT_SSH_VARIANT?
>
> What is the recommended value so that the Jenkins git client plugin
> can remain most compatible with previous behavior?

"ssh" is probably the most compatible, because I'm pretty sure prior
to 2.16 that behavior was the assumed behavior.

>
> No value was assigned previously to GIT_SSH_VARIANT in the Jenkins git
> client plugin.  I was expecting that git would choose "ssh" as the
> default value for GIT_SSH_VARIANT, rather than apparently choosing
> "simple".
>
> Is this a bug, or is this the new, intentional behavior of git?

It's intentional. There was some discussion (again, see [1]) about
whether the change needed further changes, but it doesn't appear any
further changes were made.

>
> Thanks,
> Mark Waite

[1]: 
https://public-inbox.org/git/cagyf7-fqp4q2vvh1ponqvmvdtu0himsk1jkytqz4o1i0mcn...@mail.gmail.com/

Hope this helps!
Bryan

Re: Can't squash merge with merge.ff set to false

[Bryan Turner]

On Fri, Jan 5, 2018 at 12:35 PM, Robert Dailey <rcdailey.li...@gmail.com> wrote:
> On Fri, Jan 5, 2018 at 2:26 PM, Paul Smith <p...@mad-scientist.net> wrote:
>> On Fri, 2018-01-05 at 12:12 -0800, Bryan Turner wrote:
>>> On Fri, Jan 5, 2018 at 11:59 AM, Robert Dailey <rcdailey.li...@gmail.com> 
>>> wrote:
>>>
>>> As for why the two aren't allowed together, my assumption would be
>>> because if you're only squashing a single commit "--squash" and that
>>> commit is fast-forward from the target, a new commit is not created
>>> and instead the target branch is fast-forwarded. With "--no-ff", it's
>>> questionable what "--squash" should do in that case. Fast-forward
>>> anyway? Rewrite the commit simply to get new committer details and
>>> SHA-1?
>>
>> If it only failed when you were squash-merging a single commit that was
>> also fast-forwardable, I guess that would be one thing.  But even if I
>> have multiple commits and I want to squash-merge them, which clearly is
>> a separate operation giving different results, I get this error.

I think there's a reasonable argument that having the failure be
consistent is easier to reason about, and therefore provides a
"better" user experience (to some definition of "better" which all
people may not share in common).

If the failure was delayed until "git merge --squash" decided it
wanted to fast-forward, the failure might seem more arbitrary.

>>
>> I don't think Git should try to be clever here (if that's what it's
>> doing--I always assumed it was just a missing configuration case in the
>> error check).  If I asked for a squash-merge then Git should give me a
>> squash merge.
>>
>> So in answer to your question, --squash should give me a squash merge
>> and the setting of --ff / --no-ff should be completely ignored, as it's
>> irrelevant.
>>
>> My $0.02.
>
> Seems like --ff works, but is also misleading since in my case (more
> than one commit) I'm not doing a ff merge and there's no possibility
> of it.

"--ff" doesn't say "git merge" _must_ fast-forward ("--ff-only"); it
says that it _can_. At a general level with "--squash", that seems to
be exactly correct. A "--squash" merge can create a new commit, or it
can fast-forward an existing commit if the situation allows. Based on
that, passing "--ff" doesn't seem misleading to me.

> I think your idea of the 2 being distinctly separate makes
> sense. Basically, --squash takes precedence and if the mechanism to
> implement squash in certain scenarios (such as single commit) is
> fast-forward merge, then that decision is made for the user and is no
> longer something they can control.

The two _aren't_ distinctly separate, though. "git merge --squash
--ff-only" has very different semantics to "git merge --squash --ff",
in that it will only create a new squashed commit (or fast-forward a
single commit) if the incoming commit(s) are fast-forward from the
target. So there _is_ a setting for the fast-forward mode (given
"--ff", "--ff-only", and "--no-ff" are a tri-state switch, and
therefore comprise a single setting) that does impact squashing.

Re: Can't squash merge with merge.ff set to false

[Bryan Turner]

On Fri, Jan 5, 2018 at 11:59 AM, Robert Dailey <rcdailey.li...@gmail.com> wrote:
> Not sure if this is intended or a bug, but with the following configuration:
>
> $ git config --global merge.ff false
>
> I am not able to merge my topic branch into master with squash option:
>
> $ git checkout master
> $ git merge --squash topic
> fatal: You cannot combine --squash with --no-ff.
>
> I'm not sure why a non-fast-forward merge would prevent a squash
> merge, since by its very nature a squashed merge is not a fast forward
> merge (or maybe it is if you only have one commit).
>
> Is there an issue here? I like fast forward merges to be off by
> default, since I want to control when they happen. Most of my merges
> do not use --squash, so I'm catering to the common case.
>
> Need advice on how to get past this issue. Thanks in advance.

The easiest way to move forward is probably to pass "--ff" on the
command line to override the config, when you're using "--squash".

As for why the two aren't allowed together, my assumption would be
because if you're only squashing a single commit "--squash" and that
commit is fast-forward from the target, a new commit is not created
and instead the target branch is fast-forwarded. With "--no-ff", it's
questionable what "--squash" should do in that case. Fast-forward
anyway? Rewrite the commit simply to get new committer details and
SHA-1?

Hope this helps!
Bryan Turner

Re: [ANNOUNCE] Git v2.16.0-rc0

[Bryan Turner]

On Tue, Jan 2, 2018 at 9:07 PM, Jonathan Nieder <jrnie...@gmail.com> wrote:
> Hi Bryan,
>
> Bryan Turner wrote:
>
>> Our test environment is still on Ubuntu 12.04 LTS (it's a long story,
>> but one I doubt is unique to us), which means it's using OpenSSH 5.9.
>> ssh -G was added in OpenSSH 6.8 [1], circa March 2015, which means the
>> "auto" detection "fails" and chooses "simple" instead of "ssh". But
>> OpenSSH 5.9 _does_ support -4, -6 and -p. As a result, commands which
>> have been working without issue on all previous versions of Git start
>> to fail saying
>>
>> git -c gc.auto=0 -c credential.helper= fetch --force --prune --progress 
>> ssh://localhost:64281/repo.git +refs/*:refs/*' exited with code 128 saying: 
>> fatal: ssh variant 'simple' does not support setting port
>
> Hm, that's not expected.  git-config(1) says:
>
> By default, Git determines the command line arguments to use
> based on the basename of the configured SSH command
> (configured using the environment variable GIT_SSH or
> GIT_SSH_COMMAND or the config setting core.sshCommand). If the
> basename is unrecognized, Git will attempt to detect support
> of OpenSSH options by [...]
>
> So my first question is why the basename detection is not working for
> you.  What value of GIT_SSH, GIT_SSH_COMMAND, or core.sshCommand are
> you using?

So I'd been digging further into this for the last hour because I
wasn't seeing quite the behavior I was expecting when I ran Git from
the command line on Ubuntu 12.04 or 14.04, and this nudged me to the
right answer: We're setting GIT_SSH to a wrapper script. In our case,
that wrapper script is just calling OpenSSH's ssh with all the
provided arguments (plus a couple extra ones), but because we're
setting GIT_SSH at all, that's why the auto variant code is running.
That being the case, explicitly setting GIT_SSH_VARIANT=ssh may be the
correct thing to do, to tell Git that we want to be treated like
"normal" OpenSSH, as opposed to expecting Git to assume we behave like
OpenSSH (when the Android repo use case clearly shows that assumption
also doesn't hold).

Based on that, I'm not sure if you _actually_ need to change anything,
and I apologize for not turning up that we were setting GIT_SSH before
I bothered you.

Thanks immensely for looking into it, Jonathan, and my apologies again
for wasting your time!
Bryan Turner

Re: [ANNOUNCE] Git v2.16.0-rc0

[Bryan Turner]

On Thu, Dec 28, 2017 at 8:30 PM, Junio C Hamano <gits...@pobox.com> wrote:
> An early preview release Git v2.16.0-rc0 is now available for
> testing at the usual places.  It is comprised of 435 non-merge
> commits since v2.15.0, contributed by 76 people, 22 of which are
> new faces.

> Brandon Williams (24):
>   ssh: introduce a 'simple' ssh variant

> Jonathan Nieder (10):
>   ssh test: make copy_ssh_wrapper_as clean up after itself
>   connect: move no_fork fallback to git_tcp_connect
>   connect: split git:// setup into a separate function
>   connect: split ssh command line options into separate function
>   connect: split ssh option computation to its own function
>   ssh: 'auto' variant to select between 'ssh' and 'simple'
>   ssh: 'simple' variant does not support -4/-6
>   ssh: 'simple' variant does not support --port
>   connect: correct style of C-style comment
>   generate-cmdlist: avoid non-deterministic output

Sorry for being late to the party on the "simple" variant for SSH, but
we've been doing some testing with 2.16.0-rc0 and noticed an
unexpected issue.

Our test environment is still on Ubuntu 12.04 LTS (it's a long story,
but one I doubt is unique to us), which means it's using OpenSSH 5.9.
ssh -G was added in OpenSSH 6.8 [1], circa March 2015, which means the
"auto" detection "fails" and chooses "simple" instead of "ssh". But
OpenSSH 5.9 _does_ support -4, -6 and -p. As a result, commands which
have been working without issue on all previous versions of Git start
to fail saying:

git -c gc.auto=0 -c credential.helper= fetch --force --prune
--progress ssh://localhost:64281/repo.git +refs/*:refs/*' exited with
code 128 saying: fatal: ssh variant 'simple' does not support setting
port

I know Ubuntu 12.04 LTS is end-of-life, but 14.04 LTS, which is
running OpenSSH 6.6 [2], has the same issue. The following is from a
fully patched 14.04.5:

bturner@ubuntu:~$ cat /etc/*ease | head -4
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.5 LTS"

bturner@ubuntu:~$ ssh -V
OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.8, OpenSSL 1.0.1f 6 Jan 2014

bturner@ubuntu:~$ ssh -G -p 7999 localhost
unknown option -- G
usage: ssh [-1246AaCfgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec]
   [-D [bind_address:]port] [-E log_file] [-e escape_char]
   [-F configfile] [-I pkcs11] [-i identity_file]
   [-L [bind_address:]port:host:hostport] [-l login_name] [-m mac_spec]
   [-O ctl_cmd] [-o option] [-p port]
   [-Q cipher | cipher-auth | mac | kex | key]
   [-R [bind_address:]port:host:hostport] [-S ctl_path] [-W host:port]
   [-w local_tun[:remote_tun]] [user@]hostname [command]

Is it possible to adjust the check, somehow, so it doesn't impact
older OpenSSH versions like this? As it stands, it seems likely a fair
number of users who have an SSH command that does support -4, -6 and
-p are going to end up getting "penalized" because it doesn't also
support -G, and have to manually set their SSH variant to "ssh" (or
something other than "auto") to avoid the automatic detection.

I'd love to say I have a brilliant idea for how to work around this,
oh and here's a patch, but I don't. One option might be trying to
actually review the output, and another might be to run "ssh -V", but
both of those have their own flaws (and the extra process forks aren't
"free").

[1] https://www.openssh.com/txt/release-6.8
[2] https://launchpad.net/ubuntu/+source/openssh

Best regards,
Bryan Turner

Re: Git Hooks

[Bryan Turner]

On Fri, Dec 15, 2017 at 11:12 AM, Satyakiran Duggina
<satya0...@gmail.com> wrote:
> I see that `git init` creates a .git directory and hooks are to be
> placed in that directory and these hooks are not tracked by version
> control. To achieve tracked hooks, either each developer has to copy
> the hooks or use tools like overcommit, pre-commit, husky etc.
>
> I'm wondering why hooks are not made external like .gitignore. I guess
> it would be better to have two git configuration directories in a
> repo, one hosting all the metadata managed by git and the other with
> user configured data (hooks, ignore/exclude, repo config etc).

Hooks are not external because they're not trusted. It essentially
amounts to allowing someone to download an arbitrary script or program
onto your computer which you then execute. It's extremely unsafe, and
is intentionally not possible. To get hooks in your instance, you have
to _manually_ install them. This gives you a chance to _review_ them
before they start executing on your system. Any other approach and the
hooks become an attack vector.

>
> Kindly let me know why the current design choice is made and if the
> proposed change would introduce unseen issues.
>
>
> Thanks,
> Satya

Hope this helps!
Bryan Turner

Re: why can *some* git commands not be run from within .git directory?

[Bryan Turner]

On Tue, Nov 14, 2017 at 1:18 AM, Robert P. J. Day  wrote:
>
>   just noticed something i was unaware of -- some git commands can't
> be run if i'm in the .git directory, while others can. for example,
> if i "cd .git", commands like this work just fine:
>
>   $ git show
>   $ git branch
>   $ git log
>
> but others seem unwilling to determine the "working tree":

Once Git finds a .git directory, or determines it's in one, it doesn't
look "higher"; instead, it runs like it's in a bare clone. That means
any command that requires a work tree or an index generally won't
work. Of course, you can still tell Git where the work tree is and
then the commands work fine from the .git directory:

incom@Jael MINGW64 /c/Temp/example/.git (GIT_DIR!)
$ GIT_WORK_TREE=.. git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working tree clean

>
>   $ git status
>   fatal: This operation must be run in a work tree
>   $
>
> and:
>
>   $ git stash list
>   fatal: /usr/libexec/git-core/git-stash cannot be used without a working 
> tree.
>   $
>
> what's the distinction between commands that can work this way, and
> commands that can't?
>
> rday
>
> p.s. i will refrain from pointing out the inconsistency in using the
> phrases "work tree" and "working tree" to mean the same thing, when
> there is a distinct "worktree" entity.

No you won't, clearly.

>
> --
>
> 
> Robert P. J. Day Ottawa, Ontario, CANADA
> 
> https://urldefense.proofpoint.com/v2/url?u=http-3A__crashcourse.ca=DwIBAg=wBUwXtM9sKhff6UeHOQgvw=uBedA6EFFVX1HiLgmpdrBrv8bIDAScKjk1yk9LOASBM=yPGW_MCaDCBECSMjz40m8vsxUhljCB3dnrQ4TBi8PtE=-JNZXqYtyY7CIJGy65WWa88Kq7BaelyajFehPQZkvo4=
>
> Twitter:   
> https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_rpjday=DwIBAg=wBUwXtM9sKhff6UeHOQgvw=uBedA6EFFVX1HiLgmpdrBrv8bIDAScKjk1yk9LOASBM=yPGW_MCaDCBECSMjz40m8vsxUhljCB3dnrQ4TBi8PtE=-9gIAxcdRGPmIz1rpLLb7nl14azEwoE-fOJ-IxAYi18=
> LinkedIn:   
> https://urldefense.proofpoint.com/v2/url?u=http-3A__ca.linkedin.com_in_rpjday=DwIBAg=wBUwXtM9sKhff6UeHOQgvw=uBedA6EFFVX1HiLgmpdrBrv8bIDAScKjk1yk9LOASBM=yPGW_MCaDCBECSMjz40m8vsxUhljCB3dnrQ4TBi8PtE=prnIhreU-Vhx0V6lRCBQEPbJk2P8bPlHcRFus1X_6wM=
> 

Re: [PATCH 0/6] Create Git/Packet.pm

[Bryan Turner]

On Thu, Oct 26, 2017 at 2:07 AM, Jacob Keller  wrote:
> On Wed, Oct 25, 2017 at 10:38 PM, Junio C Hamano  wrote:
>> Johannes Schindelin  writes:
>>
>>> Note that the correct blib path starts with `C:\BuildAgent\_work` and
>>> the line
>>>
>>>   use lib (split(/:/, $ENV{GITPERLLIB}));
>>>
>>> splits off the drive letter from the rest of the path. Obviously, this
>>> fails to Do The Right Thing, and simply points to Yet Another Portability
>>> Problem with Git's reliance on Unix scripting.
>>
>> In our C code, we have "#define PATH_SEP ';'", and encourage our
>> code to be careful and use it.  Is there something similar for Perl
>> scripts, I wonder.
>>
>> I notice that t/{t0202,t9000,t9700}/test.pl share the same
>> split(/:/, $ENV{GITPERLLIB}); forcing this specific variable to use
>> the non-platform convention to accomodate the use of split(/:/)
>> certainly is a workaround, but it does feel dirty.
>>
>> It is hard to imagine that we were the first people who wants to
>> split the value of a variable into a list, where the value is a list
>> of paths, concatenated into a single string with a delimiter that
>> may be platform specific.  I wonder if we are going against a best
>> practice established in the Perl world, simply because we don't know
>> about it (i.e. basically, it would say "don't split at a colon
>> because not all world is Unix; use $this_module instead", similar to
>> "don't split at a slash, use File::Spec instead to extract path
>> components").
>>
>
> I thought there was a way to do this in File::Spec, but that's only
> for splitting regular paths, and not for splitting a list of paths
> separated by ":" or ";"
>
> We probably should find a better solution to allow this to work with
> windows style paths...? I know that python provides os.pathsep, but I
> haven't seen an equivalent for perl yet.
>
> The Env[1] core modules suggests using $Config::Config{path_sep}[2]..
> maybe we should be using this?

I was testing this recently on the Perl included with Git for Windows
and it returns : for the path separator even though it's on Windows,
so I don't think that would work. The Perl in Git for Windows seems to
want UNIX-style inputs (something Dscho seemed to allude to in his
response earlier.). I'm not sure why it's that way, but he probably
knows.

Bryan

(Pardon my previous blank message; Gmail fail.)

>
> Thanks,
> Jake
>
> [1] https://perldoc.perl.org/Env.html
> [2] https://perldoc.perl.org/Config.html

Re: [PATCH 0/6] Create Git/Packet.pm

[Bryan Turner]

On Thu, Oct 26, 2017 at 2:07 AM, Jacob Keller  wrote:
> On Wed, Oct 25, 2017 at 10:38 PM, Junio C Hamano  wrote:
>> Johannes Schindelin  writes:
>>
>>> Note that the correct blib path starts with `C:\BuildAgent\_work` and
>>> the line
>>>
>>>   use lib (split(/:/, $ENV{GITPERLLIB}));
>>>
>>> splits off the drive letter from the rest of the path. Obviously, this
>>> fails to Do The Right Thing, and simply points to Yet Another Portability
>>> Problem with Git's reliance on Unix scripting.
>>
>> In our C code, we have "#define PATH_SEP ';'", and encourage our
>> code to be careful and use it.  Is there something similar for Perl
>> scripts, I wonder.
>>
>> I notice that t/{t0202,t9000,t9700}/test.pl share the same
>> split(/:/, $ENV{GITPERLLIB}); forcing this specific variable to use
>> the non-platform convention to accomodate the use of split(/:/)
>> certainly is a workaround, but it does feel dirty.
>>
>> It is hard to imagine that we were the first people who wants to
>> split the value of a variable into a list, where the value is a list
>> of paths, concatenated into a single string with a delimiter that
>> may be platform specific.  I wonder if we are going against a best
>> practice established in the Perl world, simply because we don't know
>> about it (i.e. basically, it would say "don't split at a colon
>> because not all world is Unix; use $this_module instead", similar to
>> "don't split at a slash, use File::Spec instead to extract path
>> components").
>>
>
> I thought there was a way to do this in File::Spec, but that's only
> for splitting regular paths, and not for splitting a list of paths
> separated by ":" or ";"
>
> We probably should find a better solution to allow this to work with
> windows style paths...? I know that python provides os.pathsep, but I
> haven't seen an equivalent for perl yet.
>
> The Env[1] core modules suggests using $Config::Config{path_sep}[2]..
> maybe we should be using this?
>
> Thanks,
> Jake
>
> [1] https://perldoc.perl.org/Env.html
> [2] https://perldoc.perl.org/Config.html

Re: Git for Windows v2.15.0-rc prerelease, was Re: [ANNOUNCE] Git v2.15.0-rc2

[Bryan Turner]

On Fri, Oct 20, 2017 at 3:22 PM, Johannes Schindelin
<johannes.schinde...@gmx.de> wrote:
> Hi team,
>
> [cutting linux-kernel]
>
> On Fri, 20 Oct 2017, Junio C Hamano wrote:
>
>> A release candidate Git v2.15.0-rc2 is now available for testing
>> at the usual places.
>
> The Git for Windows equivalent is now available from
>
> 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_git-2Dfor-2Dwindows_git_releases_tag_v2.15.0-2Drc2.windows.1=DwIBAg=wBUwXtM9sKhff6UeHOQgvw=uBedA6EFFVX1HiLgmpdrBrv8bIDAScKjk1yk9LOASBM=3DdPEJGQe01zvIuHjX8rNURKuGEY_cPkUXvnur9xlNg=ZC45D6NoNiE4A8qs_F1ZDMJlGMdXcQ9DwDIpE1-whrU=
>
> Please test at your convenience,

Thanks for publishing this, Johannes. I've run it through our Windows
test matrix for Bitbucket Server (~1450 tests) and all pass. I've also
added it to our CI build as a canary (pending the final 2.15.0
release). I've done the same for 2.15.0-rc2 on Linux as well.

Best regards,
Bryan Turner

Re: what is git's position on "classic" mac -only end of lines?

[Bryan Turner]

On Sun, Oct 1, 2017 at 10:52 AM, Robert P. J. Day <rpj...@crashcourse.ca> wrote:
>
>   sorry for more pedantic nitpickery, but i'm trying to write a
> section on how to properly process mixtures of EOLs in git, and when i
> read "man git-config", everything seems to refer to Mac OS X and macOS
> (and linux, of course) using  for EOL, with very little mention of
> what one does if faced with "classic" mac EOL of just .

 No command in Git that I'm aware of considers a standalone  to be
a line ending. A file containing only s is treated as a single
line by every Git command I've used. I'm not sure whether that
behavior is configurable. For files with standalone s mixed with
other line endings ( or , either or both), the  and
 endings are both considered line endings while the standalone
s are not.

That's just based on my experience with them, though. In general, `git
blame` and `git diff`, for example, don't seem honor them. Perhaps
someone else knows of some useful knows of which I'm not aware.

Best regards,
Bryan Turner

Re: git merge algorithm question

[Bryan Turner]

On Tue, Sep 5, 2017 at 5:53 PM, Daniel Biran  wrote:
>
>>> I'm trying to better understand one of the merge algorithms as I had some 
>>> triumphs and tribulations with using a set of commands during a merge. 
>>> tldr: can a git merge -s recursive -X patience; // result in a fast-forward 
>>> merge? will --no-ff stop it
>>>
>>> So, the scenario is this:
>>>  - Merging a master branch into a feature branch that is 2+ years old
>>>  - We found this command was more beneficial when merging a large 20k 
>>> line text file:
>>>  - git merge -s recursive -X patience master
>>>  - In a recent merge using this approach the reflog shows that the 
>>> merge was performed using a fast-forward from the feature branch's head
>>>  - 082517-1, feature/branch) HEAD@{23}: merge feature/branch: 
>>> Fast-forward
>>>
>>>
>>> My question is, is it possible for that command to use a fast-forward like 
>>> this? (or did something else go horribly wrong? possibly an atlassian git 
>>> GUI tool corrupting the work):
>>>  - If it is possible for the command to fast-forward the merge when 
>>> making the commit does --no-ff force the command to never use fast-forward 
>>> in this case

Unless you specify --no-ff, git merge is always free to create a
fast-forward "merge", even when you request the recursive strategy
explicitly:

$ git init recursive-merge
Initialized empty Git repository in C:/Temp/recursive-merge/.git/

$ cd recursive-merge/

$ echo "Test" > file.txt

$ git add file.txt

$ git commit -m "Initial commit"
[master (root-commit) ad48617] Initial commit
 1 file changed, 1 insertion(+)
 create mode 100644 file.txt

$ git checkout -b feature-branch
Switched to a new branch 'feature-branch'

$ echo "Edit" >> file.txt

$ git commit -am "Feature branch change"
[feature-branch b226557] Feature branch change
 1 file changed, 1 insertion(+)

$ git checkout master
Switched to branch 'master'

$ git merge -s recursive -X patience feature-branch
Updating ad48617..b226557
Fast-forward
 file.txt | 1 +
 1 file changed, 1 insertion(+)

With --no-ff:

$ git reset --hard ad48617
HEAD is now at ad48617 Initial commit

$ git merge --no-ff -s recursive -X patience feature-branch
Merge made by the 'recursive' strategy.
 file.txt | 1 +
 1 file changed, 1 insertion(+)

With fast-forwarding disabled, you can see the recursive strategy is
used as requested.

>>>
>>> Thanks for the help,
>>> Daniel
>>
>

Re: [RFC 0/7] transitioning to protocol v2

[Bryan Turner]

On Wed, Aug 30, 2017 at 2:12 PM, Brandon Williams <bmw...@google.com> wrote:
> On 08/30, Bryan Turner wrote:
>> On Fri, Aug 25, 2017 at 10:29 AM, Jeff King <p...@peff.net> wrote:
>> > On Thu, Aug 24, 2017 at 03:53:21PM -0700, Brandon Williams wrote:
>> >
>> >> The biggest question I'm trying to answer is if these are reasonable ways 
>> >> with
>> >> which to communicate a request to a server to use a newer protocol, 
>> >> without
>> >> breaking current servers/clients.  As far as I've tested, with patches 1-5
>> >> applied I can still communicate with current servers without causing any
>> >> problems.
>> >
>> > Current git.git servers, I assume?. How much do we want to care about
>> > alternate implementations? I would not be surprised if other git://
>> > implementations are more picky about cruft after the virtual-host field
>> > (though I double-checked GitHub's implementation at least, and it is
>> > fine).
>> >
>> > I don't think libgit2 implements the server side. That leaves probably
>> > JGit, Microsoft's VSTS (which I think is custom), and whatever Atlassian
>> > and GitLab use.
>>
>> Before I manually apply the patches to test how they work with
>> Bitbucket Server, are they applied on a branch somewhere where I can
>> just fetch them? If not, I'll apply them manually and verify.
>
> I just pushed this set of patches up to: 
> https://github.com/bmwill/git/tree/protocol-v2
> so you should be able to fetch them from there (saves you from having to
> manually applying the patches).

Thanks for that! It made testing a lot simpler.

>
>> Just based on the description, though, I expect no issues. We don't
>> currently support the git:// protocol. Our HTTP handling passes
>> headers through to the receive-pack and upload-pack processes as
>> environment variables (with a little massaging), but doesn't consider
>> them itself; it only considers the URL and "service" query parameter
>> to decide what command to run and to detect "dumb" requests. Our SSH
>> handling ignores any environment variables provided and does not
>> forward them to the git process, similar to VSTS.
>>
>> I'll confirm explicitly, to be certain, but just based on reading the
>> overview and knowing our code I think the described approaches should
>> work fine.
>
> Perfect!  Thanks for taking the time to verify that this will work.

With 2 small tweaks on top of "protocol-v2", I was able to run our
full command and hosting (HTTP and SSH) test suites without issues.

diff --git a/transport.c b/transport.c
index c05e167..37b5d83 100644
--- a/transport.c
+++ b/transport.c
@@ -222,7 +222,8 @@ static struct ref *get_refs_via_connect(struct
transport *transport, int for_pus
switch(determine_version(data->fd[0], )) {
case 2:
/* The server understands Protocol v2 */
-   fprintf(stderr, "Server understands Protocol v2!\n");
+   if (transport->verbose >= 0)
+   fprintf(stderr, "Server understands Protocol v2!\n");
break;
case 1:
/* Server is speaking Protocol v1 and sent a ref so
process it */
diff --git a/upload-pack.c b/upload-pack.c
index 0f85315..7c59495 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -1075,7 +1075,7 @@ int cmd_main(int argc, const char **argv)
git_config(upload_pack_config, NULL);

version = getenv("GIT_PROTOCOL");
-   if (!strcmp(version, "2"))
+   if (version && !strcmp(version, "2"))
upload_pack_v2();

upload_pack();

I'd imagine the "Server understands Protocol v2!" message won't
actually *ship* in Git, so it's likely making that honor "--quiet"
doesn't actually matter; I only adjusted it because we have a test
that verifies "git clone --quiet" is quiet. The "if (version" change I
commented on separately in the 7/7 patch that introduced the check.

Thanks again for publishing this, and for letting me test it!

>
> --
> Brandon Williams

Re: [RFC 7/7] upload-pack: ack version 2

[Bryan Turner]

On Thu, Aug 24, 2017 at 3:53 PM, Brandon Williams  wrote:
> +
> +   version = getenv("GIT_PROTOCOL");
> +   if (!strcmp(version, "2"))
> +   upload_pack_v2();
> +

I think the "if" check here needs some type of NULL check for
"version" before calling "strcmp". Without that, if the "GIT_PROTOCOL"
environment variable isn't set, git-upload-pack SEGVs.

This came up when I was testing the "protocol-v2" branch with
Bitbucket Server. For performance reasons, we sometimes run ref
advertisements as a separate process, when serving fetches, to allow
throttling the ref advertisement separately from actually generating
and sending a packfile. Since CI servers continuously poll for
changes, but usually don't find any, we want to be able to serve ref
advertisements, which have minimal overhead, with much higher
parallelism than serving packs. To do that, we run "git-upload-pack
--advertize-refs" directly, and that command has never *required*
"GIT_PROTOCOL" before this change.

Re: [RFC 0/7] transitioning to protocol v2

[Bryan Turner]

On Fri, Aug 25, 2017 at 10:29 AM, Jeff King <p...@peff.net> wrote:
> On Thu, Aug 24, 2017 at 03:53:21PM -0700, Brandon Williams wrote:
>
>> The biggest question I'm trying to answer is if these are reasonable ways 
>> with
>> which to communicate a request to a server to use a newer protocol, without
>> breaking current servers/clients.  As far as I've tested, with patches 1-5
>> applied I can still communicate with current servers without causing any
>> problems.
>
> Current git.git servers, I assume?. How much do we want to care about
> alternate implementations? I would not be surprised if other git://
> implementations are more picky about cruft after the virtual-host field
> (though I double-checked GitHub's implementation at least, and it is
> fine).
>
> I don't think libgit2 implements the server side. That leaves probably
> JGit, Microsoft's VSTS (which I think is custom), and whatever Atlassian
> and GitLab use.

Before I manually apply the patches to test how they work with
Bitbucket Server, are they applied on a branch somewhere where I can
just fetch them? If not, I'll apply them manually and verify.

Just based on the description, though, I expect no issues. We don't
currently support the git:// protocol. Our HTTP handling passes
headers through to the receive-pack and upload-pack processes as
environment variables (with a little massaging), but doesn't consider
them itself; it only considers the URL and "service" query parameter
to decide what command to run and to detect "dumb" requests. Our SSH
handling ignores any environment variables provided and does not
forward them to the git process, similar to VSTS.

I'll confirm explicitly, to be certain, but just based on reading the
overview and knowing our code I think the described approaches should
work fine.

Best regards,
Bryan Turner

Re: upgarding GIT

[Bryan Turner]

On Mon, Aug 7, 2017 at 4:20 PM, James Wells <james.we...@statseeker.com> wrote:
>
> As you can see my version of git is not supported with the current version of 
> bitbucket. I will have to perform a two stage upgrade anyway as the version 
> of STASH I am running cannot be upgraded directly to bitbucket 5.2 as well.
>
> Is there an easy way just to install a higher support version of git like 2.9 
> on the same server and then move all the repos and basically everything 
> across. Can you install another TAR ball later version on top of another git 
> , so it's like overwriting it.

Hey James, Bitbucket Server developer here. Sorry for not responding
sooner. I just wanted to check in with you and see if you'd gotten
this all resolved? If not, feel free to message me directly; I'm happy
to help you off the list. (Or on list, for that matter; I just figure
the general Git list isn't interested in this issue.)

Best regards,
Bryan Turner

Re: Binary files

[Bryan Turner]

On Thu, Jul 20, 2017 at 12:41 AM, Volodymyr Sendetskyi
<volodymy...@devcom.com> wrote:
> It is known, that git handles badly storing binary files in its
> repositories at all.
> This is especially about large files: even without any changes to
> these files, their copies are snapshotted on each commit. So even
> repositories with a small amount of code can grove very fast in size
> if they contain some great binary files. Alongside this, the SVN is
> much better about that, because it make changes to the server version
> of file only if some changes were done.
>
> So the question is: why not implementing some feature, that would
> somehow handle this problem?

Like Git LFS or git annex? Features have been implemented to better
handle large files; they're just not necessarily part of core Git.
Have you checked whether one of those solutions might work for your
use case?

Best regards,
Bryan Turner

Re: Flurries of 'git reflog expire'

[Bryan Turner]

On Mon, Jul 10, 2017 at 9:45 PM, Andreas Krey <a.k...@gmx.de> wrote:
> On Thu, 06 Jul 2017 10:01:05 +0000, Bryan Turner wrote:
> 
>> I also want to add that Bitbucket Server 5.x includes totally
>> rewritten GC handling. 5.0.x automatically disables auto GC in all
>> repositories and manages it explicitly, and 5.1.x fully removes use of
>> "git gc" in favor of running relevant plumbing commands directly.
>
> That's the part that irks me. This shouldn't be necessary - git itself
> should make sure auto GC isn't run in parallel. Now I probably can't
> evaluate whether a git upgrade would fix this, but given that you
> are going the do-gc-ourselves route I suppose it wouldn't.
>

I believe I've seen some commits on the mailing list that suggest "git
gc --auto" manages its concurrency better in newer versions than it
used to, but even then it can only manage its concurrency within a
single repository. For a hosting server with thousands, or tens of
thousands, of active repositories, there still wouldn't be any
protection against "git gc --auto" running concurrently in dozens of
them at the same time.

But it's not only about concurrency. "git gc" (and by extension "git
gc --auto") is a general purpose tool, designed to generally do what
you need, and to mostly stay out of your way while it does it. I'd
hazard to say it's not really designed for managing heavily-trafficked
repositories on busy hosting services, though, and as a result, there
are things it can't do.

For example, I can configure auto GC to run based on how many loose
objects or packs I have, but there's no heuristic to make it repack
refs when I have a lot of loose ones, or configure it to _only_ pack
refs without repacking objects or pruning reflogs. There are knobs for
various things (like "gc.*.reflogExpire"), but those don't give
complete control. Even if I set "gc.reflogExpire=never", "git gc"
still forks "git reflog expire --all" (compared to
"gc.packRefs=false", which completely prevents forking "git
pack-refs").

A trace on "git gc" shows this:
$ GIT_TRACE=1 git gc
00:10:45.058066 git.c:437   trace: built-in: git 'gc'
00:10:45.067075 run-command.c:369   trace: run_command:
'pack-refs' '--all' '--prune'
00:10:45.077086 git.c:437   trace: built-in: git
'pack-refs' '--all' '--prune'
00:10:45.084098 run-command.c:369   trace: run_command: 'reflog'
'expire' '--all'
00:10:45.093102 git.c:437   trace: built-in: git 'reflog'
'expire' '--all'
00:10:45.097088 run-command.c:369   trace: run_command: 'repack'
'-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago'
00:10:45.106096 git.c:437   trace: built-in: git 'repack'
'-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago'
00:10:45.107098 run-command.c:369   trace: run_command:
'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty'
'--all' '--reflog' '--indexed-objects'
'--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset'
'objects/pack/.tmp-15212-pack'
00:10:45.127117 git.c:437   trace: built-in: git
'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty'
'--all' '--reflog' '--indexed-objects'
'--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset'
'objects/pack/.tmp-15212-pack'
Counting objects: 6, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (6/6), done.
Total 6 (delta 0), reused 6 (delta 0)
00:10:45.173161 run-command.c:369   trace: run_command: 'prune'
'--expire' '2.weeks.ago'
00:10:45.184171 git.c:437   trace: built-in: git 'prune'
'--expire' '2.weeks.ago'
00:10:45.199202 run-command.c:369   trace: run_command: 'worktree'
'prune' '--expire' '3.months.ago'
00:10:45.208193 git.c:437   trace: built-in: git
'worktree' 'prune' '--expire' '3.months.ago'
00:10:45.212198 run-command.c:369   trace: run_command: 'rerere' 'gc'
00:10:45.221223 git.c:437   trace: built-in: git 'rerere' 'gc'

The bare repositories used by Bitbucket Server:
* Don't have reflogs enabled generally, and for the ones that are
enabled "gc.*.reflogExpire" is set to "never"
* Never have worktrees, so they don't need to be pruned
* Never use rerere, so that doesn't need to GC
* Have pruning disabled if they've been forked, due to using
alternates to manage disk space

That means of all the commands "git gc" runs, under the covers, at
most only "pack-refs", "repack" and sometimes "prune" have any value.
"reflog expire --all" in particular is extremely likely to fail. Which
brings up another consideration.

"git gc --auto" has no sense of context, or adjacent behavior. Even if
it correctly guards against concurrency, it still doesn't know what
else

Re: name-rev: anchor pattern without --exclude?

[Bryan Turner]

On Thu, Jul 6, 2017 at 10:23 AM, Kyle Meyer  wrote:
> Junio C Hamano  writes:
>> Kyle Meyer  writes:
>
>> What is the answer desired by your application when two or more
>> branches point at the same commit you are interested in?  Pick one at
>> random?  An error saying it cannot decide where to place the snapshot?
>
> Our use of name-rev is just to get a friendly name for a hash.  If two
> branches point to the same commit, we're fine with whatever decision
> "git name-rev" makes; we just want to limit it to refs in the
> "refs/heads/" namespace.

If you don't need the ancestor traversals "git name-rev" provides,
"git for-each-ref --count 1 --format "%(refname:short)" --points-at
 refs/heads/" might work. That only goes back to Git 2.7.0,
though; still quite a ways off your 1.9 target. ("git branch
--points-at" does the same thing, I should add, and only for branches,
but you can't directly limit its output like you can with
"for-each-ref".. Perhaps that doesn't matter for your use case.) If
you want names like "master~2", from your example, though,
"--points-at" won't do what you need.

Bryan

>
> --
> Kyle

Re: Flurries of 'git reflog expire'

[Bryan Turner]

I'm one of the Bitbucket Server developers. My apologies; I just
noticed this thread or I would have jumped in sooner!

On Thu, Jul 6, 2017 at 6:31 AM, Andreas Krey  wrote:
> On Wed, 05 Jul 2017 04:20:27 +, Jeff King wrote:
>> On Tue, Jul 04, 2017 at 09:57:58AM +0200, Andreas Krey wrote:
> ...
>> And what does the process tree look like?
>
> Lots (~ 10) of
>
>   \_ /usr/bin/git receive-pack 
> /opt/apps/atlassian/bitbucket-data/shared/data/repositories/68
>   |   \_ git gc --auto --quiet
>   |   \_ git reflog expire --all
>
> plus another dozen gc/expire pairs where the parent is already gone.
> All with the same arguments - auto GC.

Do you know what version of Bitbucket Server is in use? Based on the
fact that it's "git gc --auto" triggered from a "git receive-pack",
that implies two things:
- You're on a 4.x version of Bitbucket Server
- The repository (68) has never been forked

Depending on your Bitbucket Server version (this being the reason I
asked), there are a couple different fixes available:

- Fork the repository. You don't need to _use_ the fork, but having a
fork existing will trigger Bitbucket Server to disable auto GC and
fully manage that itself. That includes managing both _concurrency_
and _frequency_ of GC. This works on all versions of Bitbucket Server.

- Run "git config gc.auto 0" in
/opt/apps/atlassian/bitbucket-data/shared/data/repositories/68 to
disable auto GC yourself. This may be preferable to forking the
repository, which, in addition to disabling auto GC, also disables
object pruning. However, you must be running at least Bitbucket Server
4.6.0 for this approach to work. Otherwise auto GC will simply be
reenabled the first time Bitbucket Server goes to trigger GC, when it
detects that the repository has no forks.

Assuming you're on 4.6.0 or newer, either approach should fix the
issue. If you're on 4.5 or older, forking is the only viable approach
unless you upgrade Bitbucket Server first.

I also want to add that Bitbucket Server 5.x includes totally
rewritten GC handling. 5.0.x automatically disables auto GC in all
repositories and manages it explicitly, and 5.1.x fully removes use of
"git gc" in favor of running relevant plumbing commands directly. We
moved away from "git gc" specifically to avoid the "git reflog expire
--all", because there's no config setting that _fully disables_
forking that process. By default our bare clones only have reflogs for
pull request refs, and we've explicitly configured those to never
expire, so all "git reflog expire --all" can do is use up I/O and,
quite frequently, fail because refs are updated. Since we stopped
running "git gc", we've not yet seen any GC failures on our internal
Bitbucket Server clusters.

Bitbucket Server 5.1.x also includes a new "gc.log" (not to be
confused with the one Git itself writes) which retains a record of
every GC-related process we run in each repository, and how long that
process took to complete. That can be useful for getting clearer
insight into both how often GC work is being done, and how long it's
taking.

Upgrading to 5.x can be a bit of an undertaking, since the major
version brings API changes, so it's totally understandable that many
organizations haven't upgraded yet. I'm just noting that these
improvements are there for when such an upgrade becomes viable.

Hope this helps!
Bryan

>
> I'd wager that each push sees that a GC is in order,
> and doesn't notice that there is one already running.
>
> - Andreas
>
> --
> "Totally trivial. Famous last words."
> From: Linus Torvalds 
> Date: Fri, 22 Jan 2010 07:29:21 -0800

Re: [BUG] Failed to sign commit

[Bryan Turner]

> $ GIT_TRACE=1 git commit --allow-empty -v -m "lol"
> 11:37:24.594795 git.c:369   trace: built-in: git 'commit'
> '--allow-empty' '-v' '-m' 'lol'
> 11:37:24.605842 run-command.c:369   trace: run_command: 'gpg'
> '--status-fd=2' '-bsau' '8AEC0DB537A9FC7E'
> error: gpg failed to sign the data
> fatal: failed to write commit object
>
> It seems more a gpg problem no? something not well configured after
> the update perhaps?

Have you tried running `export GPG_TTY=$(tty)` before running `git
commit`? I had a similar gpg error that was solved by that.

Re: Git smart http: parsing commit messages in git-receive-pack

[Bryan Turner]

On Sat, May 6, 2017 at 5:30 AM, akos tajti <akos.ta...@gmail.com> wrote:
> Dear All,
>
> we implemented a java servlet around the git-http-backend. This servlet 
> intercepts the requests sent by the git client when pushing. One thing I want 
> to achieve is parsing the commit messages in the pre push phase (request 
> param service==git-receive-pack) and after checking if the commit messages 
> contain a given string reject/accept the request. The problem is that I 
> couldn't find a way of parsing the commit messages in git-receive-pack (I can 
> do this in the post  push phase but in that case I cannot reject the push 
> because the changes are already on the server). Is there a way of getting the 
> commit messages before they're actually on the server so that I can reject 
> the push?

The protocol isn't really intended for use to try and parse pack data
before it's spooled to disk. It might be possible, but it's likely to
be brittle (due to things like delta compression, for example).

I believe what you're looking for is a pre-receive hook[1] on your
server. This is how Bitbucket Server (which is also written in Java)
works. It opens a socket on localhost and installs a pre-receive hook
into each repository which uses a Perl script to pipe all of its input
to the listening socket and then read any response from the server
back. After the push has written all of its objects to disk, Git
invokes any pre-receive hooks and passes in all the ref changes. The
Perl script pipes those to the server, which applies whatever checks
are desired and writes any errors or info back to the hook, which then
prints it for Git to use before exiting with the correct status code
(0 to accept the push, 1 to reject it).

This means the objects are on the server, but in Git 2.11 Peff added
"quarantine" functionality which writes those new objects into an
"incoming" directory[2]. That means, while they're on disk, they're
_not_ in the repository itself. If the pre-receive hook rejects the
push, those objects are immediately deleted from disk.

That means if you pair a pre-receive hook with Git 2.11 or newer on
the server, you should be able to achieve what you're looking for.
Once the objects have been written to the quarantine directory, you
can use normal Git commands, like cat-file, rev-list or log, to review
them. If they don't meet your requirements, just reject the push and
Git will delete the objects automatically

Hope this helps!
Bryan Turner

>
> Thanks in advance,
> Ákos Tajti
>

[1]: https://git-scm.com/docs/githooks includes documentation for
pre-receive inputs and outputs
[2]: 
https://github.com/git/git/blob/master/Documentation/git-receive-pack.txt#L219-L243
includes some additional documentation about the quarantine
environment

Re: Git checkout issue - deleting file without apparent reason

[Bryan Turner]

On Tue, May 2, 2017 at 6:33 PM, Paul van Wichen
 wrote:
> Hi,
>
> We are having a strange issue that we haven't been able to pin down.
> Scenario: master branch and feature branch both have a specific file.
> 1. Master checked out.
> 2. git status show no changes / clean staging area.
> 3. Checkout feature branch .
> 4. git status show no changes / clean staging area.
> 5. Checkout master again.
> 6. git status shows that a file has been deleted (i.e. the file was
> removed from the file system and the staging area shows it as
> deleted).
>
> The file exists in both the feature branch and the master branch. As
> best as we can tell, the file is identical on both commits.
> The issue occurs on multiple platforms - tested on Windows and OS X.
> It only occurs for 1 specific file.
>
> Based on the activity of the file, nothing stands out as unusual.
>
> How we go about troubleshooting this and determining the cause?
>
> Thanks,
> Paul van Wichen.

Re: [RFC] - url-safe base64 commit-id's

[Bryan Turner]

On Mon, Feb 27, 2017 at 6:27 PM, G. Sylvie Davies
 wrote:
> Is there any appetite for base64'd commit-id's, using the url-safe
> variant (e.g. RFC 4648 [1] with padding removed)?
>
> And so this:
> 712bad335dfa9c410a83f9873614a19726acb3a8
>
> Becomes this:
> cSutM136nEEKg_mHNhShlyass6g
>
>
> Under the hood things cannot change (e.g., ".git/objects/71/") because
> file systems are not always case sensitive.
>
> But for "git log" and "git show" output it would be nice. And helps
> with ambiguous commit id's too if you only want to specify a 7
> character commit-id, since that encodes 42 bits instead of 28 bits.
> I've run into problems with maximum command length on windows (32767
> chars) because I was specifying so many commit-ids on the command-line
> that I blew past that limit. This would help there, too.

Depending on the command, have you considered using stdin instead? git log,
for example, is perfectly happy to read commit IDs from stdin instead of
the command line.

In general, I think the pattern of getting away from command line arguments
is better than trying to shoehorn more data into the same character limit.
Base64 encoding might help get a few more arguments into the available
limit, but in the end it's not going to solve the underlying problem.

>
> Might be particularly helpful with the transition to a new hash.
> e.g., a 43 character Base64 id instead of a 64 character hex id.
>
>
> - Sylvie
>
> [1] - https://tools.ietf.org/html/rfc4648#page-7

Re: Races on ref .lock files?

[Bryan Turner]

Andreas,

Bitbucket Server developer here. Typically these errors on your client
are indicative of git gc --auto being triggered by git-receive-pack on
the server. Auto GC directly attached to a push in a repository with
pull requests often fails due to concurrent ref updates linked to
background pull request processing.

If you'd like to investigate more in depth, I'd encourage you to
create a ticket on support.atlassian.com so we can work with you.
Otherwise, if you just want to prevent seeing these messages, you can
either fork the relevant repository in Bitbucket Server (which
disables auto GC), or run "git config gc.auto 0" in
/opt/apps/.../repositories/68. Once auto GC is disabled, Bitbucket
Server will automatically take over managing GC for the repository
without any additional configuration required.

Note that we're working on revamping our GC handling such that auto GC
will always be disabled for all repositories and managed explicitly
within Bitbucket Server instead, so a future upgrade should
automatically prevent these messages from appearing on clients.

Best regards,
Bryan Turner

On Fri, Dec 16, 2016 at 9:20 AM, Junio C Hamano <gits...@pobox.com> wrote:
> Andreas Krey <a.k...@gmx.de> writes:
>
>> We're occasionally seeing a lot of
>>
>>   error: cannot lock ref 'stash-refs/pull-requests/18978/to': Unable to 
>> create 
>> '/opt/apps//repositories/68/stash-refs/pull-requests/18978/to.lock': 
>> File exists.
>>
>> from the server side with fetches as well as pushes. (Bitbucket server.)
>>
>> What I find strange is that neither the fetches nor the pushes even
>> touch these refs (but the bitbucket triggers underneath might).
>>
>> But my question is whether there are race conditions that can cause
>> such messages in regular operation - they continue with 'If no other git
>> process is currently running, this probably means a git process crashed
>> in this repository earlier.' which indicates some level of anticipation.
>
> I think (and I think you also think) these messages come from the
> Bitbucket side, not your "git push" (or "git fetch").  Not having
> seen Bitbucket's sources, I can only guess, but assuming that its
> pull-request is triggered from their Web frontend like GitHub's
> does, it is quite possible when you try to "push" into (or "fetch"
> from, for that matter) a repository, somebody is clicking a button
> to create that ref.  We do not know what their "receive-pack" that
> responds to your "git push" (or "upload-pack" for "git fetch") does
> when there are locked refs.  I'd naively think that unless you are
> pushing to that ref you showed an error message for, the receiving
> end shouldn't care if the ref is being written by somebody else, but
> who knows ;-) They may have their own reasons wanting to lock that
> ref that we think would be irrelevant for the operation, causing
> errors.
>
>
>

Re: [PATCH 07/18] link_alt_odb_entry: handle normalize_path errors

[Bryan Turner]

>
>> Is there anything I can do to help? I'm happy to test out changes.
>
> The patch at the end of his mail obviously passes the newly-added tests
> for me, but please confirm that it fixes your test suite.
>
> I gather your suite is about noticing behavior changes between different
> versions. For cases where we know there is an obvious right behavior, it
> would be nice if you could contribute them as patches to git's test
> suite. This case was overlooked because there was no test coverage at
> all.
>
> Barring that, running your suite and giving easily-reproducible problem
> reports is valuable. The earlier the better. So I am happy to see this
> on -rc0, and not on the final release. Periodically running it on
> "master" during the development cycle would have caught it even sooner.

I've applied your patch to the tip of the 2.11.0-rc0 tag (just to make
sure I don't accidentally pick up anything else on master; I'll test
that separately) and my full test suite passes without issue.

I'm going to investigate whether I can setup a version of this build
that runs "periodically" (I'm not sure what that period will be)
against git/git master. I've got a lot of the infrastructure in place,
but I'm going to need to automate a few things to make it really
viable.

As for contributing extensions to the test suite, that's a good idea.
I need to fast track getting a development environment setup.

Re: [PATCH 07/18] link_alt_odb_entry: handle normalize_path errors

[Bryan Turner]

On Mon, Nov 7, 2016 at 4:30 PM, Jeff King <p...@peff.net> wrote:
> On Mon, Nov 07, 2016 at 03:42:35PM -0800, Bryan Turner wrote:
>
>> > @@ -335,7 +340,9 @@ static void link_alt_odb_entries(const char *alt, int 
>> > len, int sep,
>> > }
>> >
>> > strbuf_add_absolute_path(, get_object_directory());
>> > -   normalize_path_copy(objdirbuf.buf, objdirbuf.buf);
>> > +   if (strbuf_normalize_path() < 0)
>> > +   die("unable to normalize object directory: %s",
>> > +   objdirbuf.buf);
>>
>> This appears to break the ability to use a relative alternate via an
>> environment variable, since normalize_path_copy_len is explicitly
>> documented "Returns failure (non-zero) if a ".." component appears as
>> first path"
>
> That shouldn't happen, though, because the path we are normalizing has
> been converted to an absolute path via strbuf_add_absolute_path. IOW, if
> your relative path is "../../../foo", we should be feeding something
> like "/path/to/repo/.git/objects/../../../foo" and normalizing that to
> "/path/to/foo".
>
> But in your example, you see:
>
>   error: unable to normalize alternate object path: ../0/objects
>
> which cannot come from the code above, which calls die(). It should be
> coming from the call in link_alt_odb_entry().

Ah, of course. I should have looked more closely at the call.



> No, I had no intention of disallowing relative alternates (and as you
> noticed, a commit from the same series actually expands the use of
> relative alternates). My use has been entirely within info/alternates
> files, though, not via the environment.
>
> As I said, I'm not sure this was ever meant to work, but as far as I can
> tell it mostly _has_ worked, modulo some quirks. So I think we should
> consider it a regression for it to stop working in v2.11.
>
> The obvious solution is one of:
>
>   1. Stop calling normalize() at all when we do not have a relative base
>  and the path is not absolute. This restores the original quirky
>  behavior (plus makes the "foo/../../bar" case work).
>
>  If we want to do the minimum before releasing v2.11, it would be
>  that. I'm not sure it leaves things in a very sane state, but at
>  least v2.11 does no harm, and anybody who cares can build saner
>  semantics for v2.12.
>
>   2. Fix it for real. Pass a real relative_base when linking from the
>  environment. The question is: what is the correct relative base? I
>  suppose "getcwd() at the time we prepare the alt odb" is
>  reasonable, and would behave similarly to older versions ($GIT_DIR
>  for bare repos, top of the working tree otherwise).
>
>  If we were designing from scratch, I think saner semantics would
>  probably be always relative from $GIT_DIR, or even always relative
>  from the object directory (i.e., behave as if the paths were given
>  in objects/info/alternates). But that breaks compatibility with
>  older versions. If we are treating this as a regression, it is not
>  very friendly to say "you are still broken, but you might just need
>  to add an extra '..' to your path".
>
> So I dunno. I guess that inclines me towards (1), as it lets us punt on
> the harder question.

Is there anything I can do to help? I'm happy to test out changes.
I've got a set of ~1,040 tests that verify all sorts of different Git
behaviors (those tests flagged this change, for example, and found a
regression in git diff-tree in 2.0.2/2.0.3, among other things). I run
them on the "newest" patch release for every feature-bearing line of
Git from 1.8.x up to 2.10 (currently 1.8.0.3, 1.8.1.5, 1.8.2.3,
1.8.3.4, 1.8.4.5, 1.8.5.6, 1.9.5, 2.0.5, 2.1.4, 2.2.3, 2.3.10, 2.4.11,
2.5.5, 2.6.6, 2.7.4, 2.8.4, 2.9.3 and 2.10.2), and I add in RCs of new
as soon as they become available. (I also test Git for Windows; at the
moment I've got 1.8.0, 1.8.1.2, 1.8.3, 1.8.4, 1.8.5.2 and 1.9.5.1 from
msysgit and 2.3.7.1, 2.4.6, 2.5.3, 2.6.4, 2.7.4, 2.8.4, 2.9.3 and
2.10.2 from Git for Windows. 2.11.0-rc0 on Windows passes my test
suite; it looks like it's not tagging the same git/git commit as
v2.11.0-rc0 is.) I wish there was an easy way for me to open this up.
At the moment, it's something I can really only run in-house, as it
were.

At the moment I have limited ability to actually try to submit patches
myself. I really need to sit down and setup a working development
environment for Git. (My current dream, if I could get such an
environment running, is to follow up on your git blame-tree work.

>
> -Peff

Re: [PATCH 07/18] link_alt_odb_entry: handle normalize_path errors

[Bryan Turner]

On Mon, Oct 3, 2016 at 1:34 PM, Jeff King <p...@peff.net> wrote:
> When we add a new alternate to the list, we try to normalize
> out any redundant "..", etc. However, we do not look at the
> return value of normalize_path_copy(), and will happily
> continue with a path that could not be normalized. Worse,
> the normalizing process is done in-place, so we are left
> with whatever half-finished working state the normalizing
> function was in.
>



> @@ -335,7 +340,9 @@ static void link_alt_odb_entries(const char *alt, int 
> len, int sep,
> }
>
> strbuf_add_absolute_path(, get_object_directory());
> -   normalize_path_copy(objdirbuf.buf, objdirbuf.buf);
> +   if (strbuf_normalize_path() < 0)
> +   die("unable to normalize object directory: %s",
> +   objdirbuf.buf);

This appears to break the ability to use a relative alternate via an
environment variable, since normalize_path_copy_len is explicitly
documented "Returns failure (non-zero) if a ".." component appears as
first path"

For example, when trying to run a rev-list over commits in two
repositories using GIT_ALTERNATE_OBJECT_DIRECTORIES, in 2.10.x and
prior the following command works. I know the alternate worked
previously because I'm passing a commit that does not exist in the
repository I'm running the command in; it only exists in a repository
linked by alternate, as shown by the "fatal: bad object" when the
alternates are rejected.

Before, using Git 2.7.4 (but I've verified this behavior through to
and including 2.10.2):

bturner@elysoun /tmp/1478561282706-0/shared/data/repositories/3 $
GIT_ALTERNATE_OBJECT_DIRECTORIES=../0/objects:../1/objects git
rev-list --format="%H" 2d8897c9ac29ce42c3442cf80ac977057045e7f6
74de5497dfca9731e455d60552f9a8906e5dc1ac
^6053a1eaa1c009dd11092d09a72f3c41af1b59ad
^017caf31eca7c46eb3d1800fcac431cfa7147a01 --
commit 74de5497dfca9731e455d60552f9a8906e5dc1ac
74de5497dfca9731e455d60552f9a8906e5dc1ac
commit 3528cf690cb37f6adb85b7bd40cc7a6118d4b598
3528cf690cb37f6adb85b7bd40cc7a6118d4b598
commit 2d8897c9ac29ce42c3442cf80ac977057045e7f6
2d8897c9ac29ce42c3442cf80ac977057045e7f6
commit 9c05f43f859375e392d90d23a13717c16d0fdcda
9c05f43f859375e392d90d23a13717c16d0fdcda

Now, using Git 2.11.0-rc0

bturner@elysoun /tmp/1478561282706-0/shared/data/repositories/3 $
GIT_ALTERNATE_OBJECT_DIRECTORIES=../0/objects:../1/objects
/opt/git/2.11.0-rc0/bin/git rev-list --format="%H"
2d8897c9ac29ce42c3442cf80ac977057045e7f6
74de5497dfca9731e455d60552f9a8906e5dc1ac
^6053a1eaa1c009dd11092d09a72f3c41af1b59ad
^017caf31eca7c46eb3d1800fcac431cfa7147a01 --
error: unable to normalize alternate object path: ../0/objects
error: unable to normalize alternate object path: ../1/objects
fatal: bad object 74de5497dfca9731e455d60552f9a8906e5dc1ac

Other commits, like [1], suggest the ability to use relative paths in
alternates is something still actively developed and enhanced. Is it
intentional that this breaks the ability to use relative alternates?
If this is to be the "new normal", is there any other option when
using environment variables besides using absolute paths?

Best regards,
Bryan Turner

[1]: https://github.com/git/git/commit/087b6d584062f5b704356286d6445bcc84d686fb
-- Also newly tagged in 2.11.0-rc0

Re: v2.9.3 and v2.10.0: `name-ref' HEAD gives wrong branch name

[Bryan Turner]

On Tue, Sep 20, 2016 at 9:23 AM, Steffen Nurpmeso  wrote:
> Hello again,
>
> yah, sorry, i'm back again..
> I try to find a way to find the name of the current branch in an
> automated way, because i need to ensure that a commit happens on
> it and no other branch.  Now the problem arises that the commit
> ref at the time of that commit maybe shared in between several
> different branches, but no more thereafter, of course:
>
>   ?0[steffen@wales ]$ git branch|grep '^*'
>   * stable/v14.9
>   ?0[steffen@wales ]$ git name-rev --name-only HEAD
>   stable/v14.8
>
> Is there another way except looking into .git/HEAD or using sed(1)
> on the output of `branch' to find the right name?

Have you tried "git symbolic-ref HEAD"?

$ git symbolic-ref HEAD
refs/heads/master

If you don't want the fully-qualified ref, you can add --short:

$ git symbolic-ref --short HEAD
master

> Thank you.
> Ciao!
>
> --steffen

Re: --abbrev-commit gives longer hash than necessary

[Bryan Turner]

Steffen,

Git commands generally have a 7 character minimum by default when
abbreviating hashes, even if fewer characters are still (currently)
unique. Per the documentation:

   core.abbrev
   Set the length object names are abbreviated to. If unspecified,
   many commands abbreviate to 7 hexdigits, which may not be enough
   for abbreviated object names to stay unique for sufficiently long
   time.

You may be able to set core.abbrev to a smaller value to get even
shorter hashes, but the shorter you go the more likely you are to find
once-unique hashes no longer are, as your repository grows over time.
The default isn't just about what's likely to be unique now; it's
about what's likely to stay unique for a period of time.

Hope this helps!
Bryan Turner

On Thu, Jun 30, 2016 at 12:38 PM, Steffen Nurpmeso <stef...@sdaoden.eu> wrote:
> Hello, for your possible interest.
>
> For some time (currently with 2.9.0) know see that a single commit
> gives a longer hash than necessary, even though there is no
> ambiguity:
>
>   ?0[steffen@wales ]$ git longca|
>   awk 'BEGIN{l7=0;l8=0}\
> /^[[:alnum:]]{7} /{++l7;next}\
> /^[[:alnum:]]{8} /{++l8;print}\
>   END{print "L7 " l7 " L8 " l8}'
>   786d0c9c [mimepipe.2] send.c:sendpart(): force iconv(3)+ for TEXT part 
> handlers..
>   L7 3364 L8 1
>
> So it is only this single commit.. but why?
>
>   ?0[steffen@wales ]$ git long1 786d0c9
>   786d0c9c [mimepipe.2] send.c:sendpart(): force iconv(3)+ for TEXT part 
> handlers..
>   ?0[steffen@wales ]$ git long1 786d0c
>   786d0c9c [mimepipe.2] send.c:sendpart(): force iconv(3)+ for TEXT part 
> handlers..
>   ?0[steffen@wales ]$ git long1 786d0
>   786d0c9c [mimepipe.2] send.c:sendpart(): force iconv(3)+ for TEXT part 
> handlers..
>   ?0[steffen@wales ]$ git long1 786d
>   786d0c9c [mimepipe.2] send.c:sendpart(): force iconv(3)+ for TEXT part 
> handlers..
>
> Not really ambiguous:
>
>   ?0[steffen@wales ]$ git long|cut -f1 -d' '|grep ^786
>   786d0c9c
>   786f219
>
> Ciao!
>
> --steffen
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: credentials are being erased from credential manager via git credential-wincred erase command

[Bryan Turner]

Ken,

I'm one of the developers for Atlassian Bitbucket Server (formerly
Stash), which I believe you're running.

>From the credentials code in Git (which I could be mis-reading; I'm
sure others on the list will correct me if I'm wrong), it appears the
erase is done after a cached credential is rejected by the server.
That implies that, periodically, authentication with your Stash server
is failing and that that failed authentication results in Git clearing
the "bad" credential. That's likely why you see this happen on
seemingly random builds.

To follow up on the possibility that authentication with Stash is
periodically failing, I'd recommend opening a support request at
support.atlassian.com. My belief is that the remote Git client is
doing what it's intended to do in response to an authentication
failure. That suggests we need to look at your Stash instance to
determine why authentication is failing. If you do open a support
request, please mention me in your description so that our support
engineers can attach me to the issue!

Best regards,
Bryan Turner

On Wed, Apr 13, 2016 at 12:49 PM, O'Connell, Ken
<Ken.O'conn...@cognex.com> wrote:
> Good afternoon,
>
> My company setup wincred for management of our git/stash user credentials for 
> our automated build scripts. It works great for days, sometimes a couple 
> weeks.
> Then one day build haven't run and we discover the scripts are randomly found 
> at a user prompt awaiting stash user credentials.
>
> We look at Windows Credential Manager store in Windows and find the Stash 
> user credentials are erased from Credential Manager!
>
> We discovered via GIT_TRACE, that the command used to retrieve the 
> credentials, is being followed up by a git command to erase the credentials. 
> -not all the time, but seemingly in a random way.
> Looking at the trace logs on one server, we see the following commands:
> 'git-remote-https' 'origin' https://stash-server/bla/bla/bla.git
> 'git-credential-wincred' 'get'
> 'git credential-wincred erase'
>
> Do you have ideas on how to track down the root cause of why this command is 
> running?
>
> Info:
> Windows 7 machines
> Git version 1.9.5-msysgit, and Git 2.7.4 windows (on one machine to see if 
> upgrading helped) -it did not.
> Stash version 3.11.2
>
> Please let me know if I can get any more information to help diagnose.
> Thanks,
> -Ken
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Git clone sends first an empty authorization header

[Bryan Turner]

On Fri, Mar 4, 2016 at 9:51 PM, Guilherme  wrote:
> Hi,
>
> When doing basic authentication using git clone by passing the
> username and password in the url git clone will first send a GET
> request without the authorization header set.
>
> Am i seeing this right?

I believe this is an intentional behavior in either cURL or how Git
uses it. Credentials aren't sent until the server returns a challenge
for them, even if you include them in your clone URL or elsewhere. So
yes, you're seeing it right.

>
> This means that if the counterpart allows anonymous cloning but not
> pushing and the user provided a wrong usernam/password, it has two
> options:

I'm not sure why this would be true. If the remote server allows
anonymous clone/fetch, then you never get prompted for credentials
and, even if they're supplied, they're never sent to the remote
server. If you then try to push, if the server is working correctly it
should detect that anonymous users can't push and it should return a
401 with a WWW-Authenticate header. When the client receives the 401,
it should send the credentials it has (or prompt for them if it
doesn't have them) and the push should work without issue.

Perhaps there's an issue with how your server is setup to handle permissions?

>
> 1. Allow the access and leave the user to figure out why he is not able to 
> push.
>
> 2. Reply by setting the WWW-Authentication header and see if a
> password/username is provided. This has the downside that if no
> username and password is provided the user will still get a login
> prompt for password and username. Upon entering twice nothing he will
> still be able to clone. This can be confusing.
>
> Can this behaviour of git clone (and I guess all the other parts that
> do basic auth) be changed to provide the authentication header right
> on the first request? Or am I doing/interpreting it wrong?
>
> Thank you.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Git for Windows crashes on Windows 10

[Bryan Turner]

On Tue, Feb 2, 2016 at 10:28 PM, 孙乾程  wrote:
> I'm not a native English speaker. I'm sorry if I didn't explain the problem 
> clearly.
> I'm using Windows 10 Enterprise Insider Preview (I'll use Win10 instead 
> below). Until yesterday, I'm using Win10 build 11102, and Git for Windows 
> works well. But today I upgraded my system to Win10 build 14251, then 
> something wrong happened.

I'm pretty sure you're running into
https://github.com/git-for-windows/git/issues/627. Since it's a bug in
Windows itself, there's not a fix in Git. Your options are:

- Downgrade back to 11102
- Wait for the next preview build after 14251
- Downgrade to msysgit 1.9.5

> I opened a "cmd" in my repository and entered "git pull", then a dialog 
> window appeared, saying "Git for Windows has stopped working".
> The version of the git on my system was 2.6.4, and I found out that the 
> latest version was 2.7.0.2. So I downloaded a installer from 
> "https://git-scm.com/;. I opened the installer, but as soon as the 
> installation finished, the same error dialog appeared. I closed the dialog, 
> but it appeared again, so I closed it again. I opened a "cmd" and entered 
> "git", of course, it crashes and the error dialog appeared.
> By the way, I installed GitHub for Windows on my system and it crashes too 
> when I pressed the "Sync" button in it.
> In my surprise, GitHub Extension for Visual Studio works correctly.
>
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: bug report : 2.3.5 ssh repo not found

[Bryan Turner]

On Sat, Apr 25, 2015 at 3:57 PM, Chris chrisb.d...@gmail.com wrote:
 Hello,

 Using git version 2.3.5 with kernel 3.19.3-3-ARCH #1 SMP PREEMPT x86_64 I
 see the following error message when pulling or cloning a repo over ssh:

 
 git clone ssh://user@mydomain:/home/user/path/to/project.git
 Cloning into 'project'...
 ssh: Could not resolve hostname mydomain:: Name or service not known
 fatal: Could not read from remote repository.

 Please make sure you have the correct access rights
 and the repository exists.
 

I believe that's [1]

Remove the : between the domain and the path and it should work, per
Jeff King [2]

[1] http://thread.gmane.org/gmane.comp.version-control.git/266649
[2] http://article.gmane.org/gmane.comp.version-control.git/266659


 Obviously I changed the url to hide credential info

 After ensuring DNS was OK and being able to ssh to that instance directly I
 tried downgrading git to my distro's last installed version of git version
 2.2.2 and now I can clone / pull / push to/from that repo without issue.

 Is this a bug?

 Best,
 Chris


 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] Git v2.4.0-rc2

[Bryan Turner]

On Wed, Apr 15, 2015 at 7:48 AM, Junio C Hamano gits...@pobox.com wrote:

 A release candidate Git v2.4.0-rc2 is now available for testing at
 the usual places.  The difference since -rc1 is mostly l10n and a
 handful of documentation clean-ups.

 The tarballs are found at:

 https://www.kernel.org/pub/software/scm/git/testing/

 The following public repositories all have a copy of the 'v2.4.0-rc2'
 tag and the 'master' branch that the tag points at:

   url = https://kernel.googlesource.com/pub/scm/git/git
   url = git://repo.or.cz/alt-git.git
   url = https://code.google.com/p/git-core/
   url = git://git.sourceforge.jp/gitroot/git-core/git.git
   url = git://git-core.git.sourceforge.net/gitroot/git-core/git-core
   url = https://github.com/gitster/git

 

 Git 2.4 Release Notes (draft)
 =

 Backward compatibility warning(s)
 -

 This release has a few changes in the user-visible output from
 Porcelain commands. These are not meant to be parsed by scripts, but
 the users still may want to be aware of the changes:

  * Output from git log --decorate (and %d format specifier used in
the userformat --format=string parameter git log family of
command takes) used to list HEAD just like other tips of branch
names, separated with a comma in between.  E.g.

  $ git log --decorate -1 master
  commit bdb0f6788fa5e3cacc4315e9ff318a27b2676ff4 (HEAD, master)
  ...

This release updates the output slightly when HEAD refers to the tip
of a branch whose name is also shown in the output.  The above is
shown as:

  $ git log --decorate -1 master
  commit bdb0f6788fa5e3cacc4315e9ff318a27b2676ff4 (HEAD - master)
  ...

  * The phrasing git branch uses to describe a detached HEAD has been
updated to match that of git status:

 - When the HEAD is at the same commit as it was originally
   detached, they now both show detached at commit object name.

 - When the HEAD has moved since it was originally detached,
   they now both show detached from commit object name.

 Earlier git branch always used from


 Updates since v2.3
 --

 Ports

  * Our default I/O size (8 MiB) for large files was too large for some
platforms with smaller SSIZE_MAX, leading to read(2)/write(2)
failures.

  * We did not check the curl library version before using
CURLOPT_PROXYAUTH feature that may not exist.

  * We now detect number of CPUs on older BSD-derived systems.

  * Portability fixes and workarounds for shell scripts have been added
to help BSD-derived systems.


 UI, Workflows  Features

  * The command usage info strings given by git cmd -h and in
documentation have been tweaked for consistency.

  * The sync subcommand of git p4 now allows users to exclude
subdirectories like its clone subcommand does.

  * git log --invert-grep --grep=WIP will show only commits that do
not have the string WIP in their messages.

  * git push has been taught a --atomic option that makes push to
update more than one ref an all-or-none affair.

  * Extending the push to deploy added in 2.3, the behaviour of git
push when updating the branch that is checked out can now be
tweaked by push-to-checkout hook.

  * Using environment variable LANGUAGE and friends on the client side,
HTTP-based transports now send Accept-Language when making requests.

  * git send-email used to accept a mistaken y (or yes) as an
answer to What encoding do you want to use [UTF-8]?  without
questioning.  Now it asks for confirmation when the answer looks
too short to be a valid encoding name.

  * When git apply --whitespace=fix fixed whitespace errors in the
common context lines, the command reports that it did so.

  * git status now allows the -v to be given twice to show the
differences that are left in the working tree not to be committed.

  * git cherry-pick used to clean-up the log message even when it is
merely replaying an existing commit.  It now replays the message
verbatim unless you are editing the message of resulting commits.

  * git archive can now be told to set the 'text' attribute in the
resulting zip archive.

  * Output from git log --decorate mentions HEAD when it points at a
tip of an branch differently from a detached HEAD.

This is a potentially backward-incompatible change.

  * git branch on a detached HEAD always said (detached from xyz),
even when git status would report detached at xyz.  The HEAD is
actually at xyz and haven't been moved since it was detached in
such a case, but the user cannot read what the current value of
HEAD is when detached from is used.

  * git -C '' subcmd refused to work in the current directory, unlike
cd '' which silently behaves as a no-op.
(merge 6a536e2 kn/git-cd-to-empty later to maint).

  * The 

Re: folder naming bug?

[Bryan Turner]

Are you, by any chance, on MacOS? HFS+ by default is
case-insensitive-but-case-preserving, and Git on MacOS by default runs
with core.ignorecase = true as a result.

If you set that to false does it change the behavior?

Hope this helps,
Bryan Turner

On Tue, Feb 3, 2015 at 12:56 PM, Kevin Coleman
kevin.cole...@sparkstart.io wrote:
 git isn’t tracking folder renames when the case of the letters change, but it 
 will track it if the folder changes names.  Is this intentional?

 Here is an example:

 08:51:26 ~/test $ git init
 Initialized empty Git repository in /Users/kcoleman/test/.git/
 08:51:29 ~/test (master #) $ mkdir main
 08:51:44 ~/test (master #) $ cd main
 08:51:46 ~/test/main (master #) $ touch readme.md
 08:51:50 ~/test/main (master #) $ ls
 readme.md
 08:51:53 ~/test/main (master #) $ cd ..
 08:51:54 ~/test (master #) $ git add .
 08:51:59 ~/test (master #) $ git commit -m one
 [master (root-commit) b0fddf6] one
  1 file changed, 0 insertions(+), 0 deletions(-)
  create mode 100644 main/readme.md
 08:52:04 ~/test (master) $ cd main
 08:52:14 ~/test/main (master) $ cd ..
 08:52:27 ~/test (master) $ mv main Main
 08:53:51 ~/test (master) $ git status
 On branch master
 nothing to commit, working directory clean
 08:53:53 ~/test (master) $ ls
 Main
 08:54:02 ~/test (master) $ mv Main MainA
 08:55:44 ~/test (master *) $ git status
 On branch master
 Changes not staged for commit:
   (use git add/rm file... to update what will be committed)
   (use git checkout -- file... to discard changes in working directory)

 deleted:main/readme.md

 Untracked files:
   (use git add file... to include in what will be committed)

 MainA/

 no changes added to commit (use git add and/or git commit -a)
 08:55:45 ~/test (master *) $--
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Big repo not shrinking on repack or gc?

[Bryan Turner]

On Thu, Jan 15, 2015 at 6:43 PM, Andreas Krey a.k...@gmx.de wrote:
 On Thu, 15 Jan 2015 18:05:46 +, Bryan Turner wrote:
 ...

 They do. So it seems it was forked once upon a time, but...

 /opt/apps/atlassian/stash-data/shared/data/repositories $ grep '' 
 */objects/info/alternates
 158/objects/info/alternates:/data/opt_apps/atlassian/stash-data/shared/data/repositories/20/objects
 45/objects/info/alternates:/data/opt_apps/atlassian/stash-data/shared/data/repositories/33/objects
 93/objects/info/alternates:/data/opt_apps/atlassian/stash-data/shared/data/repositories/91/objects

 ...there is no trace of a fork still existing (the repo in question is 143).

Yes, the system doesn't currently detect when a repository becomes
un-forked because it's not a common use case.

At this point I think we should probably take this off-list. You can
either e-mail me directly (bturner at atlassian dot com), or, better
still, raise a ticket on support.atlassian.com. Either way I'll work
with you directly to un-fork the repository on disk and allow it to
clean itself up.


 Andreas

 --
 Totally trivial. Famous last words.
 From: Linus Torvalds torvalds@*.org
 Date: Fri, 22 Jan 2010 07:29:21 -0800
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Big repo not shrinking on repack or gc?

[Bryan Turner]

On Thu, Jan 15, 2015 at 4:24 AM, Junio C Hamano gits...@pobox.com wrote:
 Andreas Krey a.k...@gmx.de writes:

 On Wed, 14 Jan 2015 07:49:36 +, Jeff King wrote:
 ...
 You don't need the -f here. Just git repack -ad should be enough
 (and the -f probably makes it _way_ slower).

 Indeed, factor four.

 However, my expectation is that a repack -ad will remove all the
 old pack files, as what is in there is either referenced and put
 into the new pack, or dropped = there should be a single pack file
 afterwards.

 This is not the case. :-( (Done only with 1.8.2 due to
 lack of compilers for this box.)

 Guess in the dark: ls -l .git/objects/pack
 Do you see any .keep files?

I'm one of the Stash developers and just noticed this thread. If the
repository in question has been forked via Stash there likely _will_
be .keep files. Stash uses alternates for forks, so it's possible, by
deleting those kept packs and pruning objects (which you've already
done I see) that you will corrupt, or have already corrupted, some
number of the forks. (At the moment Stash packs garbage into a dead
pack which it flags with a .keep, to ensure forks don't lose access
to objects that once existed upstream that they still reference.)

 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Big repo not shrinking on repack or gc?

[Bryan Turner]

On Thu, Jan 15, 2015 at 5:38 PM, Andreas Krey a.k...@gmx.de wrote:
 On Thu, 15 Jan 2015 12:23:00 +, Bryan Turner wrote:
 ...
  Guess in the dark: ls -l .git/objects/pack
  Do you see any .keep files?

 Lots of. :-(

 I'm one of the Stash developers and just noticed this thread. If the
 repository in question has been forked via Stash there likely _will_
 be .keep files. Stash uses alternates for forks, so it's possible, by
 deleting those kept packs and pruning objects (which you've already
 done I see) that you will corrupt, or have already corrupted, some
 number of the forks.

 There are a few forks in this stash instance, but the repository in
 question is neither the source nor the destination of any.

 So, git seems to be mostly out of the equation now (gc and repack
 apparently doing what they are supposed to do), and the question
 moves to 'how can stash let such a repo grow to that size'.


 (At the moment Stash packs garbage into a dead
 pack which it flags with a .keep, to ensure forks don't lose access
 to objects that once existed upstream that they still reference.)

 Does it do so in any case even if there is no actual fork? That would
 explain a lot - we are daily (force-)pushing new commit in there (and
 potentially big ones) that become garbage the next day, and should
 be cleaned up rather fast.

No, Stash will only do that in a repository which has been forked. In
any non-forked repository, Stash does not interact with garbage
collection in any way. Auto GC is left enabled, and all pruning
settings are left at their defaults. The default pruning interval is
two weeks, so if your development approach is rebase-heavy you may
need to adjust them.

What are the contents of some of those .keep files? If they're written
by Stash they contain a message saying so. (GENERATED BY ATLASSIAN
STASH - DO NOT REMOVE)


 (We're pulling them into another non-stash repo for longer-term keeping -
 these are backups of dev repos in the form of git stash commits including
 untracked files.)

 Andreas

 --
 Totally trivial. Famous last words.
 From: Linus Torvalds torvalds@*.org
 Date: Fri, 22 Jan 2010 07:29:21 -0800
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git status / git diff -C not detecting file copy

[Bryan Turner]

On Tue, Dec 2, 2014 at 5:55 PM, Jeff King p...@peff.net wrote:

 So from these timings, I'd conclude that:

   1. It's probably fine to turn on copies for git status.

   2. It's probably even OK to use -C -C for some projects. Even though
  22s looks scary there, that's only 11ms for git.git (remember,
  spread across 2000 commits). For linux.git, it's much, much worse.
  I killed my -C -C run after 10 minutes, and it had only gone
  through 1/20th of the commits. Extrapolating, you're looking at
  500ms or so added to a git status run.

  So you'd almost certainly want this to be configurable.

 Does either of you want to try your hand at a patch? Just enabling
 copies should be a one-liner. Making it configurable is more involved,
 but should also be pretty straightforward.

I'm interested in taking a stab at a patch, but I'd like to confirm
which way to go. Based on Junio's reply I'm not certain the simple
patch could get accepted (assuming I do all the submission parts
properly and the implementation itself passes review). Does that mean
the only real option is the configurable patch?


 -Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http-protocol question

[Bryan Turner]

In Documentation/technical/http-protocol.txt, there is the following statement:

S: Parse the git-upload-pack request:

Verify all objects in `want` are directly reachable from refs.

The server MAY walk backwards through history or through
the reflog to permit slightly stale requests.

Is there actually logic somewhere in Git that does that MAY walk
backwards step? Or is that something another implementation of Git
may do but the standard Git does not?

Thanks,
Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: http-protocol question

[Bryan Turner]

On Tue, Dec 2, 2014 at 2:34 PM, Jonathan Nieder jrnie...@gmail.com wrote:
 Hi Bryan,

 Bryan Turner wrote:

 Is there actually logic somewhere in Git that does that MAY walk
 backwards step?

 Yes.  See upload-pack.c::check_non_tip and
 http://thread.gmane.org/gmane.comp.version-control.git/178814.

Jonathan,

Thanks for the reply! I realize now I didn't really cite the part of
that documentation that I'm most interested in, which is: through
history _or through the reflog_. It's the reflog bit I'm looking for.
Sorry for not being more clear. check_non_tip appears to look at
ancestry, walking back up (or down, depending on your view) the DAG to
see if the requested SHA-1 is reachable from a current ref, so it
looks like that's covering the through history part of that blurb.

The reason I ask is that there is a race condition that exists where
the ref advertisement lists refs/heads/foo at abc1234, and then foo is
deleted before the actual upload-pack request comes in. In that
situation, walking backwards through _history_ will only allow the
upload-pack to complete unless the deleted ref was merged into another
ref prior to being deleted (if I understand the code correctly). It
seems like looking at the reflogs, and seeing that refs/heads/foo did
in fact exist and did in fact refer to abc1234, is the only approach
that could allow the upload-pack to complete. The documentation hints
that such a thing is possible, but I don't see any code in Git that
seems to do that.

Does that make more sense? Does that functionality exist and I've just
overlooked it?

Thanks again,
Bryan Turner


 Hope that helps,
 Jonathan
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: http-protocol question

[Bryan Turner]

On Tue, Dec 2, 2014 at 3:28 PM, Bryan Turner btur...@atlassian.com wrote:
 On Tue, Dec 2, 2014 at 2:34 PM, Jonathan Nieder jrnie...@gmail.com wrote:
 Hi Bryan,

 Bryan Turner wrote:

 Is there actually logic somewhere in Git that does that MAY walk
 backwards step?

 Yes.  See upload-pack.c::check_non_tip and
 http://thread.gmane.org/gmane.comp.version-control.git/178814.

 Jonathan,

 Thanks for the reply! I realize now I didn't really cite the part of
 that documentation that I'm most interested in, which is: through
 history _or through the reflog_. It's the reflog bit I'm looking for.
 Sorry for not being more clear. check_non_tip appears to look at
 ancestry, walking back up (or down, depending on your view) the DAG to
 see if the requested SHA-1 is reachable from a current ref, so it
 looks like that's covering the through history part of that blurb.

 The reason I ask is that there is a race condition that exists where
 the ref advertisement lists refs/heads/foo at abc1234, and then foo is
 deleted before the actual upload-pack request comes in. In that
 situation, walking backwards through _history_ will only allow the
 upload-pack to complete unless the deleted ref was merged into another

s/unless/if, sorry. In that situation, walking backwards through
history will only allow the upload-pack to complete if the deleted ref
was merged into another ref.

 ref prior to being deleted (if I understand the code correctly). It
 seems like looking at the reflogs, and seeing that refs/heads/foo did
 in fact exist and did in fact refer to abc1234, is the only approach
 that could allow the upload-pack to complete. The documentation hints
 that such a thing is possible, but I don't see any code in Git that
 seems to do that.

 Does that make more sense? Does that functionality exist and I've just
 overlooked it?

 Thanks again,
 Bryan Turner


 Hope that helps,
 Jonathan
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: http-protocol question

[Bryan Turner]

On Tue, Dec 2, 2014 at 3:45 PM, Jonathan Nieder jrnie...@gmail.com wrote:
 Hi,

 Bryan Turner wrote:

 The reason I ask is that there is a race condition that exists where
 the ref advertisement lists refs/heads/foo at abc1234, and then foo is
 deleted before the actual upload-pack request comes in.

 Can you say more about the context?  For example, was this actually
 happening, or is this for the sake of understanding the protocol
 better?

I ask because it's actually happening. Heavy CI load sometimes has
builds fail because git clone dies with not our ref. That's the
specific context I'm working to try and address right now. Some teams
use rebase-heavy workflows, which also evades the check_non_tip easing
and fails with not our ref, so I can't be 100% certain it's ref
deletion in specific causing it (but I guess which of those it is is
probably largely academic; as long as hosting spans multiple requests
it seems like this type of race condition is unavoidable).

I'm trying to decide if there is something I can enable or tune to
prevent it, and the type of twilighting hinted at by the http-protocol
documentation seemed like it could be just the thing.


 I ask because knowing the context might help us give more specific
 advice.

 Sometimes when people delete a branch they really want those objects
 to be inaccessible *right away*.  So for such people, git's behavior
 of failing the request unless the objects are still accessible by
 some other path is helpful.

 A stateful server could theoretically cache the list of objects they
 have advertised for a short time, to avoid clients having to suffer
 when something becomes inaccessible during the window between the
 upload-pack advertisement and upload-pack request.  Or a permissive
 server can allow all wants except a specific blacklist (and some
 people do that).

 Jonathan
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: http-protocol question

[Bryan Turner]

On Tue, Dec 2, 2014 at 4:33 PM, Jeff King p...@peff.net wrote:
 On Tue, Dec 02, 2014 at 04:04:11PM +1100, Bryan Turner wrote:

  Can you say more about the context?  For example, was this actually
  happening, or is this for the sake of understanding the protocol
  better?

 I ask because it's actually happening. Heavy CI load sometimes has
 builds fail because git clone dies with not our ref. That's the
 specific context I'm working to try and address right now. Some teams
 use rebase-heavy workflows, which also evades the check_non_tip easing
 and fails with not our ref, so I can't be 100% certain it's ref
 deletion in specific causing it (but I guess which of those it is is
 probably largely academic; as long as hosting spans multiple requests
 it seems like this type of race condition is unavoidable).

 There is a practical reason to care. Ref deletion will also delete the
 reflog, leaving no trace of the reachability. Whereas a non-fast-forward
 push could be resolved by looking in the reflog.

A fair point. I had mistakenly thought that reflogs would survive the
ref's deletion and be pruned as part of garbage collection, but a
quick test shows that, as I'm sure you already know, that's not true.

Thanks for correcting my mistake!
Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git status / git diff -C not detecting file copy

[Bryan Turner]

Pol,

By default, -C only finds copies when the source file was also
modified in the same commit. Since you did not modify hello.txt in the
same commit where you copied it to copied.txt, it will not be
considered.

If you pass -C -C (twice), or use --find-copies-harder, Git will
consider all files in the repository. Note that this can be slower,
which is the reason why it's not the default.

The documentation for git diff describes the -C (--find-copies) and
--find-copies-harder flags and their limitations.

Hope this helps,
Bryan Turner

On Sun, Nov 30, 2014 at 11:35 AM, Pol Online i...@pol-online.net wrote:
 Hi,

 The documentation for git status at http://git-scm.com/docs/git-status
 implies that it should be able to detect both renames and copies (with
 the R and C states). The command git diff -C should do it as well.

 However I can't get either to detect copies in this simple test case -
 what is happening?


 mkdir test
 cd test/
 git init
 echo 'Hello World!'  hello.txt
 echo 'Goodbye World!'  goodbye.txt
 git add -A
 git commit -m Initial commit

 cp hello.txt copied.txt
 mv goodbye.txt moved.txt
 git add -A

 $ git status --short
 A  copied.txt   NO COPY DETECTED
 R  goodbye.txt - moved.txt

 $ git diff -M -C --summary --cached
  create mode 100644 copied.txt   NO COPY DETECTED
  rename goodbye.txt = moved.txt (100%)

 $ git commit -m Test
 $ git diff -M -C --summary HEAD~
   create mode 100644 copied.txt   NO COPY DETECTED
   rename goodbye.txt = moved.txt (100%)


 -Pol
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git status / git diff -C not detecting file copy

[Bryan Turner]

Pol,

It's the same set of limitations. Git does not track renames or copies
as such. It uses heuristics to compute a similarity index and then
decide. All the tree stores for your copy is a file add, and that's
the status you're seeing. I don't think there is any way to turn on
aggressive copy detection for git status. However, before you run git
commit, you could run git diff --find-copies-harder --cached instead
and it should show the copy.

I'll let someone a little more intimately familiar with the internals
of git status comment on why the documentation for that mentions
copies.

Hope this helps,
Bryan Turner

On Sun, Nov 30, 2014 at 12:30 PM, Pol Online i...@pol-online.net wrote:
 Hi Bryan,

 OK that explains the behavior of git diff, but what about git status?
 The doc implies it should be able to detect copies in the index /
 staging area since it has a C state.

 - Pol

 On Sun, Nov 30, 2014 at 10:03 AM, Bryan Turner btur...@atlassian.com wrote:
 Pol,

 By default, -C only finds copies when the source file was also
 modified in the same commit. Since you did not modify hello.txt in the
 same commit where you copied it to copied.txt, it will not be
 considered.

 If you pass -C -C (twice), or use --find-copies-harder, Git will
 consider all files in the repository. Note that this can be slower,
 which is the reason why it's not the default.

 The documentation for git diff describes the -C (--find-copies) and
 --find-copies-harder flags and their limitations.

 Hope this helps,
 Bryan Turner

 On Sun, Nov 30, 2014 at 11:35 AM, Pol Online i...@pol-online.net wrote:
 Hi,

 The documentation for git status at http://git-scm.com/docs/git-status
 implies that it should be able to detect both renames and copies (with
 the R and C states). The command git diff -C should do it as well.

 However I can't get either to detect copies in this simple test case -
 what is happening?


 mkdir test
 cd test/
 git init
 echo 'Hello World!'  hello.txt
 echo 'Goodbye World!'  goodbye.txt
 git add -A
 git commit -m Initial commit

 cp hello.txt copied.txt
 mv goodbye.txt moved.txt
 git add -A

 $ git status --short
 A  copied.txt   NO COPY DETECTED
 R  goodbye.txt - moved.txt

 $ git diff -M -C --summary --cached
  create mode 100644 copied.txt   NO COPY DETECTED
  rename goodbye.txt = moved.txt (100%)

 $ git commit -m Test
 $ git diff -M -C --summary HEAD~
   create mode 100644 copied.txt   NO COPY DETECTED
   rename goodbye.txt = moved.txt (100%)


 -Pol
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

2.2.0-rc behavior changes (1/2)

[Bryan Turner]

I've been running a test suite we use to verify Git behaviors across
versions, and the 2.2.0 RCs (0 and 1 both) have a couple of small
behavioral differences. I'm sending them in separate e-mails just to
make the contents easier to grok.

Important: It's entirely possible neither of these is a _bug_; they
may both be intentional changes in behavior.

First change: git update-ref -d /refs/heads/nonexistent
some-valid-sha1 now produces an error about ref locking that it
didn't produce before

Git 2.1.x and prior produced this output:
error: unable to resolve reference refs/heads/nonexistent: No such
file or directory

Now, in the 2.2.0 RCs, it says:
error: unable to resolve reference refs/heads/nonexistent: No such
file or directory
error: Cannot lock the ref 'refs/heads/nonexistent'.

This one feels more like a bug, but again may not be. I say it feels
like a bug because of the order of the messages: If git has decided
the ref doesn't exist, why is it still trying to lock it?

This change bisects to:

bturner@felurian:~/Development/git/git$ git bisect bad
7521cc4611a783f4a8174bd0fcec5f4a47357ac1 is the first bad commit
commit 7521cc4611a783f4a8174bd0fcec5f4a47357ac1
Author: Ronnie Sahlberg sahlb...@google.com
Date:   Wed Apr 30 09:22:45 2014 -0700

refs.c: make delete_ref use a transaction

Best regards,
Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

2.2.0-rc behavior changes (2/2)

[Bryan Turner]

I've been running a test suite we use to verify Git behaviors across
versions, and the 2.2.0 RCs (0 and 1 both) have a couple of small
behavioral differences. I'm sending them in separate e-mails just to
make the contents easier to grok.

Important: It's entirely possible neither of these is a _bug_; they
may both be intentional changes in behavior.

Second change: git gc --auto now fails if there are loose empty blobs.

We have a test which just touched empty files in objects/17 to trigger
the gc --auto preconditions (Note: I realize this is completely
invalid, and I've changed the test to no longer do this; I'm only
surfacing the behavioral change).

On Xubuntu 14.10 I can reproduce this using bash with the following steps:
git init gc
cd gc
echo Hello, world  file.txt
git add file.txt
git commit -m Initial commit
mkdir .git/objects/17
git config gc.auto 2
git config --bool gc.autodetach false
for i in $(seq 1 20); do touch .git/objects/17/$(head -n 4096
/dev/urandom | openssl sha1 | cut -c 10-47); done

(openssl sha1 on my machine prefixes the SHA-1s with (stdin)= , so
the cut is both to shorten the SHA-1 and to drop that prefix)

In 2.1.x and prior, git gc --auto appears to ignore those objects and
exit 0, although it does note that there are still too many loose
objects:
bturner@felurian:~/tmp/gc$ git version
git version 2.1.0
bturner@felurian:~/tmp/gc$ git gc --auto
Auto packing the repository for optimum performance.
See git help gc for manual housekeeping.
Nothing new to pack.
warning: There are too many unreachable loose objects; run 'git prune'
to remove them.

In the 2.2.0 RCs git gc --auto exits with 255 and the following errors:
bturner@felurian:~/tmp/gc$ /opt/git-2.2.0-rc1/bin/git gc --auto
Auto packing the repository for optimum performance.
See git help gc for manual housekeeping.
Nothing new to pack.
error: object file
.git/objects/17/02d54e8fba95ef9968a0c9b183fe22ec551c86 is empty
fatal: unable to get object info for 1702d54e8fba95ef9968a0c9b183fe22ec551c86
error: failed to run prune

Making git gc more sensitive to invalid objects may be a good thing. I
only point out this behavior change because the change it bisects to
doesn't really cite this as an intentional change.

This change bisects to:

bturner@felurian:~/Development/git/git$ git bisect good
d3038d22f91aad9620bd8e6fc43fc67c16219738 is the first bad commit
commit d3038d22f91aad9620bd8e6fc43fc67c16219738
Author: Jeff King p...@peff.net
Date:   Wed Oct 15 18:41:35 2014 -0400

prune: keep objects reachable from recent objects

Best regards,
Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2.2.0-rc behavior changes (1/2)

[Bryan Turner]

On Mon, Nov 10, 2014 at 8:22 PM, Jeff King p...@peff.net wrote:
 On Mon, Nov 10, 2014 at 07:47:32PM +1100, Bryan Turner wrote:

 First change: git update-ref -d /refs/heads/nonexistent
 some-valid-sha1 now produces an error about ref locking that it
 didn't produce before

 Git 2.1.x and prior produced this output:
 error: unable to resolve reference refs/heads/nonexistent: No such
 file or directory

 Now, in the 2.2.0 RCs, it says:
 error: unable to resolve reference refs/heads/nonexistent: No such
 file or directory
 error: Cannot lock the ref 'refs/heads/nonexistent'.

 This one feels more like a bug, but again may not be. I say it feels
 like a bug because of the order of the messages: If git has decided
 the ref doesn't exist, why is it still trying to lock it?

 I don't think this is a bug. The order you see is because the code goes
 something like this:

   1. the parent function calls a sub-function to lock

   2. the sub-function generates the error no such file or directory
  and returns failure to the caller

   3. the caller reports that acquiring the lock failed

 The only thing that has changed between the two is step (3), but it is
 not an extra lock action after the error. It is just a more verbose
 report of the same error.

 That being said, the sub-function (lock_ref_sha1_basic) gives a much
 more useful message. So it would be a nice enhancement to make sure that
 it prints something useful in every return case, and then drop the
 message from the caller.

 As an aside, I'm also slightly confused by your output. Are you feeding
 /refs/heads/nonexistent (with a leading slash), or
 refs/heads/nonexistent (no leading slash)? If the latter, then that
 should silently succeed (and seems to in my tests). If the former, then
 the real problem is not ENOENT, but rather EINVAL; that name is not a
 valid refname.

 Older versions of git would produce:

   error: unable to resolve reference /refs/heads/nonexistent: No such file or 
 directory

 which is like the error you showed, but note that the refname is
 reported with the leading slash. In v2.2.0-rc1, this is:

   error: unable to resolve reference /refs/heads/nonexistent: Invalid argument
   error: Cannot lock the ref '/refs/heads/nonexistent'.

 which is more accurate. I could explain the differences in our output
 from some simple transcription errors when writing your email, but I
 wanted to make sure I am not missing something.

Sorry, no, you're not missing anything. That is indeed a transcription
error from my e-mail. The test in question is using
refs/heads/nonexistent.

Thanks for the quick response, Jeff. With the sub-function the
ordering of the messages makes perfect sense.


 -Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2.2.0-rc behavior changes (2/2)

[Bryan Turner]

On Mon, Nov 10, 2014 at 8:31 PM, Jeff King p...@peff.net wrote:
 On Mon, Nov 10, 2014 at 07:51:00PM +1100, Bryan Turner wrote:

 Second change: git gc --auto now fails if there are loose empty blobs.

 We have a test which just touched empty files in objects/17 to trigger
 the gc --auto preconditions (Note: I realize this is completely
 invalid, and I've changed the test to no longer do this; I'm only
 surfacing the behavioral change).

 This is expected. As you noticed here:

 error: object file
 .git/objects/17/02d54e8fba95ef9968a0c9b183fe22ec551c86 is empty
 fatal: unable to get object info for 1702d54e8fba95ef9968a0c9b183fe22ec551c86
 error: failed to run prune

 the error comes from git prune failing. It is using unreachable but
 recent objects as graph tips to keep around. If we can't load a tip
 object, we abort the prune, because we would not want to delete objects
 that might have been referenced (e.g., if it were a real corrupted
 object that we could not read).

 The second behavior (die on broken objects) has been around for a while.
 The new change (in the commit you bisected to) is that we are
 considering more objects.

 I'll admit I didn't really consider the impact on sticking broken object
 files into the object directory, but I think the code is doing the
 sensible and safe thing.

I agree. I wasn't aware this particular test was using such a
questionable mechanism to trigger automatic garbage collection. I
think the change to make git gc --auto more sensitive to this type of
thing is definitely an improvement. As noted, I just wanted to surface
the behavior change.

Thanks again for your quick response!


 -Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

git diff-tree commit detail bug in 2.0.2 and 2.0.3

[Bryan Turner]

Using git diff-tree --stdin on 2.0.2 and 2.0.3 produces incorrect
commit messages.

Here's an example to reproduce the issue:

bturner@ubuntu:/tmp$ git init --bare test.git
Initialized empty Git repository in /tmp/test.git/
bturner@ubuntu:/tmp$ git clone test.git
Cloning into 'test'...
warning: You appear to have cloned an empty repository.
done.
bturner@ubuntu:/tmp$ cd test
bturner@ubuntu:/tmp/test$ echo Hello  file.txt
bturner@ubuntu:/tmp/test$ git add file.txt
bturner@ubuntu:/tmp/test$ git commit -m Initial commit
[master (root-commit) c5e16f3] Initial commit
 1 file changed, 1 insertion(+)
 create mode 100644 file.txt
bturner@ubuntu:/tmp/test$ echo World  file.txt
bturner@ubuntu:/tmp/test$ git commit -am Second commit
[master 9214ac7] Second commit
 1 file changed, 1 insertion(+)
bturner@ubuntu:/tmp/test$ git push origin HEAD
Counting objects: 6, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (6/6), 446 bytes | 0 bytes/s, done.
Total 6 (delta 0), reused 0 (delta 0)
To /tmp/test.git
 * [new branch]  HEAD - master
bturner@ubuntu:/tmp/test$ cd ../test.git/
bturner@ubuntu:/tmp/test.git$ git rev-list -1
--format=%H|%h|%P|%p|%aN|%aE|%at%n%B%n 9214ac7
commit 9214ac79728424a971244c34432c6d948754198d
9214ac79728424a971244c34432c6d948754198d|9214ac7|c5e16f37164f1b7411685def64d7390775437f07|c5e16f3|Bryan
Turner|btur...@atlassian.com|1406539558
Second commit


bturner@ubuntu:/tmp/test.git$ /opt/git/2.0.3/bin/git diff-tree
--no-renames --always --format=commit
%H%n%H|%h|%P|%p|%aN|%aE|%at|%B%n --root
9214ac79728424a971244c34432c6d948754198d
commit 9214ac79728424a971244c34432c6d948754198d
9214ac79728424a971244c34432c6d948754198d|9214ac79728424a971244c34432c6d948754198d|c5e16f37164f1b7411685def64d7390775437f07|c5e16f37164f1b7411685def64d7390775437f07|Bryan
Turner|btur...@atlassian.com|1406539558|Second commit



:100644 100644 e965047ad7c57865823c7d992b1d046ea66edf78
f9264f7fbd31ae7a18b7931ed8946fb0aebb0af3 Mfile.txt
bturner@ubuntu:/tmp/test.git$ /opt/git/2.0.3/bin/git diff-tree
--no-renames --always --format=commit
%H%n%H|%h|%P|%p|%aN|%aE|%at|%B%n --root --stdin
--9214ac79728424a971244c34432c6d948754198d
commit 9214ac79728424a971244c34432c6d948754198d
9214ac79728424a971244c34432c6d948754198d|9214ac79728424a971244c34432c6d948754198d|c5e16f37164f1b7411685def64d7390775437f07|c5e16f37164f1b7411685def64d7390775437f07|Bryan
Turner|btur...@atlassian.com|1406539543|Initial commit



:100644 100644 e965047ad7c57865823c7d992b1d046ea66edf78
f9264f7fbd31ae7a18b7931ed8946fb0aebb0af3 Mfile.txt
bturner@ubuntu:/tmp/test.git$

Running a git bisect between v2.0.1, which does not manifest this
issue, and v2.0.2 fingers the following commit:
bturner@ubuntu:~/Development/oss/git/git$ git bisect bad
c1b3c71f4b4571abb2b2a457122fd100dc9f7eb0 is the first bad commit
commit c1b3c71f4b4571abb2b2a457122fd100dc9f7eb0
Author: Jeff King p...@peff.net
Date:   Tue Jun 10 17:43:02 2014 -0400

commit: convert commit-buffer to a slab

Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git diff-tree commit detail bug in 2.0.2 and 2.0.3

[Bryan Turner]

On Mon, Jul 28, 2014 at 8:44 PM, Jeff King p...@peff.net wrote:
 On Mon, Jul 28, 2014 at 06:35:04AM -0400, Jeff King wrote:

 I haven't reproduced here yet, but this is almost certainly the bug
 where lookup_unknown_object causes a bogus commit-index field (and
 prior to the commit you found, diff-tree did not use commit-index).

 The series that Junio has in jk/alloc-commit-id should fix the problem
 (it's in master already, and slated for v2.1.0).

 Yep, that's definitely it. Here's the minimum reproduction:

   git init
   git commit --allow-empty -m one
   git commit --allow-empty -m two
   git rev-list HEAD | git diff-tree --stdin --always --format=%s

 That yields:

   one
   one

 on v2.0.3, but merging in jk/alloc-commit-id yields:

   two
   one

 -Peff

Thanks for digging into it, Jeff. I should have tried it against 2.1.0
myself. I've run my entire matrix of tests now against 2.1.0-rc0 and
the diff-tree bug appears fixed on that tag. I noticed a different
change, though:

bturner@ubuntu:~/tmp/test$ /opt/git/2.1.0-rc0/bin/git check-ref-format
ref/with/trailing/dot.
bturner@ubuntu:~/tmp/test$ echo $?
0
bturner@ubuntu:~/tmp/test$ /opt/git/2.0.3/bin/git check-ref-format
ref/with/trailing/dot.
bturner@ubuntu:~/tmp/test$ echo $?
1

It looks like refs ending in a dot are now legal in 2.1.0? Is that
intentional? A quick git bisect is fingering:
bturner@ubuntu:~/Development/oss/git/git$ git bisect bad
745224e04a03e4544c58d5d38d3c54f67100f8eb is the first bad commit
commit 745224e04a03e4544c58d5d38d3c54f67100f8eb
Author: David Turner dtur...@twopensource.com
Date:   Wed Jun 18 01:54:42 2014 -0400

Best regards,
Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git diff-tree commit detail bug in 2.0.2 and 2.0.3

[Bryan Turner]

On Tue, Jul 29, 2014 at 10:11 AM, Junio C Hamano gits...@pobox.com wrote:
 Junio C Hamano gits...@pobox.com writes:

 Yeah, I'm fine with a straight revert, too (I think it is fine to keep
 in master, though). I think jk/alloc-commit-id is built right on top of
 the original commit-slab topic, so it should be easy to do either way.

 Thanks for dealing with it.

 Whatever we do, perhaps it is worth applying the test below on top?

 Yeah, thanks.  I think that is a good idea.  I was preparing a patch
 to tuck your minimum reproduction at the end of 4202, but your version
 and placement makes good sense.

 OK, I pushed out updated 'maint' and 'master'.  The former merges
 a rebased version of jk/alloc-commit-id in to make the reorganize
 the way we manage the in-core commit data topic, and the latter
 reverts the Use SSE to micro-optimize a leaf function to check the
 format of a ref string.

 Please give them some quick sanity check.

 Thanks.

Thanks both of you; I appreciate your efforts! I've run my suite of
tests against the tips of master and maint and all 681 pass for each.
Looks good to me.

Best regards,
Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Returning error message from custom smart http server

[Bryan Turner]

On Sat, May 17, 2014 at 9:01 AM, brian m. carlson
sand...@crustytoothpaste.net wrote:
 On Tue, May 13, 2014 at 09:39:59AM +0200, Ákos, Tajti wrote:
 Dear List,

 we implemented our own git smart http server to be able to check permissions
 and other thing before pushes. It works fine, however, the error messages we
 generate on the server side are not displayed by the command line client. On
 the server we generate error messages like this:

 response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
 response.getWriter().write(msg);

 On the command line we get this:

 Total 0 (delta 0), reused 0 (delta 0)
 POST git-receive-pack (290 bytes)
 efrror: RPC failed; result=22, HTTP code = 401
 atal: The remote end hung up unexpectedly
 fatal: The remote end hung up unexpectedly

 The server message is completely missing. Is there a solution for this?

You should not need a patched git; the wire protocol itself has a
mechanism for sending smart error messages. It's not particularly
_obvious_, but it's there.

For starters, to return an error message, your status must be 200 OK.
You can't return any other status code or Git will interpret your
error as some form of _HTTP_ error rather than a _git_ error.

In the smart protocol the client sends a service to the server as a
query parameter, like ?service=git-receive-pack. For such a request,
you need to:
- Set the content type to application/x-service-advertisement
(e.g. application/x-git-receive-pack-advertisement) (Not all command
line Git versions require this, but JGit does)
- Set the status code as 200 OK
- Write back a payload where the first 4 bytes are the hex-encoded
length of the text (where  is max length for a single packet).
Note that the 4 bytes for the size are _part_ of that length, so if
you're writing Test the length is 8, not 4
- After the size, you write # service=service (e.g. #
service=git-receive-pack; note the space after the #) This is the
metadata. For an error, you don't really have much to say.
- After that, an empty packet, which is  (four zeros) This
separates the metadata from the ref advertisement
- After that you can write your message, beginning with ERR  (note
the trailing space there). The ERR  tells Git what you're writing
isn't a ref, it's an error. I'd recommend appending a newline (and add
1 more to your length for it), because when Git echoes your error
message it doesn't seem to do that

I'm not sure whether there's a document that describes all of this; I
found it by digging into the Git source code (you can find the ERR
handling in connect.c, get_remote_heads). This may be exploiting the
protocol, I'll leave that to someone more knowledgeable on how they
_intended_ this all to be used, but it works for us.

A full example looks something like this: 0036#
service=git-receive-packERR This is a test\n

Hope this helps,
Bryan Turner


 It does look that way.  Does the following patch work for you?

 -- 8 --
 Subject: [PATCH] http: provide server's error message on RPC failure

 The server might provide a custom error message that is useful to the user.
 Provide this message to the user if HTTP RPC fails.

 Signed-off-by: brian m. carlson sand...@crustytoothpaste.net
 ---
  remote-curl.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

 diff --git a/remote-curl.c b/remote-curl.c
 index 52c2d96..5984d35 100644
 --- a/remote-curl.c
 +++ b/remote-curl.c
 @@ -426,8 +426,8 @@ static int run_slot(struct active_request_slot *slot,
 err = run_one_slot(slot, results);

 if (err != HTTP_OK  err != HTTP_REAUTH) {
 -   error(RPC failed; result=%d, HTTP code = %ld,
 - results-curl_result, results-http_code);
 +   error(RPC failed; result=%d, HTTP code = %ld (%s),
 + results-curl_result, results-http_code, 
 curl_errorstr);
 }

 return err;
 -- 8 --

 --
 brian m. carlson / brian with sandals: Houston, Texas, US
 +1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
 OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Pack bitmaps on Git for Windows

[Bryan Turner]

It looks like the documentation for bitmaps is being included in the
1.9.2 release of Git for Windows but the functionality itself is not
present. For example, doc\git\html\git-config.html includes settings
like these:

pack.useBitmaps

When true, git will use pack bitmaps (if available) when packing to
stdout (e.g., during the server side of a fetch). Defaults to true.
You should not generally need to turn this off unless you are
debugging pack bitmaps.

pack.writebitmaps

When true, git will write a bitmap index when packing all objects to
disk (e.g., when git repack -a is run). This index can speed up the
counting objects phase of subsequent packs created for clones and
fetches, at the cost of some disk space and extra time spent on the
initial repack. Defaults to false.


Repacking repositories with pack.writebitmaps set to true doesn't
write a bitmap, suggesting the functionality is not actually present.

This does not appear to be an issue with normal git, per the
documentation for git config at
http://www.git-scm.com/docs/git-config.

Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Re: [Bug report] 'git status' always says Your branch is up-to-date with 'origin/master'

[Bryan Turner]

On 6 January 2014 01:24, Thomas Ackermann th.ac...@arcor.de wrote:


 Hi Jiang,

 this happens with all of my repo clones (I am using V1.8.5.2
 on Windows and on Linux). Steps to reproduce:

 mkdir repo_a  cd repo_a  git init .
 echo 1foo  git add foo  git commit -m 1
 cd ..
 git clone repo_a repo_b
 cd repo_a
 echo 2foo  git add foo  git commit -m 2
 cd ../repo_b
 git status
 git checkout -b branch
 git checkout master

 'git status' and 'git checkout master' in repo_b are now
 reporting Your branch is up-to-date with 'origin/master'
 which is obviously wrong.


Unfortunately that's not true. In repo_b your ref for origin/master
has not moved. It has remotely (meaning refs/heads/master in repo_a
has moved), but git status is not hitting the remote to find out; it
only looks at the local state. To see what I mean, run git fetch in
repo_b. Once you do that, you'll see that git status correctly reports
you're behind.


 ---
 Thomas

 - Original Nachricht 
 Von: Jiang Xin worldhello@gmail.com
 An:  Thomas Ackermann th.ac...@arcor.de
 Datum:   06.01.2014 06:31
 Betreff: Re: [Bug report] 'git status' always says Your branch is up-to-date
  with 'origin/master'

  2014/1/5 Thomas Ackermann th.ac...@arcor.de:
   Since f223459 status: always show tracking branch even no change
   'git status' (and 'git checkout master' always says
   Your branch is up-to-date with 'origin/master'
   even if 'origin/master' is way ahead from local 'master'.
 
  Hi, Thomas
 
  Can you provide your operations so that I can reproduce this issue?
 
  In the commit you mentioned above, there was a new test case named
  'checkout (up-to-date with upstream)' added in 't6040'. I also add two
  test-cases locally in order to reproduce the issue you report, and run
  them in arbitrary orders, but they all look fine:
 
  ok 4 - checkout (behind upstream)
  ok 5 - checkout (ahead upstream)
  ok 6 - checkout (diverged from upstream)
  ok 7 - checkout with local tracked branch
  ok 8 - checkout (upstream is gone)
  ok 9 - checkout (up-to-date with upstream)
  ok 10 - checkout (upstream is gone)
  ok 11 - checkout with local tracked branch
  ok 12 - checkout (diverged from upstream)
  ok 13 - checkout (ahead upstream)
  ok 14 - checkout (behind upstream)
  ok 15 - checkout (diverged from upstream)
  ok 16 - checkout (upstream is gone)
  ok 17 - checkout (ahead upstream)
  ok 18 - checkout with local tracked branch
  ok 19 - checkout (behind upstream)
 
 
  The two additional test cases I used locally are:
 
  checkout_test1() {
  test_expect_success 'checkout (behind upstream)' '
  (
  cd test  git checkout b3
  ) actual 
  test_i18ngrep is behind .* by 1 commit, and can be
  fast-forwarded actual
  '
  }
 
  checkout_test_2() {
  test_expect_success 'checkout (ahead upstream)' '
  (
  cd test  git checkout b4
  ) actual 
  test_i18ngrep is ahead of .* by 2 commits actual
  '
  }
 
  --
  Jiang Xin
 

 ---
 Thomas
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Symbolic refs break ref advertisement on 1.8.4.3+

[Bryan Turner]

According to a git bisect between the v1.8.4 and v1.8.4.3 tags, it
appears the changes in 5e7dcad, upload-pack: send non-HEAD symbolic
refs, cause the ref advertisement to fail of the repository has more
than a handful of symbolic refs. Here's a simple reproduce case
(tested on Bash):

Aphrael:example bturner$ git version
git version 1.8.4.3
Aphrael:symbolic-refs bturner$ git init example
Initialized empty Git repository in /Users/bturner/example/.git/
Aphrael:symbolic-refs bturner$ cd example
Aphrael:example bturner$ echo Testing...  file.txt
Aphrael:example bturner$ git add file.txt
Aphrael:example bturner$ git commit -m Initial commit
[master (root-commit) b4c4b2a] Initial commit
 1 file changed, 1 insertion(+)
 create mode 100644 file.txt
Aphrael:example bturner$ for ((i=1;i21;i++)); do git symbolic-ref
refs/heads/syms/$i refs/heads/master; done
Aphrael:example bturner$ git ls-remote .
fatal: protocol error: impossibly long line
fatal: Could not read from remote repository.

A symref= entry is written into the first packet of the ref
advertisement, right after the capabilities, for each symbolic ref in
the repository. Unfortunately, no splitting is done on that value and
so once you have 15-20 symbolic refs (more or less depending on path
lengths), you blow the 996 byte limit in format_packet (pkt-line.c)
and all further clone/fetch operations fail.

I've verified this same issue exists in all 1.8.5 RCs.

Best regards,
Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Error message: unable to open object pack directory: .git/objects/pack: Too many open files

[Bryan Turner]

On 13 November 2013 13:48, Duy Nguyen pclo...@gmail.com wrote:
 On Tue, Nov 12, 2013 at 9:38 PM, Лежанкин Иван abys...@gmail.com wrote:
 I use linux.

 I can't publish repo - it's proprietary, it weights ~300M unpacked,
 and it uses references ~3Gb.
 Error message doesn't contain remote: prefix.
 The majority of opened files have name like:
 objects/pack/pack-hash.[ pack | idx ]
 They all are from referenced repo.

 There's a fix related to too many open packs, 88d0db5 (sha1_file:
 introduce close_one_pack() to close packs on fd pressure - 2013-08-01)
 but it's only available in v1.5.0-rc0 or rc1. Could you try that
 version?

I believe you mean 1.8.5-rc0 or 1.8.5-rc1, is that correct?



 I have disabled gc in the referenced repo - to prevent all local repos
 from breaking. May this be the cause of problem?

 On 12 November 2013 18:19, Duy Nguyen pclo...@gmail.com wrote:
 On Tue, Nov 12, 2013 at 3:02 PM, Лежанкин Иван abys...@gmail.com wrote:
 I get this error message every time I want to do a `git push`.
 git version: 1.8.4.2

 Is it a known issue? Do you need additional info to investigate it?

 What OS do you use? If the repository can be published, please do.
 Compress the whole .git directory, don't push or anything. Does the
 error message begin with remote: ? If not you could try strace and
 at least identify what files are opened.
 --
 Duy



 --
 Duy
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: GIT Hooks and security

[Bryan Turner]

No, the .git/hooks directory in your clone is created from your local
templates, installed with your Git distribution, not the remote hooks.
On Linux distributions, these templates are often in someplace like
/usr/share/git-core/templates (for normal packages), and on Windows
with msysgit they are in share\git-core\templates under your
installation directory. If you look in this directory you will see a
hooks directory containing the sample hooks.

Hooks from a remote repository are never cloned. As far as I'm aware,
nothing from the .git directory (aside from refs and packs, of course)
is cloned, including configuration. Your .git directory after a clone
is completely new, assembled from scratch. There's nothing in the Git
wire protocol (currently) for moving other data like configuration or
hooks, and this sort of malicious code injection is one of the reasons
I've seen discussed on the list for why that's the case.

Hope this helps,
Bryan Turner


On 26 October 2013 09:25, Olivier Revollat revol...@gmail.com wrote:

 But when someone do a clone he don't have .git/hooks directory
 downloaded to his local computer ? I thought so ...

 2013/10/26 Junio C Hamano gits...@pobox.com:
  Olivier Revollat revol...@gmail.com writes:
 
  I was wondering : What if I had a malicious GIT repository who can
  inject code  via git hooks mechanism : someone clone my repo and
  some malicious code is executed when a certain GIT hook is triggered
  (for example on commit (prepare-commit-msg' hook))
 
  In that somebody else's clone, you will not have _your_ malicious
  hook installed, unless that cloner explicitly does something stupid,
  like copying that malicious hook.



 --
 Mathematics is made of 50 percent formulas, 50 percent proofs, and 50
 percent imagination.
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Moving commits from one branch to another (improving my git fu!)

[Bryan Turner]

A quick glance at your command line and the man page for git rebase
suggests the problem was you didn't actually use --onto. I believe the
correct command would be:

git rebase --onto dev stable topicA


That should start by determining which commits are one topicA but not
stable and then checkout dev and apply those new commits.


Hope this helps,
Bryan Turner

On 22 October 2013 16:38, Mandeep Sandhu mandeepsandhu@gmail.com wrote:
 Hi All,

 I'm in a bit of a pickle! :) So I've come to ask for help from the guru's 
 here.

 My story is not unique but somehow the various suggested solutions
 don't seem to work in my case.

 * I was working on a feature which was supposed to be done off our
 'dev' branch. But instead I forgot and branched out my topic branch
 from master (or as we call it 'stable').
 * I did 2 commits and finished off my work. Only later realizing that
 it had to be done off 'dev'.
 * Now I want to move my 2 commits (which are the top 2 commits on my
 topic branch) to a new branch which is branched off 'dev'.

 $ git branch
 * topicA
 * stable
 * dev

 topicA was based on stable. But now I want to base it dev.

 So I did what was most suggested, i.e use 'git rebase --onto'.

 Here's what I did when I was in topicA:

 $ git rebase dev stable topicA
 (this was suggested in the manpage as well).

 This started off a massive rebase operation, because the 'dev' branch
 is vastly diverged from stable, and a lot of conflicts started
 appearing.

 I'm not sure if I'm doing it right here, so can anyone suggest whether
 this is right or do I need to do it differently?

 PS: Please CC me (mandeepsandhu@gmail.com) in your reply as I'm
 not currently subscribed to the list.

 Thanks for your time.

 Regards,
 -mandeep
 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] More typofixes.

[Bryan Turner]

 diff --git a/pathspec.c b/pathspec.c
 index 6ea0867..27ffe77 100644
 --- a/pathspec.c
 +++ b/pathspec.c
 @@ -40,7 +40,7 @@ void add_pathspec_matches_against_index(const char 
 **pathspec,
  /*
   * Finds which of the given pathspecs match items in the index.
   *
 - * This is a one-shot wrapper around add_pathspec_matches_against_index()
 + * This is an one-shot wrapper around add_pathspec_matches_against_index()
   * which allocates, populates, and returns a seen[] array indicating the
   * nature of the closest (i.e. most specific) matches which each of the
   * given pathspecs achieves against all items in the index.


 diff --git a/builtin/log.c b/builtin/log.c
 index 2625f98..01b49b3 100644
 --- a/builtin/log.c
 +++ b/builtin/log.c
 @@ -304,7 +304,7 @@ static void setup_early_output(struct rev_info *rev)
  * tenth of a second, don't even bother doing the
  * early-output thing..
  *
 -* This is a one-time-only trigger.
 +* This is an one-time-only trigger.
  */
 early_output_timer.it_value.tv_sec = 0;
 early_output_timer.it_value.tv_usec = 10;

These two are not typos. This is a one-time is grammatically correct.


 diff --git a/commit-slab.h b/commit-slab.h
 index 7d48163..4f1c796 100644
 --- a/commit-slab.h
 +++ b/commit-slab.h
 @@ -22,7 +22,7 @@
   *
   *   Initializes the indegree slab that associates an array of integers
   *   to each commit. 'stride' specifies how big each array is.  The slab
 - *   that id initialied by the variant without _with_stride associates
 + *   that id initialized by the variant without _with_stride associates
   *   each commit with an array of one integer.
   */

To my reading, it seems like that id should be is, in addition to
the typo with initialized

That's a pretty impressive list of corrections.

Reviewed-by: Bryan Turner btur...@atlassian.com
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Flatten history

[Bryan Turner]

On 30 July 2013 13:50, Felipe Contreras felipe.contre...@gmail.com wrote:
 On Mon, Jul 29, 2013 at 8:42 PM, Hilco Wijbenga
 hilco.wijbe...@gmail.com wrote:
 Hi all,

 I have a (public) feature branch that has been kept up-to-date with
 master by regularly merging master back into it. I would now like to
 get all the changes from feature but not any of the commits.
 Basically, I want to replay all of feature's commits without creating
 those commits.

 I thought something like

 git cherry-pick -n abcd^..feature

 should do the trick (while on master, where abcd is the SHA-1 of the
 commit where feature was created) but I get conflicts.

 First, why the conflicts? I have done all the merges so cherry-pick
 should simply be able to replay them? Second, what is the correct way
 of doing this?

 Perhaps

 % git cherry-pick -n --no-merges --right-only --topo-order
 --cherry-pick abcd^..feature

 --
 Felipe Contreras

Wouldn't git merge --squash do what you're looking for? It seems like
the only way to not get conflicts trying to cherry pick is if you
never had any conflicts while you were merging master into your
feature branch. Evil merges, where you actually have to change code,
even if it's just to resolve conflicts, don't tend to replay
correctly.

It seems like this should do it:
% git checkout master
Switched to branch 'master'
% git merge --squash feature
Squash commit -- not updating HEAD
Automatic merge went well; stopped before committing as requested
% git commit

git merge --squash will just apply the changes without creating a
commit. You can then make any final changes you want to and write your
commit message for the feature.

Hope this helps,
Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] The images from picon and gravatar are always used over http://, and browsers give mixed contents warning when gitweb is served over https://.

[Bryan Turner]

This won't work correctly as-is. The secure URL for Gravatar is
https://secure.gravatar.com[1], not https://www.gravatar.com;.

[1] See the Secure Requests section on:
https://en.gravatar.com/site/implement/images/

On 29 January 2013 14:03, Junio C Hamano gits...@pobox.com wrote:

 Thanks; will queue.


 --
 To unsubscribe from this list: send the line unsubscribe git in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] The images from picon and gravatar are always used over http://, and browsers give mixed contents warning when gitweb is served over https://.

[Bryan Turner]

Interesting. I wonder if they've changed it recently. I only pointed
it out because a software product I'm working on had a bug because it
was building the URLs with https://www...; and the resulting images
were showing as X's instead of avatars. We had to change the
implementation to use https://secure...; to get the avatars to load
correctly. That's been ~8 months ago now, though, so maybe it's no
longer the case. It seems like it would be much more convenient if
they just changed the scheme.

Bryan

On 29 January 2013 15:12, Jonathan Nieder jrnie...@gmail.com wrote:
 Hi Bryan,

 Bryan Turner wrote:

 This won't work correctly as-is. The secure URL for Gravatar is
 https://secure.gravatar.com[1], not https://www.gravatar.com;.

 Odd.  https://www.gravatar.com/; also seems to work.  I've put in a
 technical support query to find out what the Gravatar admins prefer.

 Thanks,
 Jonathan
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Question: git clone --no-checkout behavior

[Bryan Turner]

I've witnessed the following behavior in both git 1.7.6 and 1.7.10.4.

Assume I have a bare clone, some-repo.git. If I run:
- git clone --shared --no-checkout /path/to/some-repo.git shared-repo
- cd shared-repo
- git status

I see that every file in the repository is _staged_ for deletion. I'm
not surprised every file shows deleted; I'm surprised that the
deletion starts out already staged. A git reset unstages all of the
deletions and I'm good to go. I'm just wondering why they're all
staged in the first place; it seems counter-intuitive.

Can anyone shed some light on this?
Bryan Turner
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html