Re: Inconsistent results of git blame --porcelain when detecting copies from other files
On Tue, Feb 21, 2017 at 12:25:37PM +, Sokolov, Konstantin wrote: > Thanks for going into the issue. As far as I understand 2.12 won't > change the discussed behavior of --procelain. We will switch to > --line-procelain. After the current discussion it seems to be less > error prone, more future-proof and our current parser can handle it > without any changes. Right, the 2.12 change is only fixing a case where the "previous" key/value line was not repeated at the correct spots. The same parsing rules still hold. -Peff
AW: Inconsistent results of git blame --porcelain when detecting copies from other files
Thanks for going into the issue. As far as I understand 2.12 won't change the discussed behavior of --procelain. We will switch to --line-procelain. After the current discussion it seems to be less error prone, more future-proof and our current parser can handle it without any changes. Regards Konstantin -Ursprüngliche Nachricht- Von: Jeff King [mailto:p...@peff.net] Gesendet: Montag, 20. Februar 2017 23:16 An: Junio C Hamano Cc: Sokolov, Konstantin (ext) (CT RDA SSI ADM-DE); git@vger.kernel.org Betreff: Re: Inconsistent results of git blame --porcelain when detecting copies from other files On Mon, Feb 20, 2017 at 01:30:29PM -0800, Junio C Hamano wrote: > "Sokolov, Konstantin" writes: > > > However, when using --porcelain DirectoryReader.java is reported as the > > origin of lines 502-504: > > ... > > This is not only inconsistent with the other outputs but the output is also > > inconsistent in itself because lines 496 -498 do not even exist in a > > previous version of DirectoryReader.java. > > Hmph, this sounds vaguely familiar with > > > http://public-inbox.org/git/20170106042051.nwjiuyyp7ljhs...@sigill.int > ra.peff.net > > which is part of Git 2.12-rc0 Yeah, I had the same thought while reading Konstantin's report. I'm not sure Git is wrong, though. I think it's just the way the porcelain output works. Here's a minimal reproduction; the interesting thing is when a commit is mentioned twice (as happens on lines 1 and 5 here): git init repo cd repo # use long lines to make sure we trigger content-movement detection for i in $(seq 1 5); do echo this is really long line number $i done >file git add file git commit -m initial sed 's/1/one/; s/5/five/' renamed git rm file git add renamed git commit -m 'rename and use english' git blame renamed git blame --line-porcelain renamed git blame --porcelain renamed The first blame output looks something like this: bab03701 renamed ... line number 1 ^dda1349 file... line number 2 ^dda1349 file... line number 3 ^dda1349 file... line number 4 bab03701 renamed ... line number 5 so we can see it's the same case. The --line-porcelain similarly matches the commits and filenames. But the --porcelain output is: bab037010dcabaf0509db27bf232d25659b180fa 1 1 1 ... filename renamed this is really long line number one dda1349d41da859f4c37e018dbed714ba6c1aa18 2 2 3 ... filename file this is really long line number 2 dda1349d41da859f4c37e018dbed714ba6c1aa18 3 3 this is really long line number 3 dda1349d41da859f4c37e018dbed714ba6c1aa18 4 4 this is really long line number 4 bab037010dcabaf0509db27bf232d25659b180fa 5 5 1 this is really long line number five You might be tempted to say that the fifth line comes from "filename file", because that was the last "filename" entry we saw. But that's _not_ how the porcelain output works. That "filename" entry was associated with dda1349, but the line comes from bab0370 here. The simplest way (IMHO) to parse --porcelain output is: - maintain a mapping of commit sha1s to the commit's details - whenever you see a " []" line, any key-value fields which follow impact _only_ that sha1, and you should update the details for that map entry - when you see the actual tab-indented line content, you have gotten all of the key-value updates for that sha1. You can now safely do what you like with the line entry. Another way, if you don't want to update your mapping, is to actually pay attention to the size-of-hunk headers. In this case the middle three lines come in their own hunk (which you can see from the "2 2 3" header on the second line). The "filename" field we get applies to that hunk, but once we switch to a different one, the filename field needs to be looked up in the commit mapping. But it's definitely not correct to blindly apply one "filename" field to subsequent lines in other hunks. And yes, I do think this is probably more complex than it needs to be. I didn't write it. And I don't think it is worth the backwards compatibility headache of trying to change it now. It's possible this could be better documented (I didn't look at the documentation to write that explanation; I happened to puzzle it out for somebody else recently who had a similar case. That's what led to the bug-fix in the message you linked). -Peff
Re: Inconsistent results of git blame --porcelain when detecting copies from other files
Jeff King writes: > The simplest way (IMHO) to parse --porcelain output is: > > - maintain a mapping of commit sha1s to the commit's details > > - whenever you see a " []" > line, any key-value fields which follow impact _only_ that sha1, and > you should update the details for that map entry > > - when you see the actual tab-indented line content, you have gotten > all of the key-value updates for that sha1. You can now safely do > what you like with the line entry. Yup, that was how the output was meant to be read. At least in the mind of the person who designed the output format ;-)
Re: Inconsistent results of git blame --porcelain when detecting copies from other files
On Mon, Feb 20, 2017 at 01:30:29PM -0800, Junio C Hamano wrote: > "Sokolov, Konstantin" writes: > > > However, when using --porcelain DirectoryReader.java is reported as the > > origin of lines 502-504: > > ... > > This is not only inconsistent with the other outputs but the output is also > > inconsistent in itself because lines 496 -498 do not even exist in a > > previous version of DirectoryReader.java. > > Hmph, this sounds vaguely familiar with > > > http://public-inbox.org/git/20170106042051.nwjiuyyp7ljhs...@sigill.intra.peff.net > > which is part of Git 2.12-rc0 Yeah, I had the same thought while reading Konstantin's report. I'm not sure Git is wrong, though. I think it's just the way the porcelain output works. Here's a minimal reproduction; the interesting thing is when a commit is mentioned twice (as happens on lines 1 and 5 here): git init repo cd repo # use long lines to make sure we trigger content-movement detection for i in $(seq 1 5); do echo this is really long line number $i done >file git add file git commit -m initial sed 's/1/one/; s/5/five/' renamed git rm file git add renamed git commit -m 'rename and use english' git blame renamed git blame --line-porcelain renamed git blame --porcelain renamed The first blame output looks something like this: bab03701 renamed ... line number 1 ^dda1349 file... line number 2 ^dda1349 file... line number 3 ^dda1349 file... line number 4 bab03701 renamed ... line number 5 so we can see it's the same case. The --line-porcelain similarly matches the commits and filenames. But the --porcelain output is: bab037010dcabaf0509db27bf232d25659b180fa 1 1 1 ... filename renamed this is really long line number one dda1349d41da859f4c37e018dbed714ba6c1aa18 2 2 3 ... filename file this is really long line number 2 dda1349d41da859f4c37e018dbed714ba6c1aa18 3 3 this is really long line number 3 dda1349d41da859f4c37e018dbed714ba6c1aa18 4 4 this is really long line number 4 bab037010dcabaf0509db27bf232d25659b180fa 5 5 1 this is really long line number five You might be tempted to say that the fifth line comes from "filename file", because that was the last "filename" entry we saw. But that's _not_ how the porcelain output works. That "filename" entry was associated with dda1349, but the line comes from bab0370 here. The simplest way (IMHO) to parse --porcelain output is: - maintain a mapping of commit sha1s to the commit's details - whenever you see a " []" line, any key-value fields which follow impact _only_ that sha1, and you should update the details for that map entry - when you see the actual tab-indented line content, you have gotten all of the key-value updates for that sha1. You can now safely do what you like with the line entry. Another way, if you don't want to update your mapping, is to actually pay attention to the size-of-hunk headers. In this case the middle three lines come in their own hunk (which you can see from the "2 2 3" header on the second line). The "filename" field we get applies to that hunk, but once we switch to a different one, the filename field needs to be looked up in the commit mapping. But it's definitely not correct to blindly apply one "filename" field to subsequent lines in other hunks. And yes, I do think this is probably more complex than it needs to be. I didn't write it. And I don't think it is worth the backwards compatibility headache of trying to change it now. It's possible this could be better documented (I didn't look at the documentation to write that explanation; I happened to puzzle it out for somebody else recently who had a similar case. That's what led to the bug-fix in the message you linked). -Peff
Re: Inconsistent results of git blame --porcelain when detecting copies from other files
"Sokolov, Konstantin" writes: > However, when using --porcelain DirectoryReader.java is reported as the > origin of lines 502-504: > ... > This is not only inconsistent with the other outputs but the output is also > inconsistent in itself because lines 496 -498 do not even exist in a previous > version of DirectoryReader.java. Hmph, this sounds vaguely familiar with http://public-inbox.org/git/20170106042051.nwjiuyyp7ljhs...@sigill.intra.peff.net which is part of Git 2.12-rc0
Inconsistent results of git blame --porcelain when detecting copies from other files
Hi Folks!
The issue is best explained on an example. You can reproduce it using the
Lucene repo https://github.com/apache/lucene-solr.git. Tested with the
following versions: 1.8.1.6 (Ubuntu), 2.11.0.windows.1, 2.11.1.windows.1.
First, let's produce the correct results without using --procelain:
> git blame --show-name --show-number -s -w --abbrev=40 -C -C -C
> d1165b19726fa0cd13a539827a7cd43237a4feef..10ba9abeb208d37df985e95a742f756de067353f
> --not f5dba8b76709ff0ef8715b8b288a4b64d4993fa3 --
> lucene/src/java/org/apache/lucene/index/DirectoryReader.java
The following excerpt shows lines 501-505 from the output. In particular we can
see that lines 502-503 originate from IndexReader.java.
10ba9abeb208d37df985e95a742f756de067353f
lucene/src/java/org/apache/lucene/index/DirectoryReader.java 501 501)*
This method
^d1165b19726fa0cd13a539827a7cd43237a4fee
lucene/src/java/org/apache/lucene/index/IndexReader.java 496 502)*
returns the version recorded in the commit that the
^d1165b19726fa0cd13a539827a7cd43237a4fee
lucene/src/java/org/apache/lucene/index/IndexReader.java 497 503)*
reader opened. This version is advanced every time
^d1165b19726fa0cd13a539827a7cd43237a4fee
lucene/src/java/org/apache/lucene/index/IndexReader.java 498 504)* a
change is made with {@link IndexWriter}.
10ba9abeb208d37df985e95a742f756de067353f
lucene/src/java/org/apache/lucene/index/DirectoryReader.java 505 505)*/
The same information can be obtained as well by using --line-porcelain:
> git blame --show-name --show-number --line-porcelain -s -w --abbrev=40 -C -C
> -C
> d1165b19726fa0cd13a539827a7cd43237a4feef..10ba9abeb208d37df985e95a742f756de067353f
> --not f5dba8b76709ff0ef8715b8b288a4b64d4993fa3 --
> lucene/src/java/org/apache/lucene/index/DirectoryReader.java
Here is the output for line 502:
d1165b19726fa0cd13a539827a7cd43237a4feef 496 502 3
author Michael McCandless
author-mail
author-time 1327877325
author-tz +
committer Michael McCandless
committer-mail
committer-time 1327877325
committer-tz +
summary LUCENE-3725: add optional packing to FSTs
boundary
filename lucene/src/java/org/apache/lucene/index/IndexReader.java
* returns the version recorded in the commit that the
However, when using --porcelain DirectoryReader.java is reported as the origin
of lines 502-504:
> git blame --show-name --show-number --porcelain -s -w --abbrev=40 -C -C -C
> d1165b19726fa0cd13a539827a7cd43237a4feef..10ba9abeb208d37df985e95a742f756de067353f
> --not f5dba8b76709ff0ef8715b8b288a4b64d4993fa3 --
> lucene/src/java/org/apache/lucene/index/DirectoryReader.java
10ba9abeb208d37df985e95a742f756de067353f 501 501 1
author Uwe Schindler
author-mail
author-time 1327879145
author-tz +
committer Uwe Schindler
committer-mail
committer-time 1327879145
committer-tz +
summary Reverse merged revision(s) from lucene/dev/trunk up to 1237502
previous f5dba8b76709ff0ef8715b8b288a4b64d4993fa3
lucene/src/java/org/apache/lucene/index/DirectoryReader.java
filename lucene/src/java/org/apache/lucene/index/DirectoryReader.java
* This method
d1165b19726fa0cd13a539827a7cd43237a4feef 496 502 3
* returns the version recorded in the commit that the
d1165b19726fa0cd13a539827a7cd43237a4feef 497 503
* reader opened. This version is advanced every time
d1165b19726fa0cd13a539827a7cd43237a4feef 498 504
This is not only inconsistent with the other outputs but the output is also
inconsistent in itself because lines 496 -498 do not even exist in a previous
version of DirectoryReader.java.
Thanks for any feedback.
Kind Regards
Konstantin Sokolov
