Re: What's cooking in git.git (Mar 2018, #05; Wed, 28)

[Lars Schneider]

> On 30 Mar 2018, at 12:32, Lars Schneider  wrote:
> 
> 
>> On 30 Mar 2018, at 11:24, Ævar Arnfjörð Bjarmason  wrote:
>> 
>> 
>> On Wed, Mar 28 2018, Junio C. Hamano wrote:
>> 
>>> * ls/checkout-encoding (2018-03-16) 10 commits
>>> - convert: add round trip check based on 'core.checkRoundtripEncoding'
>>> - convert: add tracing for 'working-tree-encoding' attribute
>>> - convert: check for detectable errors in UTF encodings
>>> - convert: add 'working-tree-encoding' attribute
>>> - utf8: add function to detect a missing UTF-16/32 BOM
>>> - utf8: add function to detect prohibited UTF-16/32 BOM
>>> - utf8: teach same_encoding() alternative UTF encoding names
>>> - strbuf: add a case insensitive starts_with()
>>> - strbuf: add xstrdup_toupper()
>>> - strbuf: remove unnecessary NUL assignment in xstrdup_tolower()
>>> 
>>> The new "checkout-encoding" attribute can ask Git to convert the
>>> contents to the specified encoding when checking out to the working
>>> tree (and the other way around when checking in).
>> 
>> There's an issue in ca16fc3635 ("convert: check for detectable errors in
>> UTF encodings", 2018-03-15) flagged by Coverity:
>> https://public-inbox.org/git/CAGZ79kbAOcwaRzjuMtZ_HVsYvUr_7UAPbOcnrmPgsdE19q=p...@mail.gmail.com/
> 
> Thanks a lot for pointing me at this!
> I'll prepare a new round soonish.


The report says:

  >>> CID 1433528:  Null pointer dereferences  (FORWARD_NULL)
  >>> Passing null pointer "src" to "validate_encoding", which dereferences 
it.

  411 if (validate_encoding(path, enc, src, src_len, die_on_error))

However, validate_encoding() does not dereference it. It just passes the
pointer to has_prohibited_utf_bom() and is_missing_required_utf_bom().
These functions just pass the pointer to has_bom_prefix().

In has_bom_prefix() we pass the pointer to memcmp() which is undefined
for null pointers. I think that is what Coverity is complaining about,
right?


- Lars

Re: What's cooking in git.git (Mar 2018, #05; Wed, 28)

[Lars Schneider]

> On 30 Mar 2018, at 11:24, Ævar Arnfjörð Bjarmason  wrote:
> 
> 
> On Wed, Mar 28 2018, Junio C. Hamano wrote:
> 
>> * ls/checkout-encoding (2018-03-16) 10 commits
>> - convert: add round trip check based on 'core.checkRoundtripEncoding'
>> - convert: add tracing for 'working-tree-encoding' attribute
>> - convert: check for detectable errors in UTF encodings
>> - convert: add 'working-tree-encoding' attribute
>> - utf8: add function to detect a missing UTF-16/32 BOM
>> - utf8: add function to detect prohibited UTF-16/32 BOM
>> - utf8: teach same_encoding() alternative UTF encoding names
>> - strbuf: add a case insensitive starts_with()
>> - strbuf: add xstrdup_toupper()
>> - strbuf: remove unnecessary NUL assignment in xstrdup_tolower()
>> 
>> The new "checkout-encoding" attribute can ask Git to convert the
>> contents to the specified encoding when checking out to the working
>> tree (and the other way around when checking in).
> 
> There's an issue in ca16fc3635 ("convert: check for detectable errors in
> UTF encodings", 2018-03-15) flagged by Coverity:
> https://public-inbox.org/git/CAGZ79kbAOcwaRzjuMtZ_HVsYvUr_7UAPbOcnrmPgsdE19q=p...@mail.gmail.com/

Thanks a lot for pointing me at this!
I'll prepare a new round soonish.

- Lars

Re: What's cooking in git.git (Mar 2018, #05; Wed, 28)

[Ævar Arnfjörð Bjarmason]

On Wed, Mar 28 2018, Junio C. Hamano wrote:

> * ls/checkout-encoding (2018-03-16) 10 commits
>  - convert: add round trip check based on 'core.checkRoundtripEncoding'
>  - convert: add tracing for 'working-tree-encoding' attribute
>  - convert: check for detectable errors in UTF encodings
>  - convert: add 'working-tree-encoding' attribute
>  - utf8: add function to detect a missing UTF-16/32 BOM
>  - utf8: add function to detect prohibited UTF-16/32 BOM
>  - utf8: teach same_encoding() alternative UTF encoding names
>  - strbuf: add a case insensitive starts_with()
>  - strbuf: add xstrdup_toupper()
>  - strbuf: remove unnecessary NUL assignment in xstrdup_tolower()
>
>  The new "checkout-encoding" attribute can ask Git to convert the
>  contents to the specified encoding when checking out to the working
>  tree (and the other way around when checking in).

There's an issue in ca16fc3635 ("convert: check for detectable errors in
UTF encodings", 2018-03-15) flagged by Coverity:
https://public-inbox.org/git/CAGZ79kbAOcwaRzjuMtZ_HVsYvUr_7UAPbOcnrmPgsdE19q=p...@mail.gmail.com/

Re: What's cooking in git.git (Mar 2018, #05; Wed, 28)

[Stefan Beller]

>> * sb/packfiles-in-repository (2018-03-26) 12 commits
> ...
>>
>>  (this branch uses nd/remove-ignore-env-field and sb/object-store.)
>>
>>  Refactoring of the internal global data structure continues.
>>
>>  Waiting for a follow-up discussion.
>
> I'll review this one more closely and start a discussion.

Actually this looks good to me.

Re: What's cooking in git.git (Mar 2018, #05; Wed, 28)

[Stefan Beller]

On Wed, Mar 28, 2018 at 12:58 PM, Junio C Hamano  wrote:
>
> * ab/doc-hash-brokenness (2018-03-27) 2 commits
>  - doc hash-function-transition: clarify what SHAttered means
>  - doc hash-function-transition: clarify how older gits die on NewHash

This looked ready for next to me.

>
> * jk/branch-l-0-deprecation (2018-03-26) 3 commits
>  - branch: deprecate "-l" option
>  - t: switch "branch -l" to "branch --create-reflog"
>  - t3200: unset core.logallrefupdates when testing reflog creation
>  (this branch is used by jk/branch-l-1-removal and 
> jk/branch-l-2-reincarnation.)
>
>
> * jk/branch-l-1-removal (2018-03-26) 1 commit
>  - branch: drop deprecated "-l" option
>  (this branch is used by jk/branch-l-2-reincarnation; uses 
> jk/branch-l-0-deprecation.)
>
>
> * jk/branch-l-2-reincarnation (2018-03-26) 1 commit
>  - branch: make "-l" a synonym for "--list"
>  (this branch uses jk/branch-l-0-deprecation and jk/branch-l-1-removal.)

These three branches are interesting as all the work is done already.
We just have to wait? Anyway the first step looks good to me.

> * sb/blame-color (2018-02-13) 3 commits
>  - builtin/blame: highlight recently changed lines
>  - builtin/blame: add option to color metadata fields separately
>  - builtin/blame: dim uninteresting metadata
>
>  Expecting a reroll.
>  cf. https://public-inbox.org/git/20171110011002.10179-1-sbel...@google.com/#t
>  error messages are funny, can segfault, ...

I look into that again, I thought I had fixed the segfaults last time.

> * sb/object-store (2018-03-26) 27 commits
...
>  (this branch is used by sb/packfiles-in-repository; uses 
> nd/remove-ignore-env-field.)
>
>  Refactoring the internal global data structure to make it possible
>  to open multiple repositories, work with and then close them.
>
>  Rerolled by Duy on top of a separate preliminary clean-up topic.
>  The resulting structure of the topics looked very sensible.
>
>  Waiting for a follow-up discussion.

I had the impression the discussion had settled and this
would be ready for next.
https://public-inbox.org/git/xmqqy3iebpsw@gitster-ct.c.googlers.com/

> * sb/packfiles-in-repository (2018-03-26) 12 commits
...
>
>  (this branch uses nd/remove-ignore-env-field and sb/object-store.)
>
>  Refactoring of the internal global data structure continues.
>
>  Waiting for a follow-up discussion.

I'll review this one more closely and start a discussion.

> * bw/protocol-v2 (2018-03-15) 35 commits
>
>  The beginning of the next-gen transfer protocol.
>
>  Is everybody happy with this version?  One design decision with
>  larger consequence "to or not to build in?" has been settled in
>  favor of status quo, IIRC.

I skimmed over it and think it is ok to go.

> * en/rename-directory-detection (2018-02-27) 29 commits
>   (merged to 'next' on 2018-03-06 at d42470f86e)
...
>  Rename detection logic in "diff" family that is used in "merge" has
>  learned to guess when all of x/a, x/b and x/c have moved to z/a,
>  z/b and z/c, it is likely that x/d added in the meantime would also
>  want to move to z/d by taking the hint that the entire directory
>  'x' moved to 'z'.  A bug causing dirty files involved in a rename
>  to be overwritten during merge has also been fixed as part of this
>  work.
>
>  Will cook in 'next'.

I am excited about this one, despite having an issue this week
unrelated to this series, but related to rename detection.

The issue was an apparent rename from a tiny .c to a .java
file (also tiny), the license header was the large common part.

I wonder if we want get away from counting the lines that are
equal and instead want to go for some entropy based approach.
(When the license shows up in many files, they will become boring)

Another approach would be to give a "template" file to Git that knows
about which lines to exclude for the purpose of rename detection
(e.g. license, blank lines, indented braces, or such)

Thanks,
Stefan

Re: What's cooking in git.git (Mar 2018, #05; Wed, 28)

[Brandon Williams]

On 03/28, Junio C Hamano wrote:
> * bw/protocol-v2 (2018-03-15) 35 commits
>  - remote-curl: don't request v2 when pushing
>  - remote-curl: implement stateless-connect command
>  - http: eliminate "# service" line when using protocol v2
>  - http: don't always add Git-Protocol header
>  - http: allow providing extra headers for http requests
>  - remote-curl: store the protocol version the server responded with
>  - remote-curl: create copy of the service name
>  - pkt-line: add packet_buf_write_len function
>  - transport-helper: introduce stateless-connect
>  - transport-helper: refactor process_connect_service
>  - transport-helper: remove name parameter
>  - connect: don't request v2 when pushing
>  - connect: refactor git_connect to only get the protocol version once
>  - fetch-pack: support shallow requests
>  - fetch-pack: perform a fetch using v2
>  - upload-pack: introduce fetch server command
>  - push: pass ref prefixes when pushing
>  - fetch: pass ref prefixes when fetching
>  - ls-remote: pass ref prefixes when requesting a remote's refs
>  - transport: convert transport_get_remote_refs to take a list of ref prefixes
>  - transport: convert get_refs_list to take a list of ref prefixes
>  - connect: request remote refs using v2
>  - ls-refs: introduce ls-refs server command
>  - serve: introduce git-serve
>  - test-pkt-line: introduce a packet-line test helper
>  - protocol: introduce enum protocol_version value protocol_v2
>  - transport: store protocol version
>  - connect: discover protocol version outside of get_remote_heads
>  - connect: convert get_remote_heads to use struct packet_reader
>  - transport: use get_refs_via_connect to get refs
>  - upload-pack: factor out processing lines
>  - upload-pack: convert to a builtin
>  - pkt-line: add delim packet support
>  - pkt-line: allow peeking a packet line without consuming it
>  - pkt-line: introduce packet_read_with_status
> 
>  The beginning of the next-gen transfer protocol.
> 
>  Is everybody happy with this version?  One design decision with
>  larger consequence "to or not to build in?" has been settled in
>  favor of status quo, IIRC.

I haven't heard anymore complaints with this version so I *think* it
should be good.

-- 
Brandon Williams

What's cooking in git.git (Mar 2018, #05; Wed, 28)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed with
'-' are only in 'pu' (proposed updates) while commits prefixed with
'+' are in 'next'.  The ones marked with '.' do not appear in any of
the integration branches, but I am still holding onto them.

Git 2.17 final is expected to be tagged by early next week.

You can find the changes described here in the integration branches
of the repositories listed at

http://git-blame.blogspot.com/p/git-public-repositories.html

--
[Graduated to "master"]

* bp/refresh-cache-ent-rehash-fix (2018-03-15) 1 commit
  (merged to 'next' on 2018-03-15 at bac8745f08)
 + Fix bugs preventing adding updated cache entries to the name hash

 The codepath to replace an existing entry in the index had a bug in
 updating the name hash structure, which has been fixed.


* dp/merge-strategy-doc-fix (2018-03-19) 1 commit
  (merged to 'next' on 2018-03-20 at 317e077588)
 + Documentation/merge-strategies: typofix

 Doc fix.


* jh/fsck-promisors (2018-03-13) 1 commit
  (merged to 'next' on 2018-03-15 at 0c283dbe5e)
 + sha1_file: restore OBJECT_INFO_QUICK functionality

 A hotfix to a topic that graduated recently.


* jk/attributes-path-doc (2018-03-20) 1 commit
  (merged to 'next' on 2018-03-20 at e965f0c68c)
 + doc/gitattributes: mention non-recursive behavior

 Doc update.


* js/ming-strftime (2018-03-19) 1 commit
  (merged to 'next' on 2018-03-20 at a9ca8172c7)
 + mingw: abort on invalid strftime formats


* jt/transfer-fsck-with-promissor (2018-03-15) 2 commits
  (merged to 'next' on 2018-03-15 at 6d1ccc965b)
 + fetch-pack: do not check links for partial fetch
 + index-pack: support checking objects but not links

 The transfer.fsckobjects configuration tells "git fetch" to
 validate the data and connected-ness of objects in the received
 pack; the code to perform this check has been taught about the
 narrow clone's convention that missing objects that are reachable
 from objects in a pack that came from a promissor remote is OK.


* ks/t3200-typofix (2018-03-15) 1 commit
  (merged to 'next' on 2018-03-15 at 8b8d397787)
 + t/t3200: fix a typo in a test description

 Test typofix.


* ma/config-page-only-in-list-mode (2018-02-21) 3 commits
  (merged to 'next' on 2018-03-15 at 652430af12)
 + config: change default of `pager.config` to "on"
 + config: respect `pager.config` in list/get-mode only
 + t7006: add tests for how git config paginates

 In a way similar to how "git tag" learned to honor the pager
 setting only in the list mode, "git config" learned to ignore the
 pager setting when it is used for setting values (i.e. when the
 purpose of the operation is not to "show").


* ma/skip-writing-unchanged-index (2018-03-01) 1 commit
  (merged to 'next' on 2018-03-15 at cdbbc66464)
 + write_locked_index(): add flag to avoid writing unchanged index

 Internal API clean-up to allow write_locked_index() optionally skip
 writing the in-core index when it is not modified.


* ml/filter-branch-portability-fix (2018-03-19) 1 commit
  (merged to 'next' on 2018-03-20 at c7c17cfc8b)
 + filter-branch: use printf instead of echo -e

 Shell script portability fix.


* nd/parseopt-completion (2018-03-23) 2 commits
  (merged to 'next' on 2018-03-23 at 2bee77135e)
 + t9902: disable test on the list of merge-strategies under GETTEXT_POISON
  (merged to 'next' on 2018-03-22 at 279765c437)
 + completion: clear cached --options when sourcing the completion script
 (this branch is used by nd/parseopt-completion-more.)

 Hotfix for recently graduated topic that give help to completion
 scripts from the Git subcommands that are being completed


* pc/submodule-helper (2018-03-27) 1 commit
  (merged to 'next' on 2018-03-27 at 362e0ef09b)
 + submodule deinit: handle non existing pathspecs gracefully

 Hotfix.


* rj/http-code-cleanup (2018-03-15) 1 commit
  (merged to 'next' on 2018-03-15 at 0dfd462ff8)
 + http: fix an unused variable warning for 'curl_no_proxy'

 There was an unused file-scope static variable left in http.c when
 building for versions of libCURL that is older than 7.19.4, which
 has been fixed.

 This will become unnecessary, when we follow-through the
 jk/drop-ancient-curl topic.


* rj/warning-uninitialized-fix (2018-03-20) 2 commits
  (merged to 'next' on 2018-03-20 at 9ac9d02b0b)
 + read-cache: fix an -Wmaybe-uninitialized warning
 + -Wuninitialized: remove some 'init-self' workarounds

 Compilation fix.


* tg/stash-doc-typofix (2018-03-27) 1 commit
  (merged to 'next' on 2018-03-27 at 144a25f09c)
 + git-stash.txt: remove extra square bracket

 Hotfix.


* tz/complete-tag-delete-tagname (2018-03-19) 1 commit
  (merged to 'next' on 2018-03-20 at d63d45ff16)
 + completion: complete tags with git tag --delete/--verify


* tz/relnotes-1.7-on-perl (2018-03-16) 1 commit
  (merged to 'next' on 2018-03-20 at ed4b26e581)
 + RelNotes: add details on Perl module changes

--
[New