Re: What's cooking in git.git (Nov 2018, #06; Wed, 21)

2018-11-22 Thread Linus Torvalds
On Wed, Nov 21, 2018 at 6:08 PM Stephen P. Smith wrote: > > Picking up someones stalled patch is one thing, picking up Linus' patch is in > a different league. No, I think it works the other way - my random patches are likely the _least_ important ones, simply because I can carry them around in

Re: Hash algorithm analysis

2018-09-18 Thread Linus Torvalds
On Tue, Sep 18, 2018 at 8:18 AM Joan Daemen wrote: > > 3) The relatively large state in the sponge construction increases the > generic strength against attacks when the input contains redundancy or > has a certain form. For instance, if the input is restricted to be text in > ASCII (such as

Re: "less -F" is broken

2018-08-16 Thread Linus Torvalds
On Thu, Aug 16, 2018 at 9:50 AM Mark Nudelman wrote: > > So I'm not sure what the best solution is. Linus's proposal to disable > the line counting stuff if -X is set seems reasonable. I will look into > that and see if there are any issues with it. One option that I didn't try to go for -

Re: "less -F" is broken

2018-08-15 Thread Linus Torvalds
On Wed, Aug 15, 2018 at 2:29 PM Ævar Arnfjörð Bjarmason wrote: > > Downloading & trying versions of it locally reveals that it's as of > version 520, not 530. The last version before 520 is 487. Presumably > it's covered by this item in the changelog: > > Don't output terminal init sequence

Re: "less -F" is broken

2018-08-15 Thread Linus Torvalds
On Wed, Aug 15, 2018 at 2:29 PM Ævar Arnfjörð Bjarmason wrote: > > FWIW they're not linked from > http://www.greenwoodsoftware.com/less/download.html but you can just URL > hack and see releases http://www.greenwoodsoftware.com/less/ and change > links like

"less -F" is broken

2018-08-15 Thread Linus Torvalds
Sadly, as of less-530, the behavior of "less -F" is broken enough that I think git needs to potentially think about changing the defaults for the pager, or people should at least be aware of it. Older versions of less (at least up to less-487 - March 2017) do not have this bug. There were

Re: [PATCH v2 2/2] doc hash-function-transition: pick SHA-256 as NewHash

2018-08-03 Thread Linus Torvalds
On Fri, Aug 3, 2018 at 9:40 AM Junio C Hamano wrote: > > > [...] > >> - - 20-byte NewHash checksum of all of the above. > >> + - 20-byte SHA-256 checksum of all of the above. > > > > Likewise. > > Hmph, I've always assumed since NewHash plan was written that this > part was not about tamper

Re: [PATCH v2 2/2] doc hash-function-transition: pick SHA-256 as NewHash

2018-08-03 Thread Linus Torvalds
On Fri, Aug 3, 2018 at 12:20 AM Jonathan Nieder wrote: > > > Here I'd want to put a pile of acks, e.g.: > > Acked-by: Linus Torvalds > .. Sure, feel free to add my Ack for this. Linus

Re: [RFC PATCH v2] Add 'human' date format

2018-07-24 Thread Linus Torvalds
On Tue, Jul 24, 2018 at 2:49 PM Junio C Hamano wrote: > > But lack of TZ does not give me enough hint about which content it > happened. The fact that this was done late at night on weekend is > indeed interesting, and I may not care what time it locally was for > me, so perhaps this is an

Re: Hash algorithm analysis

2018-07-24 Thread Linus Torvalds
On Tue, Jul 24, 2018 at 12:01 PM Edward Thomson wrote: > > Switching gears, if I look at this from the perspective of the libgit2 > project, I would also prefer SHA-256 or SHA3 over blake2b. To support > blake2b, we'd have to include - and support - that code ourselves. But > to support

Re: Hash algorithm analysis

2018-07-23 Thread Linus Torvalds
On Mon, Jul 23, 2018 at 5:48 AM Sitaram Chamarty wrote: > > I would suggest (a) hash size of 256 bits and (b) choice of any hash > function that can produce such a hash. If people feel strongly that 256 > bits may also turn out to be too small (really?) then a choice of 256 or > 512, but not

Re: Hash algorithm analysis

2018-07-21 Thread Linus Torvalds
On Sat, Jul 21, 2018 at 3:39 PM Johannes Schindelin wrote: > > Do you really want to value contributors' opinion more than > cryptographers'? I mean, that's exactly what got us into this hard-coded > SHA-1 mess in the first place. Don't be silly. Other real cryptographers consider SHA256 to be

Re: [RFC PATCH v2] Add 'human' date format

2018-07-11 Thread Linus Torvalds
On Wed, Jul 11, 2018 at 2:24 PM Ævar Arnfjörð Bjarmason wrote: > > I think that's true for the likes of linux.git & git.git, but a lot of > users of git say work in some corporate setting entirely or mostly in > the same timezone. > > In that case, knowing if some commit whose sole message was

Re: [RFC PATCH v2] Add 'human' date format

2018-07-11 Thread Linus Torvalds
[ Trying to come up with crazy special cases ] On Wed, Jul 11, 2018 at 1:49 PM Linus Torvalds wrote: > > But it could be anything else invalid, of course. It could be MAX_INT > or something like that. That might be better. A timezone of -1 isn't actually a valid timezone, but I guess

Re: [RFC PATCH v2] Add 'human' date format

2018-07-11 Thread Linus Torvalds
On Wed, Jul 11, 2018 at 1:34 PM Andrei Rybak wrote: > > > + int human_tz = -1; > > Is -1 an OK initial value for timezone if local_time_tzoffset returns > negative values as well? It looks like it doesn't matter for from functional The value was intentionally picked to *not* be a valid

[RFC PATCH v2] Add 'human' date format

2018-07-07 Thread Linus Torvalds
From: Linus Torvalds This adds --date=human, which skips the timezone if it matches the current time-zone, and doesn't print the whole date if that matches (ie skip printing year for dates that are "this year", but also skip the whole date itself if it's in the last few days and w

Re: [RFC PATCH] Add 'human' date format

2018-07-07 Thread Linus Torvalds
On Sat, Jul 7, 2018 at 12:58 PM Linus Torvalds wrote: > > I'm playing with making all "today" dates just use the relative > format. Here's the incremental patch for that if people want to compare the output. With this, you never get the "just time" case, because tha

Re: [RFC PATCH] Add 'human' date format

2018-07-07 Thread Linus Torvalds
On Sat, Jul 7, 2018 at 12:39 PM Linus Torvalds wrote: > to me, but with "--date=human", right now it just says > > Date: 12:21 Side note: this is probably my least favorite of the formats. I'm playing with making all "today" dates just use the relative f

[RFC PATCH] Add 'human' date format

2018-07-07 Thread Linus Torvalds
From: Linus Torvalds This adds --date=human, which skips the timezone if it matches the current time-zone, and doesn't print the whole date if that matches (ie skip printing year for dates that are "this year", but also skip the whole date itself if it's in the last few days and w

Re: Hash algorithm analysis

2018-06-11 Thread Linus Torvalds
On Mon, Jun 11, 2018 at 4:27 PM Ævar Arnfjörð Bjarmason wrote: > > > > And no, I'm not a cryptographer. But honestly, length extension > > attacks were how both md5 and sha1 were broken in practice, so I'm > > just going "why would we go with a crypto choice that has that known > > weakness?

Re: Hash algorithm analysis

2018-06-11 Thread Linus Torvalds
On Mon, Jun 11, 2018 at 12:29 PM Jonathan Nieder wrote: > > Yves Orton and Linus Torvalds prefer[5] SHA3 over SHA2 because of how > it is constructed. Yeah, I really think that it's a mistake to switch to something that has the same problem SHA1 had. That doesn't necessarily

Re: [RFC] git gc "--prune=now" semantics considered harmful

2018-06-01 Thread Linus Torvalds
On Fri, Jun 1, 2018 at 2:04 AM Jeff King wrote: > > We'd also accept relative times like "5.minutes.ago" (in fact, the > default is a relative 2.weeks.ago, though it's long enough that the > difference between "2 weeks" and "2 weeks plus 5 minutes" may not matter > much). So we probably ought to

Re: [RFC] git gc "--prune=now" semantics considered harmful

2018-05-26 Thread Linus Torvalds
On Sat, May 26, 2018 at 4:31 PM Junio C Hamano wrote: > *That* is something I don't do. After all, I am fully aware that I > have started end-of-day ritual by that time, so I won't even look at > a new patch (or a pull request for that matter). Sounds like you're more

[RFC] git gc "--prune=now" semantics considered harmful

2018-05-26 Thread Linus Torvalds
So this is a RFC patch, I'm not sure how much people really care, but I find the current behavior of "git gc --prune=now" to be unnecessarily dangerous. There's two issues with it: (a) parse_expiry_date() considers "now" to be special, and it actually doesn't mean "now" at all, it means

Re: Silly "git gc" UI issue.

2018-04-18 Thread Linus Torvalds
On Wed, Apr 18, 2018 at 7:16 PM, Junio C Hamano wrote: > A few commands that parse --expire= command line option > behaves silly when given nonsense input. For example So this patch definitely improves on the error message. But look at what happens for the kernel:

Re: Silly "git gc" UI issue.

2018-04-18 Thread Linus Torvalds
On Wed, Apr 18, 2018 at 6:52 PM, Junio C Hamano wrote: > > Regardless of your originai "git gc" issue, we should make "prune" > say something on this error. And when we do so, I would think that > error message will come before the final "error: failed to run > prune". So to

Silly "git gc" UI issue.

2018-04-18 Thread Linus Torvalds
Ok, this is ridiculous, but I've done it several times, so I thought I'd finally mention it to somebody on the git list that may care: "My name is Linus, and I'm a klutz". what does that have to do with anything? Now, imagine you're a klutz. Imagine you want to clean up your .git directory.

Re: Optimizing writes to unchanged files during merges?

2018-04-15 Thread Linus Torvalds
On Sun, Apr 15, 2018 at 6:44 PM, Junio C Hamano wrote: > > I think Elijah's corrected was_tracked() also does not care "has > this been renamed". I'm perfectly happy with the slightly smarter patches. My patch was really just an RFC and because I had tried it out. > One thing

Re: Optimizing writes to unchanged files during merges?

2018-04-13 Thread Linus Torvalds
On Fri, Apr 13, 2018 at 10:39 AM, Stefan Beller wrote: > > Would s/read/xread/ make sense in working_tree_matches ? Makes sense, yes. That patch was really more of a RFD than anything that should be applied. I would like to see the "might be same" flag pushed down so that

Re: Optimizing writes to unchanged files during merges?

2018-04-13 Thread Linus Torvalds
On Fri, Apr 13, 2018 at 12:02 AM, Elijah Newren wrote: > > I hope you don't mind me barging into your conversation I was getting tired of my own rambling anyway.. > However, it turns out we have this awesome function called > "was_tracked(const char *path)" that was intended

Re: Optimizing writes to unchanged files during merges?

2018-04-12 Thread Linus Torvalds
[ Still talking to myself. Very soothing. ] On Thu, Apr 12, 2018 at 4:55 PM, Linus Torvalds <torva...@linux-foundation.org> wrote: > [ Talking to myself ] > > Did it perhaps mean to say > > path_renamed_outside_HEAD = path2 && !strcmp(path, p

Re: Optimizing writes to unchanged files during merges?

2018-04-12 Thread Linus Torvalds
[ Talking to myself ] On Thu, Apr 12, 2018 at 4:41 PM, Linus Torvalds <torva...@linux-foundation.org> wrote: > > Oddly, that *already* has the check: > > if (mfi.clean && !df_conflict_remains && > oid_eq(, a_oid) &&

Re: Optimizing writes to unchanged files during merges?

2018-04-12 Thread Linus Torvalds
On Thu, Apr 12, 2018 at 4:35 PM, Linus Torvalds <torva...@linux-foundation.org> wrote: > > in process_entry(), and I think we could just there add a test for if > o_old,o_mod == a_oid,a_mode or something? Actually, not process_entry, but merge_content(). Oddly, that *already

Re: Optimizing writes to unchanged files during merges?

2018-04-12 Thread Linus Torvalds
On Thu, Apr 12, 2018 at 4:17 PM, Junio C Hamano wrote: > > A bit of detour. "Change in side branch happened to be a subset of > the change in trunk and gets subsumed, but we end up writing the > same result" happens also with the simpler resolve strategy. > > Here is a fix.

Re: Optimizing writes to unchanged files during merges?

2018-04-12 Thread Linus Torvalds
On Thu, Apr 12, 2018 at 2:46 PM, Junio C Hamano wrote: > > Thanks for a clear description of the issue. It does sound > interesting. I decided to show it with a simpler case that could be scripted and doesn't need the kernel. NOTE! This obviously doesn't happen for files

Optimizing writes to unchanged files during merges?

2018-04-12 Thread Linus Torvalds
So I just had an interesting experience that has happened before too, but this time I decided to try to figure out *why* it happened. I'm obviously in the latter part of the kernel merge window, and things are slowly calming down. I do the second XFS merge during this window, and it brings in

Re: [PATCH 3/3] Makefile: optionally symlink libexec/git-core binaries to bin/git

2018-03-19 Thread Linus Torvalds
On Mon, Mar 19, 2018, 04:34 Johannes Schindelin wrote: > > This is a real problem. No it isn't. We already handle those special cases specially, and install them in the bin directory (as opposed to libexec). And it all works fine. Look into the bin directory some

Re: [PATCH 3/3] Makefile: optionally symlink libexec/git-core binaries to bin/git

2018-03-15 Thread Linus Torvalds
On Thu, Mar 15, 2018 at 10:05 AM, Johannes Schindelin wrote: > The most sensible thing, of course, would be to *not* link the builtins at > all. I mean, we deprecated the dashed form (which was a design mistake, > whether you admit it or not) a long time ago. That's

Re: [PATCH 3/3] Makefile: optionally symlink libexec/git-core binaries to bin/git

2018-03-14 Thread Linus Torvalds
On Wed, Mar 14, 2018 at 3:14 AM, Ævar Arnfjörð Bjarmason wrote: > On Wed, Mar 14 2018, Johannes Sixt jotted: >> >> It is important to leave the default at hard-linking the binaries, >> because on Windows symbolic links are second class citizens (they >> require special

Re: [PATCH] revision: drop --show-all option

2018-02-21 Thread Linus Torvalds
On Wed, Feb 21, 2018 at 3:27 PM, Jeff King wrote: > > We'll skip the usual deprecation period because this was > explicitly a debugging aid that was never documented. Ack. I don't think I've used it since, and probably nobody else ever used it. Linus

Re: linux-next: unnecessary merge in the v4l-dvb tree

2018-02-13 Thread Linus Torvalds
On Tue, Feb 13, 2018 at 9:18 AM, Junio C Hamano wrote: > > That makes me wonder if another heuristic I floated earlier is more > appropriate. When merging a tag object T, if refs/tags/T exists and > it is that tag object, then an updated "merge" would default to "--ff"; >

Re: linux-next: unnecessary merge in the v4l-dvb tree

2018-02-12 Thread Linus Torvalds
On Mon, Feb 12, 2018 at 1:44 PM, Junio C Hamano wrote: > > But I wonder why "update to upstream" is merging a signed tag in the > first place. Wouldn't downstream's "try to keep up with" pull be > grabbing from branch tips, not tags? I'm actually encouraging maintainers to

Re: linux-next: unnecessary merge in the v4l-dvb tree

2018-02-12 Thread Linus Torvalds
On Mon, Feb 12, 2018 at 1:15 PM, Linus Torvalds <torva...@linux-foundation.org> wrote: > > The reasoning is to avoid losing the signature from the tag (when > merging a signed tag, the signature gets inserted into the merge > commit itself - use "git log --show-signature&

Re: linux-next: unnecessary merge in the v4l-dvb tree

2018-02-12 Thread Linus Torvalds
On Mon, Feb 12, 2018 at 1:00 PM, Stephen Rothwell wrote: > > Linus, this happens a bit after the merge window, so I am wondering > about the rational of not doing a fast forward merge when merging a > signed tag (I forget the reasoning). The reasoning is to avoid losing

Left-over COMMIT_EDITMSG file in gitdir

2018-02-08 Thread Linus Torvalds
This may be intentional, but if so, it's not obvious.. Back long long ago, the original "git commit" shell script got rewritten in C. In that rewrite, removing some temporary files seems to have been left out. At least one: .git/COMMIT_EDITMSG. In the original commit.sh shell script, we can

Re: [PATCH] enable core.fsyncObjectFiles by default

2018-01-17 Thread Linus Torvalds
On Wed, Jan 17, 2018 at 3:52 PM, Theodore Ts'o wrote: > > Well, let's be fair; this is something *ext3* got wrong, and it was > the default file system back them. I'm pretty sure reiserfs and btrfs did too.. Linus

Re: [PATCH] enable core.fsyncObjectFiles by default

2018-01-17 Thread Linus Torvalds
On Wed, Jan 17, 2018 at 3:16 PM, Ævar Arnfjörð Bjarmason wrote: > > Or does overall FS activity and raw throughput (e.g. with an ISO copy) > matter more than general FS contention? Traditionally, yes. Also note that none of this is about "throughput". It's about waiting for a

Re: [PATCH] enable core.fsyncObjectFiles by default

2018-01-17 Thread Linus Torvalds
On Wed, Jan 17, 2018 at 2:07 PM, Linus Torvalds <torva...@linux-foundation.org> wrote: > > The original git design was very much to write each object file > without any syncing, because they don't matter since a new object file > - by definition - isn't really reachable. Then s

Re: [PATCH] enable core.fsyncObjectFiles by default

2018-01-17 Thread Linus Torvalds
On Wed, Jan 17, 2018 at 1:44 PM, Ævar Arnfjörð Bjarmason wrote: > > I ran a small test myself on CentOS 7 (3.10) with ext4 data=ordered > on the tests I thought might do a lot of loose object writes: > > $ GIT_PERF_REPEAT_COUNT=10 GIT_PERF_LARGE_REPO=~/g/linux >

Re: git pull

2017-11-19 Thread Linus Torvalds
On Sun, Nov 19, 2017 at 7:37 PM, Junio C Hamano wrote: >> ... >> Which is simple. Just create a .git/hooks/prepare-commit-msg file that >> contains >> >> #!/bin/sh >> sed -i 's|ssh://gitolite.kernel.org/|git://git.kernel.org/|g' "$1" >> >> and make it executable, and git

Re: RFC v3: Another proposed hash function transition plan

2017-10-02 Thread Linus Torvalds
On Mon, Oct 2, 2017 at 7:00 AM, Jason Cooper wrote: > > Ahhh, so if I understand you correctly, you'd prefer SHA-256 over > SHA3-256 because it's more performant for your usecase? Well, that's a > completely different animal that cryptographic suitability. In almost all

Re: [PATCH] diff: correct newline in summary for renamed files

2017-09-27 Thread Linus Torvalds
On Wed, Sep 27, 2017 at 2:58 PM, Stefan Beller wrote: > > Linus, I assumed your sign off for the original patch. Thanks for spotting. > > Adding the mode change to t4016 seems like the easiest way to test it. Looks good to me, and you don't need to give me authorship

Re: diffstat summary mode change bug

2017-09-27 Thread Linus Torvalds
On Wed, Sep 27, 2017 at 1:40 PM, Stefan Beller wrote: > > I disagree with this analysis, as the fix you propose adds the > new line unconditionally, i.e. this code path would be broken > regardless of "show filename or not". Right. Because it is what we want. The old code

diffstat summary mode change bug

2017-09-27 Thread Linus Torvalds
Current git shows file-mode changes incorrectly in the diffstat summary, as I just noted from a pull request I did on the kernel. The pull request *should* have resulted in a summary like this: ... 21 files changed, 247 insertions(+), 67 deletions(-) mode change 100644 => 100755

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Linus Torvalds
On Wed, Sep 13, 2017 at 6:43 AM, demerphq wrote: > > SHA3 however uses a completely different design where it mixes a 1088 > bit block into a 1600 bit state, for a leverage of 2:3, and the excess > is *preserved between each block*. Yes. And considering that the SHA1 attack

Re: BUG: attempt to trim too many characters

2017-09-12 Thread Linus Torvalds
but normal people would obviously do "gitk --bisect" that then does that "git rev-parse" internally and shows a UI error box instead). Linus On Tue, Sep 5, 2017 at 3:03 PM, Jeff King <p...@peff.net> wrote: > On Tue, Sep 05, 2017 at 02:55:08PM -0700, Linus T

Re: BUG: attempt to trim too many characters

2017-09-05 Thread Linus Torvalds
On Tue, Sep 5, 2017 at 3:03 PM, Jeff King wrote: > > This probably fixes it: Yup. Thanks. Linus

Re: BUG: attempt to trim too many characters

2017-09-05 Thread Linus Torvalds
On Tue, Sep 5, 2017 at 2:50 PM, Jeff King wrote: > > What version of git are you running? This should be fixed by 03df567fbf > (for_each_bisect_ref(): don't trim refnames, 2017-06-18) which is in > v2.14. I'm way more recent than 2.14. I'm at commit 238e487ea ("The fifth batch

BUG: attempt to trim too many characters

2017-09-05 Thread Linus Torvalds
I just got this with gitk --bisect while doing some bisection on my current kernel. It happens with "git rev-parse --bisect" too, but interestingly, "git log --bisect" works fine. I have not tried to figure out anything further, except that it was introduced by commit b9c8e7f2f

Re: [PATCH] strbuf: let strbuf_addftime handle %z and %Z itself

2017-06-13 Thread Linus Torvalds
On Mon, Jun 12, 2017 at 2:10 PM, Jeff King wrote: > > I suspect nobody has complained because we generally encourage real > "-0800" names when specifying zones. That's what any sane person uses, and it's what SMTP requiries. The timezone names are a (bad) joke. If a human can't

Re: Unaligned accesses in sha1dc

2017-06-02 Thread Linus Torvalds
On Fri, Jun 2, 2017 at 1:17 PM, demerphq wrote: > Most hash function implementations have code like the following > (extracted and reduced from hv_macro.h in perl.git [which only > supports little-endian hash functions]): Yes. Please do *not* try to make things overly

Re: git merges of tags

2017-05-19 Thread Linus Torvalds
On Thu, May 18, 2017 at 4:23 PM, Stephen Rothwell wrote: > > Just a reminder that if you are merging Linus' tree (or any tree > really) via a tag, git was changed some time ago so that merging a tag > will not do a fast forward (there is a good reason for this - I just >

Re: [TANGENT] run-command: use vfork instead of fork

2017-05-16 Thread Linus Torvalds
On Tue, May 16, 2017 at 12:35 PM, Eric Wong wrote: > > Fwiw, most of the vfork preparation was already done by Brandon > and myself a few weeks ago, and cooking in pu. Oh, interesting. Was that done for vfork(), or is it for something else? Some of the changes seem almost overly

Re: git rebase regression: cannot pass a shell expression directly to --exec

2017-05-16 Thread Linus Torvalds
On Tue, May 16, 2017 at 1:12 PM, Johannes Schindelin wrote: >> >> I think it would be better to just >> >> (a) get rid of the magic strcspn() entirely >> >> (b) make the 'can we optimize this' test be simply just looking up >> 'argv[0]' in $PATH > > What about > >

Re: git rebase regression: cannot pass a shell expression directly to --exec

2017-05-16 Thread Linus Torvalds
On Tue, May 16, 2017 at 10:23 AM, Jeff King wrote: > > I think the logic here would be more like: > > 1. During prepare_shell_cmd(), even if we optimize out the shell call, > still prepare a fallback argv (since we can't allocate memory > post-fork). > > 2. In the

[PATCH] Fix 'git am' in-body header continuations

2017-04-02 Thread Linus Torvalds
From: Linus Torvalds <torva...@linux-foundation.org> Date: Sat, 1 Apr 2017 12:14:39 -0700 Subject: [PATCH] Fix 'git am' in-body header continuations An empty line should stop any pending in-body headers, and start the actual body parsing. This also modifies the original test for the i

Re: Bug in "git am" when the body starts with spaces

2017-04-01 Thread Linus Torvalds
On Fri, Mar 31, 2017 at 5:52 PM, Linus Torvalds <torva...@linux-foundation.org> wrote: > > The continuation logic is oddly complex, and I can't follow the logic. > But it is completely broken in how it thinks empty lines are somehow > "continuations". The attache

Re: Bug in "git am" when the body starts with spaces

2017-03-31 Thread Linus Torvalds
ons". Jonathan? Linus On Fri, Mar 31, 2017 at 5:24 PM, Linus Torvalds <torva...@linux-foundation.org> wrote: > > I think the reason is that the "header continuation line" logic kicks > in because the lines in the body start with spaces, but that's > entirely incorr

Bug in "git am" when the body starts with spaces

2017-03-31 Thread Linus Torvalds
check how far back this goes, I guess I'll do that next. But I thought I'd report it here first in case somebody else goes "ahhh". Linus From ad65cf7ba97ac071da1f845ec854165e7bf1efdf Mon Sep 17 00:00:00 2001 From: Linus Torvalds <torva...@linux-foundation.org> Date:

Re: USE_SHA1DC is broken in pu

2017-03-23 Thread Linus Torvalds
On Thu, Mar 23, 2017 at 9:43 AM, Johannes Schindelin wrote: > > What I am saying is that this should be a more fine-grained, runtime knob. No it really shouldn't. > If I write out an index, I should not suffer the slowdown from detecting > collisions. The index case

Re: [PATCH 0/2] Re-integrate sha1dc

2017-03-16 Thread Linus Torvalds
On Thu, Mar 16, 2017 at 3:04 PM, Jeff King wrote: > > There are a few things I think are worth changing. The die() message > should mention the sha1 we computed. That will be a big help if an old > version of git tries to unknowingly push a colliding object to a newer > version.

[PATCH 2/2] Integrate the sha1dc code with the git build

2017-03-16 Thread Linus Torvalds
From: Linus Torvalds <torva...@linux-foundation.org> Date: Thu, 16 Mar 2017 13:08:38 -0700 Subject: [PATCH 2/2] Integrate the sha1dc code with the git build This adds the proper magic to actually build the sha1dc code as part of git when USE_SHA1DC is enabled. This includes - adj

Re: USE_SHA1DC is broken in pu

2017-03-16 Thread Linus Torvalds
On Thu, Mar 16, 2017 at 12:51 PM, Linus Torvalds <torva...@linux-foundation.org> wrote: > > I'll send a patch on top of 'next', which already has the header file changes. Patches sent. It all looked fairly straightforward to me, but maybe I missed something. Linus

[PATCH 0/2] Re-integrate sha1dc

2017-03-16 Thread Linus Torvalds
I suspect the first patch will not make it to the list since it's over 100kB in size, but oh well.. Junio and Jeff will see it. This is sent as two patches, just to have the original upstream code as a first step, and then the second patch does the small modifications to integrate it with

Re: USE_SHA1DC is broken in pu

2017-03-16 Thread Linus Torvalds
On Thu, Mar 16, 2017 at 12:46 PM, Junio C Hamano wrote: > > That's easy to answer. What we have on 'pu' is a fair game for > wholesale replacement. That is the whole point of not merging > topics in flux to 'next' and declaring that 'pu' will constantly > rewind. Ok. I'll

Re: USE_SHA1DC is broken in pu

2017-03-16 Thread Linus Torvalds
On Thu, Mar 16, 2017 at 12:41 PM, Jeff King wrote: > > Potentially we should just eject sha1dc from "pu" for the moment. It > needs re-rolled with the most recent version of the collision library > (and I see Marc just posted that they hit a stable point, which is > perhaps why

USE_SHA1DC is broken in pu

2017-03-16 Thread Linus Torvalds
I think there's a semantic merge error and it clashes with f18f816cb158 ("hash.h: move SHA-1 implementation selection into a header file"). Suggested possible merge resolution attached. Linus Makefile | 2 +- hash.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-)

Re: [PATCH] Put sha1dc on a diet

2017-03-13 Thread Linus Torvalds
On Mon, Mar 13, 2017 at 1:12 PM, Marc Stevens wrote: > Indeed, I've committed a fix, and a small bug fix for the new code just now. Unrelated side note: there may be some missing dependencies in the build infrastructure or something, because when I tried Jeff's script that

Re: Stable GnuPG interface, git should use GPGME

2017-03-10 Thread Linus Torvalds
On Fri, Mar 10, 2017 at 2:00 AM, Bernhard E. Reiter wrote: > > git uses an pipe-and-exec approach to running a GnuPG binary > as writen in the documentation [1]: > > gpg.program >Use this custom program instead of "gpg" found on $PATH when making >

Re: RFC: Another proposed hash function transition plan

2017-03-07 Thread Linus Torvalds
On Tue, Mar 7, 2017 at 10:57 AM, Ian Jackson wrote: > > Also I think you need to specify how abbreviated object names are > interpreted. One option might be to not use hex for the new hash, but base64 encoding. That would make the full size ASCII hash encoding

Re: RFC: Another proposed hash function transition plan

2017-03-06 Thread Linus Torvalds
On Mon, Mar 6, 2017 at 10:39 AM, Jonathan Tan wrote: > > I think "nohash" can be explained in 2 points: I do think that that was my least favorite part of the suggestion. Not just "nohash", but all the special "hash" lines too. I would honestly hope that the design

Re: Delta compression not so effective

2017-03-05 Thread Linus Torvalds
On Sat, Mar 4, 2017 at 12:27 AM, Marius Storm-Olsen wrote: > > I reran the repack with the options above (dropping the zlib=9, as you > suggested) > > $ time git -c pack.threads=4 repack -a -d -F \ >--window=350 --depth=250 --window-memory=30g > > and ended

Re: RFC: Another proposed hash function transition plan

2017-03-04 Thread Linus Torvalds
On Fri, Mar 3, 2017 at 5:12 PM, Jonathan Nieder wrote: > > This document is still in flux but I thought it best to send it out > early to start getting feedback. This actually looks very reasonable if you can implement it cleanly enough. In many ways the "convert entirely to

Re: SHA1 collisions found

2017-03-02 Thread Linus Torvalds
On Thu, Mar 2, 2017 at 5:50 PM, Mike Hommey wrote: > > What if the "object version" is a hash of the content (as opposed to > header + content like the normal git hash)? It doesn't actually matter for that attack. The concept of the attack is actually fairly simple: generate

Re: SHA1 collisions found

2017-03-02 Thread Linus Torvalds
On Thu, Mar 2, 2017 at 12:43 PM, Junio C Hamano wrote: > > My reaction heavily depends on how that "object version" thing > works. > > Would "object version" be like a truncated SHA-1 over the same data > but with different IV or something, i.e. something that guarantees >

Re: SHA1 collisions found

2017-03-02 Thread Linus Torvalds
On Thu, Mar 2, 2017 at 1:54 PM, Joey Hess wrote: > > There's a surprising result of combining iterated hash functions, that > the combination is no more difficult to attack than the strongest hash > function used. Duh. I should actually have known that. I started reading the

Re: SHA1 collisions found

2017-03-02 Thread Linus Torvalds
On Fri, Feb 24, 2017 at 4:39 PM, Linus Torvalds <torva...@linux-foundation.org> wrote: > > Honestly, I think that a primary goal for a new hash implementation > absolutely needs to be to minimize mixing. > > Not for security issues, but because of combinatorics. You w

Re: [PATCH] Put sha1dc on a diet

2017-03-02 Thread Linus Torvalds
On Thu, Mar 2, 2017 at 10:37 AM, Jeff Hostetler wrote: >> >> Now, if your _file_ index is 300-400MB (and I do think we check the >> SHA fingerprint on that even on just reading it - verify_hdr() in >> do_read_index()), then that's going to be a somewhat noticeable hit on

Re: [PATCH] Put sha1dc on a diet

2017-03-02 Thread Linus Torvalds
On Thu, Mar 2, 2017 at 6:45 AM, Johannes Schindelin wrote: > > It would probably make sense to switch the index integrity check away from > SHA-1 because we really only care about detecting bit flips there, and we > have no need for the computational overhead of using

Re: [PATCH] Put sha1dc on a diet

2017-03-01 Thread Linus Torvalds
On Wed, Mar 1, 2017 at 11:07 AM, Jeff King wrote: > > So obviously the smaller object size is nice, and the diffstat is > certainly satisfying. My only qualm would be whether this conflicts with > the optimizations that Dan is working on (probably not conceptually, but >

Re: Delta compression not so effective

2017-03-01 Thread Linus Torvalds
On Wed, Mar 1, 2017 at 4:12 PM, Marius Storm-Olsen wrote: > > No, the list of git verify-objects in the previous post was from the bottom > of the sorted list, so those are the largest blobs, ~249MB.. .. so with a 6GB window, you should easily sill have 20+ objects. Not a huge

Re: [PATCH] Put sha1dc on a diet

2017-03-01 Thread Linus Torvalds
On Wed, Mar 1, 2017 at 12:34 PM, Jeff King wrote: > > I don't think that helps. The sha1 over the pack-file takes about 1.3s > with openssl, and 5s with sha1dc. So we already know the increase there > is only a few seconds, not a few minutes. Yeah, I did a few statistics by adding

Re: [PATCH] Put sha1dc on a diet

2017-03-01 Thread Linus Torvalds
On Wed, Mar 1, 2017 at 2:51 PM, Johannes Schindelin wrote: > > But I think bigger than just developers on Windows OS. There are many > developers out there working on large repositories (yes, much larger than > Linux). Also using Macs and Linux. I am not at all sure

Re: [PATCH] Put sha1dc on a diet

2017-03-01 Thread Linus Torvalds
On Wed, Mar 1, 2017 at 1:56 PM, Johannes Schindelin wrote: > Footnote *1*: I know, it is easy to forget that some developers cannot > choose their tools, or even their hardware. In the past, we seemed to take > appropriate care, though. I don't think you need to worry

Re: Delta compression not so effective

2017-03-01 Thread Linus Torvalds
On Wed, Mar 1, 2017 at 9:57 AM, Marius Storm-Olsen wrote: > > Indeed, I did do a > -c pack.threads=20 --window-memory=6g > to 'git repack', since the machine is a 20-core (40 threads) machine with > 126GB of RAM. > > So I guess with these sized objects, even at 6GB per

Re: [PATCH] Put sha1dc on a diet

2017-03-01 Thread Linus Torvalds
On Wed, Mar 1, 2017 at 10:42 AM, Junio C Hamano wrote: > I see //c99 comments sha1dc is already full of // style comments. I just followed the existing practice. > and also T array[] = { [58] = val } both of > which I think we stay away from (and the former is from the

Re: Delta compression not so effective

2017-03-01 Thread Linus Torvalds
On Wed, Mar 1, 2017 at 5:51 AM, Marius Storm-Olsen wrote: > > When first importing, I disabled gc to avoid any repacking until completed. > When done importing, there was 209GB of all loose objects (~670k files). > With the hopes of quick consolidation, I did a > git -c

[PATCH] Put sha1dc on a diet

2017-02-28 Thread Linus Torvalds
From: Linus Torvalds <torva...@linux-foundation.org> Date: Tue, 28 Feb 2017 16:12:32 -0800 Subject: [PATCH] Put sha1dc on a diet This removes the unnecessary parts of the sha1dc code, shrinking things from [torvalds@i7 git]$ size sha1dc/*.o textdata bss

Re: Typesafer git hash patch

2017-02-28 Thread Linus Torvalds
On Tue, Feb 28, 2017 at 12:19 PM, brian m. carlson wrote: > > The bigger issue is the assumptions in the code base that assume a given > hash size. Absolutely. And I think those are going to be the "real" patches. I actually saw your status report about "After

Re: SHA1 collisions found

2017-02-28 Thread Linus Torvalds
On Tue, Feb 28, 2017 at 2:50 PM, Marc Stevens wrote: > > Because we only have 32 disturbance vectors to check, we have DVMASKSIZE > equal to 1 and maski always 0. > In the more general case when we add disturbance vectors this will not > remain the case. Ok, I didn't get why

  1   2   3   4   5   6   >