Re: Optimizing writes to unchanged files during merges?

On Tue, Apr 17, 2018 at 10:27 AM, Lars Schneider wrote: > >> On 16 Apr 2018, at 19:45, Jacob Keller wrote: >> >> On Mon, Apr 16, 2018 at 10:43 AM, Jacob Keller >> wrote: >>> On Mon, Apr 16, 2018 at 9:07 AM, Lars

Re: Optimizing writes to unchanged files during merges?

> On 16 Apr 2018, at 19:45, Jacob Keller wrote: > > On Mon, Apr 16, 2018 at 10:43 AM, Jacob Keller wrote: >> On Mon, Apr 16, 2018 at 9:07 AM, Lars Schneider >> wrote: >>> What if Git kept a LRU list that contains file

Re: Optimizing writes to unchanged files during merges?

> On 16 Apr 2018, at 19:04, Ævar Arnfjörð Bjarmason wrote: > > > On Mon, Apr 16 2018, Lars Schneider wrote: > >>> On 16 Apr 2018, at 04:03, Linus Torvalds >>> wrote: >>> >>> On Sun, Apr 15, 2018 at 6:44 PM, Junio C Hamano

Re: Optimizing writes to unchanged files during merges?

On Sun, Apr 15, 2018 at 6:44 PM, Junio C Hamano wrote: > One thing that makes me curious is what happens (and what we want to > happen) when such a "we already have the changes the side branch > tries to bring in" path has local (i.e. not yet in the index) > changes. For a

Re: Optimizing writes to unchanged files during merges?

On Sun, Apr 15, 2018 at 7:03 PM, Linus Torvalds wrote: > On Sun, Apr 15, 2018 at 6:44 PM, Junio C Hamano wrote: >> One thing that makes me curious is what happens (and what we want to >> happen) when such a "we already have the changes the side

Re: Optimizing writes to unchanged files during merges?

Jacob Keller writes: > On Mon, Apr 16, 2018 at 10:43 AM, Jacob Keller wrote: > ... > I think a better solution for your problem would be to extend the > build system you're using to avoid rebuilding when the contents > haven't changed since last

Re: Optimizing writes to unchanged files during merges?

Lars Schneider wrote: > An engineer works on a task branch and runs incremental builds — all > is good. The engineer switches to another branch to review another > engineer's work. This other branch changes a low-level header file, > but no rebuild is triggered. The

Re: Optimizing writes to unchanged files during merges?

On 16/04/18 17:07, Lars Schneider wrote: > > I am happy to see this discussion and the patches, because long rebuilds > are a constant annoyance for us. We might have been bitten by the exact > case discussed here, but more often, we have a slightly different > situation: > > An engineer

Re: Optimizing writes to unchanged files during merges?

On Mon, Apr 16, 2018 at 10:43 AM, Jacob Keller wrote: > On Mon, Apr 16, 2018 at 9:07 AM, Lars Schneider > wrote: >> What if Git kept a LRU list that contains file path, content hash, and >> mtime of any file that is removed or modified during a

Re: Optimizing writes to unchanged files during merges?

On Mon, Apr 16, 2018 at 9:07 AM, Lars Schneider wrote: > >> On 16 Apr 2018, at 04:03, Linus Torvalds >> wrote: >> >> On Sun, Apr 15, 2018 at 6:44 PM, Junio C Hamano wrote: >>> >>> I think Elijah's corrected

Re: Optimizing writes to unchanged files during merges?

On Mon, Apr 16 2018, Lars Schneider wrote: >> On 16 Apr 2018, at 04:03, Linus Torvalds >> wrote: >> >> On Sun, Apr 15, 2018 at 6:44 PM, Junio C Hamano wrote: >>> >>> I think Elijah's corrected was_tracked() also does not care "has >>> this

Re: Optimizing writes to unchanged files during merges?

> On 16 Apr 2018, at 04:03, Linus Torvalds > wrote: > > On Sun, Apr 15, 2018 at 6:44 PM, Junio C Hamano wrote: >> >> I think Elijah's corrected was_tracked() also does not care "has >> this been renamed". > > I'm perfectly happy with the

Re: Optimizing writes to unchanged files during merges?

On Sun, Apr 15, 2018 at 6:44 PM, Junio C Hamano wrote: > > I think Elijah's corrected was_tracked() also does not care "has > this been renamed". I'm perfectly happy with the slightly smarter patches. My patch was really just an RFC and because I had tried it out. > One thing

Re: Optimizing writes to unchanged files during merges?

Linus Torvalds writes: > How about we take a completely different approach? Instead of relying > on fragile (but clever) tests, why not rely on stupid brute force? > > Yeah, yeah, it's bad to be stupid, but sometimes simple and stupid > really does work. > > See

Re: Optimizing writes to unchanged files during merges?

Elijah Newren writes: > Yes, precisely. Checking the *current* index is not reliable in the > presence of renames. > > Trying to use the current index as a proxy for what was in the index > before the merge started is a problem. But we had a copy of the index > before the

Re: Optimizing writes to unchanged files during merges?

On Fri, Apr 13, 2018 at 10:14 AM, Linus Torvalds wrote: > On Fri, Apr 13, 2018 at 12:02 AM, Elijah Newren wrote: >> However, it turns out we have this awesome function called >> "was_tracked(const char *path)" that was intended for answering this

Re: Optimizing writes to unchanged files during merges?

On Fri, Apr 13, 2018 at 10:39 AM, Stefan Beller wrote: > > Would s/read/xread/ make sense in working_tree_matches ? Makes sense, yes. That patch was really more of a RFD than anything that should be applied. I would like to see the "might be same" flag pushed down so that

Re: Optimizing writes to unchanged files during merges?

Hi Linus, On Fri, Apr 13, 2018 at 10:14 AM, Linus Torvalds wrote: > > Comments? Because considering the problems this code has had, maybe > "stupid" really is the right approach... Would s/read/xread/ make sense in working_tree_matches ? If read returns

Re: Optimizing writes to unchanged files during merges?

On Fri, Apr 13, 2018 at 12:02 AM, Elijah Newren wrote: > > I hope you don't mind me barging into your conversation I was getting tired of my own rambling anyway.. > However, it turns out we have this awesome function called > "was_tracked(const char *path)" that was intended

Re: Optimizing writes to unchanged files during merges?

On Thu, Apr 12, 2018 at 5:01 PM, Linus Torvalds wrote: > [ Still talking to myself. Very soothing. ] > > On Thu, Apr 12, 2018 at 4:55 PM, Linus Torvalds > wrote: >> [ Talking to myself ] >> >> Did it perhaps mean to say >> >>

Re: Optimizing writes to unchanged files during merges?

On Thu, Apr 12, 2018 at 2:14 PM, Linus Torvalds wrote: > So I just had an interesting experience that has happened before too, > but this time I decided to try to figure out *why* it happened. > > include/linux/mm.h > > and yeah, that rather core header file

Re: Optimizing writes to unchanged files during merges?

[ Still talking to myself. Very soothing. ] On Thu, Apr 12, 2018 at 4:55 PM, Linus Torvalds wrote: > [ Talking to myself ] > > Did it perhaps mean to say > > path_renamed_outside_HEAD = path2 && !strcmp(path, path2); > > instead? Probably not

Re: Optimizing writes to unchanged files during merges?

[ Talking to myself ] On Thu, Apr 12, 2018 at 4:41 PM, Linus Torvalds wrote: > > Oddly, that *already* has the check: > > if (mfi.clean && !df_conflict_remains && > oid_eq(, a_oid) && mfi.mode == a_mode) { > int

Re: Optimizing writes to unchanged files during merges?

On Thu, Apr 12, 2018 at 4:35 PM, Linus Torvalds wrote: > > in process_entry(), and I think we could just there add a test for if > o_old,o_mod == a_oid,a_mode or something? Actually, not process_entry, but merge_content(). Oddly, that *already* has the check:

Re: Optimizing writes to unchanged files during merges?

On Thu, Apr 12, 2018 at 4:17 PM, Junio C Hamano wrote: > > A bit of detour. "Change in side branch happened to be a subset of > the change in trunk and gets subsumed, but we end up writing the > same result" happens also with the simpler resolve strategy. > > Here is a fix.

Re: Optimizing writes to unchanged files during merges?

On Thu, Apr 12, 2018 at 2:46 PM, Junio C Hamano wrote: > > Thanks for a clear description of the issue. It does sound > interesting. I decided to show it with a simpler case that could be scripted and doesn't need the kernel. NOTE! This obviously doesn't happen for files

Re: Optimizing writes to unchanged files during merges?

Junio C Hamano writes: > Linus Torvalds writes: > >> Now, the reason it was marked as changed is that the xfs branch _had_ >> in fact changed it, but the changes were already upstream and got >> merged away. But the file still got written out

Re: Optimizing writes to unchanged files during merges?

Linus Torvalds writes: > Now, the reason it was marked as changed is that the xfs branch _had_ > in fact changed it, but the changes were already upstream and got > merged away. But the file still got written out (with the same > contents it had before the merge),

Optimizing writes to unchanged files during merges?

So I just had an interesting experience that has happened before too, but this time I decided to try to figure out *why* it happened. I'm obviously in the latter part of the kernel merge window, and things are slowly calming down. I do the second XFS merge during this window, and it brings in