Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option

[Johannes Schindelin]

Hi Philip,

On Sun, 22 Apr 2018, Philip Oakley wrote:

> From: "Johannes Schindelin" 
> > Once upon a time, this here developer thought: wouldn't it be nice if,
> > say, Git for Windows' patches on top of core Git could be represented as
> > a thicket of branches, and be rebased on top of core Git in order to
> > maintain a cherry-pick'able set of patch series?
> >
> > The original attempt to answer this was: git rebase --preserve-merges.
> >
> > However, that experiment was never intended as an interactive option,
> > and it only piggy-backed on git rebase --interactive because that
> > command's implementation looked already very, very familiar: it was
> > designed by the same person who designed --preserve-merges: yours truly.
> >
> > Some time later, some other developer (I am looking at you, Andreas!
> > ;-)) decided that it would be a good idea to allow --preserve-merges to
> > be combined with --interactive (with caveats!) and the Git maintainer
> > (well, the interim Git maintainer during Junio's absence, that is)
> > agreed, and that is when the glamor of the --preserve-merges design
> > started to fall apart rather quickly and unglamorously.
> >
> > The reason? In --preserve-merges mode, the parents of a merge commit (or
> > for that matter, of *any* commit) were not stated explicitly, but were
> > *implied* by the commit name passed to the `pick` command.
> >
> Aside: I think this para should be extracted to the --preserve-merges
> documentation to highlight what it does / why it is 'wrong' (not what would be
> expected in some case). It may also need to discuss the (figurative) Cousins
> vs. Siblings distinction [merge of branches external, or internal, to the
> rebase.

Quite honestly, I'd much rather spend time improving --rebase-merges than
improving --preserve-merges documentation. In my mind, the latter is
pretty useless, especially once the former lands in an official Git
version.

Of course, feel free to disagree with me by sending a patch to improve the
documentation of --preserve-merges ;-)

> "In --preserve-merges, the commit being selected for merging is implied by the
> commit name  passed to the `pick` command (i.e. of the original merge commit),
> not that of the rebased version of that parent."

It is much, much worse:

In --preserve-merges, no commit can change its ancestry. Every
rebased commit's parents will be the rebased original parents.

Or some such. But really, why bother describing something *that* broken?
Why not work toward a solution that makes that broken option obsolete?
Like, say, --rebase-merges? ;-)

> A similar issue occurs with (figuratively) '--ancestry-path --first parent'
> searches which lacks the alternate '--lead parent' post-walk selection. [1]. I
> don't think there is a dot notation to select the merge cousins, nor merge
> siblings either A.,B ? (that's dot-comma ;-)

I actually had missed `--ancestry-path`... I should probably use it in the
description of the "cousins".

> [... lots of quoted text...]

Could I ask you to make it easier for me by cutting quoted text that is
irrelevant to your reply? The way I read mails forces me to scroll down
(sometimes on a phone) all the way to the end, just to find that that time
was spent in vain.

Thanks,
Dscho

Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option

[Johannes Schindelin]

Hi Junio,

On Tue, 24 Apr 2018, Junio C Hamano wrote:

> "Philip Oakley"  writes:
> 
> >> +-r::
> >> +--rebase-merges::
> >> + By default, a rebase will simply drop merge commits and only rebase
> >> + the non-merge commits. With this option, it will try to preserve
> >> + the branching structure within the commits that are to be rebased,
> >> + by recreating the merge commits. If a merge commit resolved any merge

It is funny how nobody caught the missing "conflicts" in "If a merge
commit resolved any merge [conflicts]"...

> >> + or contained manual amendments, then they will have to be re-applied
> >> + manually.
> >> ++
> >> +This mode is similar in spirit to `--preserve-merges`, but in contrast to
> >> +that option works well in interactive rebases: commits can be reordered,
> >> +inserted and dropped at will.
> >> ++
> >> +It is currently only possible to recreate the merge commits using the
> >> +`recursive` merge strategy; Different merge strategies can be used only
> >> via
> >> +explicit `exec git merge -s  [...]` commands.
> >> +
> >> -p::
> >> --preserve-merges::
> >>  Recreate merge commits instead of flattening the history by replaying
> >
> > Flatten is here in the context lines but its just a blunt statement that 'it
> > is what it is'...
> 
> The first paragraph that explains --rebase-merges talks about what
> happens when the option is not given, and says "drop merge commits
> and only rebase the non-merge commits", which is not incorrect
> per-se but does not make it explicit how the resulting topology
> looks like.

Correct. And it would be the wrong place to describe in detail what a
rebase *without --rebase-merges* does, right?

> I think it is easier to understand if it mentioned "flattening" as well.
> If flatten is not the word you want, perhaps "make it linear" or
> something like that?

I fear that we all here are way too deeply in "Git think". If I ask a
random Git user what it means to "make commits linear", I am sure I would
get only puzzled, nervous looks as a response.

So I am rather certain that the suggested wording is something I want to
avoid.

Besides, we really should expect the reader to know a little about the way
the interactive rebase works by the time they read the explanation of
`--rebase-merges`.

Therefore, I think I can sidestep the entire thing by saying this instead:

-r::
--rebase-merges::
By default, a rebase will simply drop merge commits from the todo
list, and put the rebased commits into a single, linear branch.
With `--rebase-merges`, the rebase will instead try to preserve
the branching structure within the commits that are to be rebased,
by recreating the merge commits. Any resolved merge conflicts or
manual amendments in these merge commits will have to be
resolved/re-applied manually.

Thank you for helping me improve the documentation part of this patch,
which I think is really, really important,
Dscho

Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option

[Johannes Schindelin]

Hi Philip,

On Sun, 22 Apr 2018, Philip Oakley wrote:

> From: "Johannes Schindelin" 
> > Once upon a time, this here developer thought: wouldn't it be nice if,
> > say, Git for Windows' patches on top of core Git could be represented as
> > a thicket of branches, and be rebased on top of core Git in order to
> > maintain a cherry-pick'able set of patch series?
> >
> > The original attempt to answer this was: git rebase --preserve-merges.
> >
> > However, that experiment was never intended as an interactive option,
> > and it only piggy-backed on git rebase --interactive because that
> > command's implementation looked already very, very familiar: it was
> > designed by the same person who designed --preserve-merges: yours truly.
> >
> > Some time later, some other developer (I am looking at you, Andreas!
> > ;-)) decided that it would be a good idea to allow --preserve-merges to
> > be combined with --interactive (with caveats!) and the Git maintainer
> > (well, the interim Git maintainer during Junio's absence, that is)
> > agreed, and that is when the glamor of the --preserve-merges design
> > started to fall apart rather quickly and unglamorously.
> >
> > The reason? In --preserve-merges mode, the parents of a merge commit (or
> > for that matter, of *any* commit) were not stated explicitly, but were
> > *implied* by the commit name passed to the `pick` command.
> >
> > This made it impossible, for example, to reorder commits. Not to mention
> > to flatten the branch topology or, deity forbid, to split topic branches
> 
> Aside: The idea of a "flattened" topology is, to my mind, not actually
> defined though may be understood by devs working in the area. Hopefully it's
> going away as a term, though the new 'cousins' will need clarification
> (there's no dot notation for that area of topology).

Right. The point is not actually to talk about "flattening" branches. The
point is to talk about the flexibility one might expect in an
*interactive* rebase, a flexibility notably lacking from the
--preserve-merges mode.

So I changed it to

This made it impossible, for example, to reorder commits. Not to mention
to move commits between branches or, deity forbid, to split topic branches
into two.

> > into two.
> >
> > Alas, these shortcomings also prevented that mode (whose original
> > purpose was to serve Git for Windows' needs, with the additional hope
> > that it may be useful to others, too) from serving Git for Windows'
> > needs.
>
> [... please feel free to save readers time by culling quoted text that
> is irrelevant to your reply...]
>
> > diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
> > index 3277ca14327..34e0f6a69c1 100644
> > --- a/Documentation/git-rebase.txt
> > +++ b/Documentation/git-rebase.txt
> > @@ -378,6 +378,23 @@ The commit list format can be changed by setting the
> > configuration option
> > rebase.instructionFormat.  A customized instruction format will
> > automatically
> > have the long commit hash prepended to the format.
> >
> > +-r::
> > +--rebase-merges::
> > + By default, a rebase will simply drop merge commits and only rebase
> > + the non-merge commits. With this option, it will try to preserve
> > + the branching structure within the commits that are to be rebased,
> > + by recreating the merge commits. If a merge commit resolved any merge
> > + or contained manual amendments, then they will have to be re-applied
> > + manually.
> > ++
> > +This mode is similar in spirit to `--preserve-merges`, but in contrast to
> > +that option works well in interactive rebases: commits can be reordered,
> > +inserted and dropped at will.
> > ++
> > +It is currently only possible to recreate the merge commits using the
> > +`recursive` merge strategy; Different merge strategies can be used only
> > via
> > +explicit `exec git merge -s  [...]` commands.
> > +
> > -p::
> > --preserve-merges::
> >  Recreate merge commits instead of flattening the history by replaying
> 
> Flatten is here in the context lines but its just a blunt statement that 'it
> is what it is'...

Correct. This is where that way of expressing things came from.

I will *not* fix the documentation of `--preserve-merges`, though, as I
hope it can be instead retired soon enough.

Ciao,
Dscho

Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option

[Junio C Hamano]

"Philip Oakley"  writes:

>> +-r::
>> +--rebase-merges::
>> + By default, a rebase will simply drop merge commits and only rebase
>> + the non-merge commits. With this option, it will try to preserve
>> + the branching structure within the commits that are to be rebased,
>> + by recreating the merge commits. If a merge commit resolved any merge
>> + or contained manual amendments, then they will have to be re-applied
>> + manually.
>> ++
>> +This mode is similar in spirit to `--preserve-merges`, but in contrast to
>> +that option works well in interactive rebases: commits can be reordered,
>> +inserted and dropped at will.
>> ++
>> +It is currently only possible to recreate the merge commits using the
>> +`recursive` merge strategy; Different merge strategies can be used only
>> via
>> +explicit `exec git merge -s  [...]` commands.
>> +
>> -p::
>> --preserve-merges::
>>  Recreate merge commits instead of flattening the history by replaying
>
> Flatten is here in the context lines but its just a blunt statement that 'it
> is what it is'...

The first paragraph that explains --rebase-merges talks about what
happens when the option is not given, and says "drop merge commits
and only rebase the non-merge commits", which is not incorrect
per-se but does not make it explicit how the resulting topology
looks like.  I think it is easier to understand if it mentioned
"flattening" as well.  If flatten is not the word you want, perhaps
"make it linear" or something like that?

Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option

[Philip Oakley]

From: "Johannes Schindelin" 

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches


Aside: The idea of a "flattened" topology is, to my mind, not actually
defined though may be understood by devs working in the area. Hopefully it's
going away as a term, though the new 'cousins' will need clarification
(there's no dot notation for that area of topology).


into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--rebase-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge `. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin 
---
Documentation/git-rebase.txt   |  20 ++-
contrib/completion/git-completion.bash |   2 +-
git-rebase--interactive.sh |   1 +
git-rebase.sh  |   6 +
t/t3430-rebase-merges.sh   | 179 +
5 files changed, 206 insertions(+), 2 deletions(-)
create mode 100755 t/t3430-rebase-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 3277ca14327..34e0f6a69c1 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -378,6 +378,23 @@ The commit list format can be changed by setting the
configuration option
rebase.instructionFormat.  A customized instruction format will
automatically
have the long commit hash prepended to the format.

+-r::
+--rebase-merges::
+ By default, a rebase will simply drop merge commits and only rebase
+ the non-merge commits. With this option, it will try to preserve
+ the branching structure within the commits that are to be rebased,
+ by recreating the merge commits. If a merge commit resolved any merge
+ or contained manual amendments, then they will have to be re-applied
+ manually.
++
+This mode is similar in spirit to `--preserve-merges`, but in contrast to
+that option works well in interactive rebases: commits can 

Re: [PATCH v8 09/16] rebase: introduce the --rebase-merges option

[Philip Oakley]

From: "Johannes Schindelin" 

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

Aside: I think this para should be extracted to the --preserve-merges 
documentation to highlight what it does / why it is 'wrong' (not what would 
be expected in some case). It may also need to discuss the (figurative) 
Cousins vs. Siblings distinction [merge of branches external, or internal, 
to the rebase.


"In --preserve-merges, the commit being selected for merging is implied by 
the commit name  passed to the `pick` command (i.e. of the original merge 
commit), not that of the rebased version of that parent."


A similar issue occurs with (figuratively) '--ancestry-path --first parent' 
searches which lacks the alternate '--lead parent' post-walk selection. [1]. 
I don't think there is a dot notation to select the merge cousins, nor merge 
siblings either A.,B ? (that's dot-comma ;-)



This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--rebase-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge `. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin 
---
Documentation/git-rebase.txt   |  20 ++-
contrib/completion/git-completion.bash |   2 +-
git-rebase--interactive.sh |   1 +
git-rebase.sh  |   6 +
t/t3430-rebase-merges.sh   | 179 +
5 files changed, 206 insertions(+), 2 deletions(-)
create mode 100755 t/t3430-rebase-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 3277ca14327..34e0f6a69c1 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -378,6 +378,23 @@ The commit list format can be changed by setting the 
configuration option
rebase.instructionFormat.  A customized instruction format will 
automatically

have the long commit hash prepended to the 

[PATCH v8 09/16] rebase: introduce the --rebase-merges option

[Johannes Schindelin]

Once upon a time, this here developer thought: wouldn't it be nice if,
say, Git for Windows' patches on top of core Git could be represented as
a thicket of branches, and be rebased on top of core Git in order to
maintain a cherry-pick'able set of patch series?

The original attempt to answer this was: git rebase --preserve-merges.

However, that experiment was never intended as an interactive option,
and it only piggy-backed on git rebase --interactive because that
command's implementation looked already very, very familiar: it was
designed by the same person who designed --preserve-merges: yours truly.

Some time later, some other developer (I am looking at you, Andreas!
;-)) decided that it would be a good idea to allow --preserve-merges to
be combined with --interactive (with caveats!) and the Git maintainer
(well, the interim Git maintainer during Junio's absence, that is)
agreed, and that is when the glamor of the --preserve-merges design
started to fall apart rather quickly and unglamorously.

The reason? In --preserve-merges mode, the parents of a merge commit (or
for that matter, of *any* commit) were not stated explicitly, but were
*implied* by the commit name passed to the `pick` command.

This made it impossible, for example, to reorder commits. Not to mention
to flatten the branch topology or, deity forbid, to split topic branches
into two.

Alas, these shortcomings also prevented that mode (whose original
purpose was to serve Git for Windows' needs, with the additional hope
that it may be useful to others, too) from serving Git for Windows'
needs.

Five years later, when it became really untenable to have one unwieldy,
big hodge-podge patch series of partly related, partly unrelated patches
in Git for Windows that was rebased onto core Git's tags from time to
time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hg series that first obsoleted Git for Windows' competing
approach, only to be abandoned without maintainer later) was really
untenable, the "Git garden shears" were born [*1*/*2*]: a script,
piggy-backing on top of the interactive rebase, that would first
determine the branch topology of the patches to be rebased, create a
pseudo todo list for further editing, transform the result into a real
todo list (making heavy use of the `exec` command to "implement" the
missing todo list commands) and finally recreate the patch series on
top of the new base commit.

That was in 2013. And it took about three weeks to come up with the
design and implement it as an out-of-tree script. Needless to say, the
implementation needed quite a few years to stabilize, all the while the
design itself proved itself sound.

With this patch, the goodness of the Git garden shears comes to `git
rebase -i` itself. Passing the `--rebase-merges` option will generate
a todo list that can be understood readily, and where it is obvious
how to reorder commits. New branches can be introduced by inserting
`label` commands and calling `merge `. And once this mode will
have become stable and universally accepted, we can deprecate the design
mistake that was `--preserve-merges`.

Link *1*:
https://github.com/msysgit/msysgit/blob/master/share/msysGit/shears.sh
Link *2*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

Signed-off-by: Johannes Schindelin 
---
 Documentation/git-rebase.txt   |  20 ++-
 contrib/completion/git-completion.bash |   2 +-
 git-rebase--interactive.sh |   1 +
 git-rebase.sh  |   6 +
 t/t3430-rebase-merges.sh   | 179 +
 5 files changed, 206 insertions(+), 2 deletions(-)
 create mode 100755 t/t3430-rebase-merges.sh

diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt
index 3277ca14327..34e0f6a69c1 100644
--- a/Documentation/git-rebase.txt
+++ b/Documentation/git-rebase.txt
@@ -378,6 +378,23 @@ The commit list format can be changed by setting the 
configuration option
 rebase.instructionFormat.  A customized instruction format will automatically
 have the long commit hash prepended to the format.
 
+-r::
+--rebase-merges::
+   By default, a rebase will simply drop merge commits and only rebase
+   the non-merge commits. With this option, it will try to preserve
+   the branching structure within the commits that are to be rebased,
+   by recreating the merge commits. If a merge commit resolved any merge
+   or contained manual amendments, then they will have to be re-applied
+   manually.
++
+This mode is similar in spirit to `--preserve-merges`, but in contrast to
+that option works well in interactive rebases: commits can be reordered,
+inserted and dropped at will.
++
+It is currently only possible to recreate the merge commits using the
+`recursive` merge strategy; Different merge strategies can be used only via
+explicit `exec git merge -s  [...]` commands.
+
 -p::
 --preserve-merges::
Recreate