Bug#926181: want automatic forwarding of patches to the upstream branch
Hi,
On Mon, Apr 01, 2019 at 06:05:28PM +0100, Ian Jackson wrote:
> Osamu Aoki writes ("Bug#926181: want automatic forwarding of patches to the
> upstream branch"):
> > Package: dgit
> > Version: 8.4
> > Severity: wishlist
>
> Hi. Thanks for your message. I'm always interested in hearing what
> people want so please don't be discouraged by what I say next:
No problem ;-) This is more like a food-for-thought bug.
I am not particular about the use of dgit-maint-merge.
> > In case of the Debian maintainer is the same person as the upstream
> > maintainer, I would like to see automatic support to the operation
> > described in dgit-maint-merge(7). This is nice alternative to native
> > package workflow and nice upstream history.
>
> I am struggling to understand how you think this could work
> automatically. The biggest problem is that the data model in
> dgit-maint-merge(7) does not necessarily break the Debian delta into
> individual commits.
As a starter, I wrote a very broad vague wishlist. At the same time, I
understand this is a hard problem if this needs to be automatic for all
the use case.
> Also a problem is that I don't think there is an easy way of
> discovering what the upstream is like and how to submit patches there.
Let's make this simple by limiting the functionality of this wishlist
utility tool, say "dgit cherries", to function for a simple case. Let
me try with following ascii art.
human actions
X release
~ commit
+ merge
actions by "dgit cherries" on upstream
= commit (simple cherry picking)
: commit (dropping debian/* part with modified message)
. no-commit no change
First, You are upstream and released upstream package 1.0
1.0
upstream X
Then, packaged 1.0-1 *without* any change to upstream source.
1.0
upstream X
\
debian --+~~~X
1.0-1
Then, patched upstream part and released as 1.0-2, 1.0-3 as bug fix
version by changing upstream part on debian branch.
1.0
upstream X
\
debian --+~~~X~~X~~X
1.0-1 1.0-2 1.0-3
Then, found major bug and locally created 1.0-4 by committing to debian
branch. This is very convenient ;-)
1.0
upstream O
\
debian --+~~~X~~X~~X~~X
1.0-1 1.0-2 1.0-3 1.0-4
You can verify your idea for major change worked with 1.0-4.
You decide not to release it as Debian revision 1.0-4 but as a new
upstream source 1.1. Here, I think "dgit cherries" may be useful.
When HEAD is at 1.0-4 on debian branch, let's run "dgit cherries".
"dgit cherries" can find previous merge point "+".
Let "dgit cherries" cherry pick and apply cherry to the parent of merge
point on upstream branch. In this example, "O" is the point.
If change in debian branch commit doesn't include debian/* files, cherry
is applied with the original commit message and move on to next commit
on debian branch.
If change includes debian/* files, that part of cherry should be dropped
and remaining cherry should be applied with the commit message modified
to indicate change to debian/* files are not included. Then move on to
next commit on debian branch.
If change is only debian/* files, ignore that cherry and move on to next
commit on debian branch.
After all these are done, we should have
1.0
upstream O==.==:==...===.=.=.==...==.=.
\
debian --+~~~X~~X~~X~~X
1.0-1 1.0-2 1.0-3 1.0-4
This is the end point of "dgit cherries" in my POV.
Now, You being the upstream do whatever (rebase to merge and clean up
original change history or add more original changes.). Then release
upstream 1.1. (This usually involves changing some part of upstream
code manually to bump its version from 1.0 to 1.1. Also, you may wish
to update some documentation etc in upstream source now.)
1.01.1
upstream O==.=~:==-~~-~==.=.=.=~-.=.O
\
debian --+~~~X~~X~~X~~X
1.0-1 1.0-2 1.0-3 1.0-4
Then you package 1.1-1 by merging upstream to debian as follows.
1.01.1
upstream O==.==:==...===.=.=.==...==.=.O
\ \
debian --+~~~X~~X~~X~~X+~~~X
1.0-1 1.0-2 1.0-3 1.0-41.1-1
> (As an aside: I don't think any of this depends on whether the Debian
> maintainer is the same person as the upstream with a different
Bug#926181: want automatic forwarding of patches to the upstream branch
Osamu Aoki writes ("Bug#926181: want automatic forwarding of patches to the
upstream branch"):
> Package: dgit
> Version: 8.4
> Severity: wishlist
Hi. Thanks for your message. I'm always interested in hearing what
people want so please don't be discouraged by what I say next:
> In case of the Debian maintainer is the same person as the upstream
> maintainer, I would like to see automatic support to the operation
> described in dgit-maint-merge(7). This is nice alternative to native
> package workflow and nice upstream history.
I am struggling to understand how you think this could work
automatically. The biggest problem is that the data model in
dgit-maint-merge(7) does not necessarily break the Debian delta into
individual commits.
Also a problem is that I don't think there is an easy way of
discovering what the upstream is like and how to submit patches there.
(As an aside: I don't think any of this depends on whether the Debian
maintainer is the same person as the upstream with a different hat
on.)
> |FORWARDING PATCHES UPSTREAM
> | The basic steps are:
> |
> | 1. Create a new branch based off upstream's master branch.
> | 3. Push the branch somewhere and ask upstream to merge it, or use
> git-format-patch(1) or git-request-pull(1).
So (taking things out of order) these require knowing what the
upstream is and how to find its master branch and submit a merge
request.
It's true that we do have some metadata conventions for this kind of
thing in Debian. But maybe the task of automating steps 1 and 3
should be a separate program, in devscripts maybe ? It doesn't seem
dgit specific.
> | 2. git-cherry-pick(1) commits from your master branch onto your new branch.
This is the hardest part. I think in general only the maintainer will
know what commit(s) to cherry-pick.
> I want automated operation which goes like:
You seem to think that with dgit-maint-merge(7) there is a nice linear
list of the commits which are in Debian but not upstream. But there
isn't: in the general case, there is a lot of merging and the only
reasonably condensed representation of the Debian delta is the output
of `git diff' between the upstream and Debian branches.
I think what you really want to make this easy is a different data
model. Have you looked at git-debrebase or gbp pq ? Both of those
maintain the Debian delta as a linear series of commits.
git-debrebase already knows how to do many of the manipulations you
describe. For example:
> * for commit changing within debian/* only
>--> ignore
> * for commit changing outside of debian/* only
>--> apply as is with the same commit messages
> * for commit changing everything
>--> apply changes outside of debian/*
>--> use the same commit message
git-debrebase already knows how to separate commits out into "Debian"
parts and "upstream files" parts, and can split a "mixed" commit into
its two halves. So it can leave you with a tidy delta queue.
Perhaps there should be a
git-debrebase list
which shows you the delta queue and maybe some of your other
suggestions might make good git-debrebase features.
I hope my explanations make some kind of sense. If not maybe we need
a less formal but more interactive channel such as irc; feel free to
look for me as Diziet on oftc and freenode.
Ian.
Bug#926181: want automatic forwarding of patches to the upstream branch
Package: dgit Version: 8.4 Severity: wishlist In case of the Debian maintainer is the same person as the upstream maintainer, I would like to see automatic support to the operation described in dgit-maint-merge(7). This is nice alternative to native package workflow and nice upstream history. |FORWARDING PATCHES UPSTREAM | The basic steps are: | | 1. Create a new branch based off upstream's master branch. | | 2. git-cherry-pick(1) commits from your master branch onto your new branch. | | 3. Push the branch somewhere and ask upstream to merge it, or use git-format-patch(1) or git-request-pull(1). | | For example (and it is only an example): | | % # fork foo.git on GitHub | % git remote add -f fork g...@github.com:spwhitton/foo.git | % git checkout -b fix-error upstream/master | % git config branch.fix-error.pushRemote fork | % git cherry-pick master^2 | % git push | % # submit pull request on GitHub | | Note that when you merge an upstream release containing your forwarded patches, git and dgit will transparently handle "dropping" the patches that have been forwarded, "retaining" the ones that | haven't. This is quite a chore which may be helped by automation. I want automated operation which goes like: 1) Full automatic $ dgit update-upstream * for commit changing within debian/* only --> ignore * for commit changing outside of debian/* only --> apply as is with the same commit messages * for commit changing everything --> apply changes outside of debian/* --> use the same commit message * merge commit from the last upstream commit to master branch Please note changes are taken from the last commit point used for merge 2) List commit candidates $ dgit update-upstream -l drop 5c8c1e update packaging policy pick 192837 Switch from GNU Makefiles to CMake trim 5a6c7e Document the switch to CMake pick 918273 Fix detection of OpenSSL in CMake pick afbecd http: add support for TLS v1.3 pick fdbaec Fix detection of cURL in CMake on Windows 3) Exclude specific commits with commandline $ dgit update-upstream -x 12345678 -x 1a2b3c4d * Do the same as "Full automatic" but exclude commit hash 12345678 and 1a2b3c4d 4) Exclude specific commits with editor selection: $ dgit update-upstream -i * make git-rebase like editor dialog marking each commit with list sent to editor as: drop 5c8c1e update packaging policy pick 192837 Switch from GNU Makefiles to CMake trim 5a6c7e Document the switch to CMake pick 918273 Fix detection of OpenSSL in CMake pick afbecd http: add support for TLS v1.3 pick fdbaec Fix detection of cURL in CMake on Windows Here; - drop drop debian/* changes - pick pick upstream only change - trim trim debian/* changes but pick upstream only change User can specify: - drop (for any) - pick (for pick) - trim (for trim) - reword (for pick or trim) --> allow to change commit message If the last 4) is too much, maybe we can skip since this is easily done with "git rebase -i ?" after "dgit update-upstream" -- System Information: Debian Release: buster/sid APT prefers testing APT policy: (500, 'testing'), (10, 'unstable') Architecture: amd64 (x86_64) Kernel: Linux 4.19.0-4-amd64 (SMP w/4 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages dgit depends on: ii apt 1.8.0 ii ca-certificates 20190110 ii coreutils 8.30-3 ii curl7.64.0-1 ii devscripts 2.19.4 ii dpkg-dev1.19.5 ii dput1.0.3 ii git [git-core] 1:2.20.1-2 ii git-buildpackage0.9.13 pn libdigest-sha-perl ii libdpkg-perl1.19.5 ii libjson-perl4.02000-1 ii liblist-moreutils-perl 0.416-1+b4 ii liblocale-gettext-perl 1.07-3+b4 ii libtext-glob-perl 0.10-1 ii libtext-iconv-perl 1.7-5+b7 ii libwww-perl 6.36-1 ii perl5.28.1-5 Versions of packages dgit recommends: ii openssh-client [ssh-client] 1:7.9p1-9 Versions of packages dgit suggests: ii cowbuilder 0.88 ii pbuilder0.230.2 -- no debconf information
