Hi, Thanks a lot for the patches - this makes sens (and I think we discussed it at debconf)!
On Thu, Aug 18, 2016 at 01:40:35PM +0200, Maximiliano Curia wrote: > Package: git-buildpackage > Version: 0.7.5 > Severity: normal > Tags: patch > > Hi, > > Currently patch queue assumes that the upstream branch is merged in the > Debian > branch. But, at the same time having them unmerged is a valid setup for > buildpackage, even more, the pq support works assuming that the upstream You mean the rpm-pq support? > branch is not merged with the spec branch. > > The attached patches add the option to use the corresponding upstream-tag as > the base for the patch queue branch, this allows the upstream not merged in > the Debian branch workflow. > > The additional parameter for pq --pq-from is used to select either DEBIAN > (default) to use the Debian branch as the base for pq, or TAG to use the > upstream tag as the base. I'm not very good at giving names to things, so > maybe the pq-from parameter could do with a different name. > > The logic is very similar to what's currently in use by the rpm version. And > it would make sense to refactor pq and pq_rpm to share most of it, if this > get's applied. I'm glad to apply this. Two things: could you rebase against current master which has had some changes (but not many in pq). And is there a chance you add a testcase in tests/component/deb/test_pq.py ? This would be a new component test but it would be perfectly fine to only test the new functionality. In case you have a repo I can pull from therei is no need to attach all the patches to the bts. Cheers, -- Guido > > Happy hacking, > -- System Information: > Debian Release: stretch/sid > APT prefers unstable-debug > APT policy: (500, 'unstable-debug'), (500, 'buildd-unstable'), (500, > 'testing'), (500, 'stable'), (50, 'unstable'), (1, 'experimental') > Architecture: amd64 (x86_64) > Foreign Architectures: i386, armhf > > Kernel: Linux 4.6.0-1-amd64 (SMP w/4 CPU cores) > Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) > Shell: /bin/sh linked to /bin/bash > Init: systemd (via /run/systemd/system) > > Versions of packages git-buildpackage depends on: > ii devscripts 2.16.6 > ii git 1:2.8.1-1 > ii man-db 2.7.5-1 > ii python-dateutil 2.4.2-1 > ii python-pkg-resources 20.10.1-1.1 > ii python-six 1.10.0-3 > pn python:any <none> > > Versions of packages git-buildpackage recommends: > ii cowbuilder 0.80 > ii pbuilder 0.225.1 > ii pristine-tar 1.34 > ii python-requests 2.10.0-2 > ii sbuild 0.70.0-1 > > Versions of packages git-buildpackage suggests: > ii python-notify 0.1.1-4 > ii sudo 1.8.17p1-2 > ii unzip 6.0-20 > > -- no debconf information > >From 29233801826dbba2018d43c9913290f0e3cd0bb3 Mon Sep 17 00:00:00 2001 > From: Maximiliano Curia <[email protected]> > Date: Sat, 13 Aug 2016 15:05:29 +0200 > Subject: [PATCH 1/4] pq: Handle unmerged debian branches (import) > > This add two new parameters to pq, upstream_tag, and pq_from, the last one is > used to use > the canonical merged branches logic and the use upstream_tag as the base for > the pq branch. > This is similar to what's currently used in the rpm code, and could be merged > in future > updates. > --- > gbp/config.py | 4 ++++ > gbp/scripts/pq.py | 41 +++++++++++++++++++++++++++++++++++------ > 2 files changed, 39 insertions(+), 6 deletions(-) > > diff --git a/gbp/config.py b/gbp/config.py > index 575a6b1..ddbf660 100644 > --- a/gbp/config.py > +++ b/gbp/config.py > @@ -99,6 +99,7 @@ class GbpOptionParser(OptionParser): > defaults = { 'debian-branch' : 'master', > 'upstream-branch' : 'upstream', > 'upstream-tree' : 'TAG', > + 'pq-from' : 'DEBIAN', > 'pristine-tar' : 'False', > 'pristine-tar-commit': 'False', > 'filter-pristine-tar' : 'False', > @@ -183,6 +184,9 @@ class GbpOptionParser(OptionParser): > 'upstream-tree': > ("Where to generate the upstream tarball from " > "(tag or branch), default is '%(upstream-tree)s'"), > + 'pq-from': > + ("How to find the patch queue base. DEBIAN or TAG, " > + "the default is '%(pq-from)s'"), > 'debian-tag': > ("Format string for debian tags, " > "default is '%(debian-tag)s'"), > diff --git a/gbp/scripts/pq.py b/gbp/scripts/pq.py > index 9d93341..84da6a0 100755 > --- a/gbp/scripts/pq.py > +++ b/gbp/scripts/pq.py > @@ -24,7 +24,9 @@ import sys > import tempfile > import re > from gbp.config import GbpOptionParserDebian > -from gbp.git import (GitRepositoryError, GitRepository) > +from gbp.deb.source import DebianSource > +from gbp.deb.git import DebianGitRepository > +from gbp.git import GitRepositoryError > from gbp.command_wrappers import (GitCommand, CommandExecFailed) > from gbp.errors import GbpError > import gbp.log > @@ -171,6 +173,18 @@ def commit_patches(repo, branch, patches, options): > return added, removed > > > +def find_upstream_commit(repo, cp, upstream_tag): > + """ > + Find commit corresponding upstream version > + """ > + > + upstream_commit = repo.find_version(upstream_tag, cp.upstream_version) > + if not upstream_commit: > + raise GbpError("Couldn't find upstream version %s" % > + cp.upstream_version) > + return upstream_commit > + > + > def export_patches(repo, branch, options): > """Export patches from the pq branch into a patch series""" > if is_pq_branch(branch): > @@ -234,7 +248,8 @@ def safe_patches(series): > return (tmpdir, series) > > > -def import_quilt_patches(repo, branch, series, tries, force): > +def import_quilt_patches(repo, branch, series, tries, force, pq_from, > + upstream_tag): > """ > apply a series of quilt patches in the series file 'series' to branch > the patch-queue branch for 'branch' > @@ -245,6 +260,10 @@ def import_quilt_patches(repo, branch, series, tries, > force): > @param tries: try that many times to apply the patches going back one > commit in the branches history after each failure. > @param force: import the patch series even if the branch already exists > + @param pq_from: what to use as the starting point for the pq branch. > + DEBIAN indicates the current branch, TAG indicates that > + the corresponding upstream tag should be used. > + @param upstream_tag: upstream tag template to use > """ > tmpdir = None > > @@ -267,10 +286,16 @@ def import_quilt_patches(repo, branch, series, tries, > force): > % pq_branch) > > maintainer = get_maintainer_from_control(repo) > - commits = repo.get_commits(num=tries, first_parent=True) > + if pq_from.upper() == 'TAG': > + vfs = gbp.git.vfs.GitVfs(repo, branch) > + source = DebianSource(vfs) > + commits = [find_upstream_commit(repo, source.changelog, > upstream_tag)] > + else: # pq_from == 'DEBIAN' > + commits = repo.get_commits(num=tries, first_parent=True) > # If we go back in history we have to safe our pq so we always try to > apply > # the latest one > - if len(commits) > 1: > + # If we are using the upstream_tag, we always need a copy of the patches > + if len(commits) > 1 or pq_from.upper() == 'TAG': > tmpdir, series = safe_patches(series) > > queue = PatchSeries.read_series_file(series) > @@ -354,6 +379,8 @@ def build_parser(name): > dest="color_scheme") > parser.add_config_file_option(option_name="meta-closes", > dest="meta_closes") > parser.add_config_file_option(option_name="meta-closes-bugnum", > dest="meta_closes_bugnum") > + parser.add_config_file_option(option_name="pq-from", dest="pq_from") > + parser.add_config_file_option(option_name="upstream-tag", > dest="upstream_tag") > return parser > > > @@ -392,7 +419,7 @@ def main(argv): > return 1 > > try: > - repo = GitRepository(os.path.curdir) > + repo = DebianGitRepository(os.path.curdir) > except GitRepositoryError: > gbp.log.err("%s is not a git repository" % (os.path.abspath('.'))) > return 1 > @@ -404,7 +431,9 @@ def main(argv): > elif action == "import": > series = SERIES_FILE > tries = options.time_machine if (options.time_machine > 0) else 1 > - num = import_quilt_patches(repo, current, series, tries, > options.force) > + num = import_quilt_patches(repo, current, series, tries, > + options.force, options.pq_from, > + options.upstream_tag) > current = repo.get_branch() > gbp.log.info("%d patches listed in '%s' imported on '%s'" % > (num, series, current)) > -- > 2.8.1 > > >From ce9df8109929c3ed623b241da53291e51578e676 Mon Sep 17 00:00:00 2001 > From: Maximiliano Curia <[email protected]> > Date: Sun, 14 Aug 2016 16:30:51 +0200 > Subject: [PATCH 2/4] pq: Handle unmerged debian branches (rebase) > > --- > gbp/scripts/pq.py | 16 +++++++++++++--- > 1 file changed, 13 insertions(+), 3 deletions(-) > > diff --git a/gbp/scripts/pq.py b/gbp/scripts/pq.py > index 84da6a0..d756ca3 100755 > --- a/gbp/scripts/pq.py > +++ b/gbp/scripts/pq.py > @@ -336,13 +336,23 @@ def import_quilt_patches(repo, branch, series, tries, > force, pq_from, > return len(queue) > > > -def rebase_pq(repo, branch): > +def rebase_pq(repo, branch, pq_from, upstream_tag): > + > if is_pq_branch(branch): > base = pq_branch_base(branch) > else: > switch_to_pq_branch(repo, branch) > base = branch > - GitCommand("rebase")([base]) > + > + if pq_from.upper() == 'TAG': > + vfs = gbp.git.vfs.GitVfs(repo, base) > + source = DebianSource(vfs) > + upstream_commit = find_upstream_commit(repo, source.changelog, > upstream_tag) > + _from = upstream_commit > + else: > + _from = base > + > + GitCommand("rebase")([_from]) > > > def build_parser(name): > @@ -440,7 +450,7 @@ def main(argv): > elif action == "drop": > drop_pq(repo, current) > elif action == "rebase": > - rebase_pq(repo, current) > + rebase_pq(repo, current, options.pq_from, options.upstream_tag) > elif action == "apply": > patch = Patch(patchfile) > maintainer = get_maintainer_from_control(repo) > -- > 2.8.1 > > >From 5fe51cb8de3745c3bd33c5c28352d5d270a11d42 Mon Sep 17 00:00:00 2001 > From: Maximiliano Curia <[email protected]> > Date: Thu, 18 Aug 2016 12:53:16 +0200 > Subject: [PATCH 3/4] pq: Handle unmerged debian branches (export) > > --- > gbp/scripts/pq.py | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/gbp/scripts/pq.py b/gbp/scripts/pq.py > index d756ca3..9a97c09 100755 > --- a/gbp/scripts/pq.py > +++ b/gbp/scripts/pq.py > @@ -202,7 +202,16 @@ def export_patches(repo, branch, options): > else: > gbp.log.debug("%s does not exist." % PATCH_DIR) > > - patches = generate_patches(repo, branch, pq_branch, PATCH_DIR, options) > + if options.pq_from.upper() == 'TAG': > + vfs = gbp.git.vfs.GitVfs(repo, branch) > + source = DebianSource(vfs) > + upstream_commit = find_upstream_commit(repo, source.changelog, > + options.upstream_tag) > + base = upstream_commit > + else: > + base = branch > + > + patches = generate_patches(repo, base, pq_branch, PATCH_DIR, options) > > if patches: > with open(SERIES_FILE, 'w') as seriesfd: > -- > 2.8.1 > > >From 43054bf8ffe225e4432170c44b72981b7c69e5ee Mon Sep 17 00:00:00 2001 > From: Maximiliano Curia <[email protected]> > Date: Thu, 18 Aug 2016 13:11:20 +0200 > Subject: [PATCH 4/4] pq: Handle unmerged debian branches (documentation) > > --- > docs/manpages/gbp-pq.sgml | 26 ++++++++++++++++++++++++++ > 1 file changed, 26 insertions(+) > > diff --git a/docs/manpages/gbp-pq.sgml b/docs/manpages/gbp-pq.sgml > index 76f6dfb..4c1c79e 100644 > --- a/docs/manpages/gbp-pq.sgml > +++ b/docs/manpages/gbp-pq.sgml > @@ -29,6 +29,8 @@ > <arg><option>--force</option></arg> > <arg><option>--meta-closes=bug-close-tags</option></arg> > <arg><option>--meta-closes-bugnum=bug-number-format</option></arg> > + > <arg><option>--pq-from=</option><replaceable>[DEBIAN|TAG]</replaceable></arg> > + > <arg><option>--upstream-tag=</option><replaceable>tag-format</replaceable></arg> > <group choice="plain"> > <arg><option>drop</option></arg> > <arg><option>export</option></arg> > @@ -230,6 +232,30 @@ > </para> > </listitem> > </varlistentry> > + <varlistentry> > + > <term><option>--pq-from=</option><replaceable>[DEBIAN|TAG]</replaceable> > + </term> > + <listitem> > + <para> > + How to find the starting point for the patch queue base. The > options are DEBIAN, that will use the Debian branch as the base for the patch > queue branch, and TAG, that will use the corresponding upstream tag as a base > for the patch queue branch. > + </para> > + <para> > + This is only needed if your upstream branch is not merged in the > Debian branch. > + The default is <replaceable>DEBIAN</replaceable>. > + </para> > + </listitem> > + </varlistentry> > + </variablelist> > + <varlistentry> > + > <term><option>--upstream-tag=</option><replaceable>TAG-FORMAT</replaceable> > + </term> > + <listitem> > + <para> > + Use this tag format when looking for tags of upstream versions, > + default is <replaceable>upstream/%(version)s</replaceable>. > + </para> > + </listitem> > + </varlistentry> > </variablelist> > </refsect1> > <refsect1> > -- > 2.8.1 >
