Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
Quick follow up. Since yesterday, I filed a few bugs on the git-dpm package and already got back some useful information. * tag format This is configurable, so it's easy to get the gbp style tags. These commands set the style in the repo so I think it should be propagated to anybody who checks it out: git config dpm.debianTag debian/%e%v git config dpm.patchedTag patched/%e%v git config dpm.upstreamTag upstream/%e%u * patch names There are several ways of retaining the original debian/patch file names. While they can be cleaned up after the fact[*], the easiest way is just to use git-dpm import-dsc --record-patch-name. This adds a Patch-Name header to the DEP-8 fields, with the patch's name. This way it survives the checkout-patched to update-patches round trip. I've made both changes to my import-dscs.py script. http://anonscm.debian.org/cgit/users/barry/import-dscs.git/tree/ Cheers, -Barry [*] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=760578#20 signature.asc Description: PGP signature
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
Hi Martin, thanks for the information. On Sep 05, 2014, at 05:18 PM, Martin Pitt wrote: >gitpkg is rather complicated to use and set up, only about 3 people in >Debian know how it works properly, and it makes it really hard to >track a set of changes against trunk over time (i. e. the equivalent >of a quilt series, or stacked git). I kept hearing about git-debcherry at DC14. AFAICT, this is a committed, but not yet released part of gitpkg. But it sounds like from your experience, gitpkg may not be appropriate for the debian-python team. Cheers, -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140905124323.7adc3...@anarchist.wooz.org
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On 05/09/14 16:18, Martin Pitt wrote: > I don't think anyone in pkg-systemd@ has looked at git-dpm yet. In > fact we switched from gitpkg to standard git-buildpackage. Ugh, sorry. > So I'm not sure where "switched from git-dpm" came from? "smcv mis-remembering the situation", evidently. S -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5409d865.5010...@debian.org
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
Hey all, Simon McVittie [2014-09-05 16:05 +0100]: > >> It might also be worth noting that the systemd maintainers switched from > >> git-dpm to gbp-pq recently (between 204 and 208, I think), so they > >> obviously didn't think git-dpm was the better option. I don't think anyone in pkg-systemd@ has looked at git-dpm yet. In fact we switched from gitpkg to standard git-buildpackage. gbp-pq is a local development tool which greatly eases patch handling especially for new upstream versions, but as that's not exposed in the official git, but is only a glorified way of maintaining the debian/patches quilt series locally that shouldn't affect other workflows. gitpkg is rather complicated to use and set up, only about 3 people in Debian know how it works properly, and it makes it really hard to track a set of changes against trunk over time (i. e. the equivalent of a quilt series, or stacked git). So I'm not sure where "switched from git-dpm" came from? Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140905151824.gh3...@piware.de
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On 05/09/14 15:53, Barry Warsaw wrote: > On Sep 05, 2014, at 01:21 PM, Simon McVittie wrote: > >> It might also be worth noting that the systemd maintainers switched from >> git-dpm to gbp-pq recently (between 204 and 208, I think), so they >> obviously didn't think git-dpm was the better option. > > Are there any artifacts of this switch, e.g. mailing list archives, wiki > pages, etc.? I'd love to read some background on why they switched. Sorry, I'm just a bystander (in both Python and systemd). systemd maintainers, for context: the Python modules/apps packaging teams are discussing pros and cons of the git-dpm or gbp-pq repo layout for packages in git, and Barry has so far been advocating git-dpm. Since systemd switched from git-dpm to gbp-pq recently, do you have any input on why you decided against it? (Probably best to reply to d-python only, since I realise this is way outside the scope of pkg-systemd-maintainers - Reply-To set.) S -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5409d13a.5080...@debian.org
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Sep 05, 2014, at 01:21 PM, Simon McVittie wrote: >It might also be worth noting that the systemd maintainers switched from >git-dpm to gbp-pq recently (between 204 and 208, I think), so they >obviously didn't think git-dpm was the better option. Are there any artifacts of this switch, e.g. mailing list archives, wiki pages, etc.? I'd love to read some background on why they switched. >The systemd package is an interesting stress-test for patch systems, >because: > >* upstream don't do formal micro releases (there is no v208.1 and > probably never will be) but they do cherry-pick a lot of bugfixes to > a stable-branch (e.g. v208-stable), so the Debian maintainers apply > patches from the upstream v208-stable branch in bulk; > >* the Debian maintainers also apply a significant number of local > patches to preserve historical functionality of Debian's udev and > sysvinit, some of which are never going to go upstream > >so managing its patch-set is non-trivial. This might mean that the right >decision for systemd is not the same as the right decision as for a >package that will hopefully only have a couple of Debian patches; I >don't know. I think the overwhelming majority of our packages come from tarballs released on PyPI, so we'll have a fairly regular workflow. Ignoring outliers[*], I think we generally won't have tons of cherry picked patches in our packages, and hopefully no need for tons of local deviation, in the common cases. This is also why I feel strongly that we should usually use a pristine-tar branch. Whatever regime and workflow we pick, let's optimize for the common, i.e. PyPI-published case. Cheers, -Barry [*] A policy we discussed at DC14 was to allow for deviation from team policy in special cases, but only if the reason and differing workflow was documented in d/README.source. So whatever we choose, we acknowledge that some packages need special handling. -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140905105353.5732a...@anarchist.wooz.org
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Sep 05, 2014, at 01:10 PM, Simon McVittie wrote: >The systemd maintainers configured git-buildpackage (in their >debian/gbp.conf) to not use patch numbers. I'm starting to think that's >The Right Thing in general. Agreed. I've filed wishlist bug #760578 for this, and other enhancements to patch file naming. Cheers, -Barry -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140905104558.0099b...@anarchist.wooz.org
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On 05/09/14 13:10, Simon McVittie wrote: > On 04/09/14 20:40, Barry Warsaw wrote: >> The file is patched, but now I have an d/p/0005- file instead of a modified >> 0003- patch file. Sigh. > > The systemd maintainers [...] It might also be worth noting that the systemd maintainers switched from git-dpm to gbp-pq recently (between 204 and 208, I think), so they obviously didn't think git-dpm was the better option. The systemd package is an interesting stress-test for patch systems, because: * upstream don't do formal micro releases (there is no v208.1 and probably never will be) but they do cherry-pick a lot of bugfixes to a stable-branch (e.g. v208-stable), so the Debian maintainers apply patches from the upstream v208-stable branch in bulk; * the Debian maintainers also apply a significant number of local patches to preserve historical functionality of Debian's udev and sysvinit, some of which are never going to go upstream so managing its patch-set is non-trivial. This might mean that the right decision for systemd is not the same as the right decision as for a package that will hopefully only have a couple of Debian patches; I don't know. S -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5409aada.5020...@debian.org
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On 04/09/14 20:40, Barry Warsaw wrote: > The file is patched, but now I have an d/p/0005- file instead of a modified > 0003- patch file. Sigh. The systemd maintainers configured git-buildpackage (in their debian/gbp.conf) to not use patch numbers. I'm starting to think that's The Right Thing in general. S -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5409a839.6020...@debian.org
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Sep 05, 2014, at 12:25 AM, Raphael Hertzog wrote: >As others have mentionned, you should use "git rebase -i ". This is >what you want to use on your patch-queue branch to modifiy individual >commits, reorder them, or drop them. Brilliant. For git-dpm then this would be: $ git-dpm checkout-patched $ git rebase -i upstream # *not* master! $ git-dpm update-patches Joy! Cheers, -Barry signature.asc Description: PGP signature
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Thu, 04 Sep 2014, Barry Warsaw wrote: > On Sep 04, 2014, at 04:36 PM, Scott Kitterman wrote: > > >Actually, nevermind. That's not the problem you were trying to solve, > >although you could remove the patch as described and then apply the updated > >patch at the end of the series. > > Yeah, though sometimes for legitimate reasons you can't reorder patches. It > vaguely feels like with git-dpm since the patch branch is never pushed, you > could "uncommit" (`git reset --hard HEAD^`) and stash each commit until you > got to the one you wanted to amend, then unstash and recommit back up the > stack. E.g. just like quilt push/pop. As others have mentionned, you should use "git rebase -i ". This is what you want to use on your patch-queue branch to modifiy individual commits, reorder them, or drop them. Cheers, -- Raphaël Hertzog ◈ Debian Developer Discover the Debian Administrator's Handbook: → http://debian-handbook.info/get/ -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140904222559.ga7...@x230-buxy.home.ouaza.com
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Sep 04, 2014, at 04:36 PM, Scott Kitterman wrote: >Actually, nevermind. That's not the problem you were trying to solve, >although you could remove the patch as described and then apply the updated >patch at the end of the series. Yeah, though sometimes for legitimate reasons you can't reorder patches. It vaguely feels like with git-dpm since the patch branch is never pushed, you could "uncommit" (`git reset --hard HEAD^`) and stash each commit until you got to the one you wanted to amend, then unstash and recommit back up the stack. E.g. just like quilt push/pop. -Barry signature.asc Description: PGP signature
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Thursday, September 04, 2014 16:05:53 Scott Kitterman wrote: > On Thursday, September 04, 2014 15:40:42 Barry Warsaw wrote: > > That gets you a source package, but the binary package FTBFS because one > > additional test cannot be run during the build process (there's a DEP-8 > > test for full coverage). Now though, you *must* commit or stash the > > d/changelog change. > > > > Here's where things get a little frustrating. What I need to do is patch > > a > > file to add a line that skips a test. I'd like to append that to the > > 0003- > > patch but it's not the top of the quilt stack (there's an 0004- patch). > > But this doesn't seem possible, and `git-dpm apply-patch` doesn't seem to > > do this. Besides, it's very inconvenient to name the full d/p/*.patch > > file you want to apply. What I really want is the equivalent of `quilt > > push`. > > > > The best it seems you can do is: > > > > $ git-dpm checkout-patched > > > > $ git-dpm update-patches > > > > The file is patched, but now I have an d/p/0005- file instead of a > > modified > > 0003- patch file. Sigh. > > I did already run into this with pkg-clamav. See the patch in > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=754052 for one way to do > it. > > Scott K Actually, nevermind. That's not the problem you were trying to solve, although you could remove the patch as described and then apply the updated patch at the end of the series. Scott K signature.asc Description: This is a digitally signed message part.
Re: git-dpm vs gbp-pq: new upstream and patch refresh (long)
On Thursday, September 04, 2014 15:40:42 Barry Warsaw wrote: > That gets you a source package, but the binary package FTBFS because one > additional test cannot be run during the build process (there's a DEP-8 test > for full coverage). Now though, you *must* commit or stash the d/changelog > change. > > Here's where things get a little frustrating. What I need to do is patch a > file to add a line that skips a test. I'd like to append that to the 0003- > patch but it's not the top of the quilt stack (there's an 0004- patch). But > this doesn't seem possible, and `git-dpm apply-patch` doesn't seem to do > this. Besides, it's very inconvenient to name the full d/p/*.patch file you > want to apply. What I really want is the equivalent of `quilt push`. > > The best it seems you can do is: > > $ git-dpm checkout-patched > > $ git-dpm update-patches > > The file is patched, but now I have an d/p/0005- file instead of a modified > 0003- patch file. Sigh. I did already run into this with pkg-clamav. See the patch in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=754052 for one way to do it. Scott K signature.asc Description: This is a digitally signed message part.
git-dpm vs gbp-pq: new upstream and patch refresh (long)
tox has a new upstream so I decided to take the opportunity to A/B git-dpm and gbp-pq on a more complicated, but probably common task, simply stated:: upgrade to the new upstream, refresh the patches, handling any conflicts, and regenerate a source package for testing. TL;DR: You can make things work with both workflows, but both are very picky about the order in which you do things. Documentation needs to be better too. I've formed my opinion on a way forward, which I'll describe in a follow up message. == git-dpm == Since tox is still maintained under svn, I wanted to jump start maintenance under git-dpm. For this experiment, I don't care about detailed history. There's no `git-dpm import-dscs --debsnap` so I wrote my own: $ git clone git://git.debian.org/users/barry/import-dscs.git will give you import-dscs.py which will effectively use snapshots to get all the uploaded versions and call `git-dpm import-dsc --ptc` in a loop (with tagging). Then it calls `git-dpm prepare` and the repo should be ready to go. Of course, my script could be doing stupid things. WARNING: import-dscs.py requires an unreleased bug fix to python3-apt, so you'll need to grab the python-apt git repo, build, and install the latest head, otherwise the snapshots will not be applied in the correct order. http://tinyurl.com/mjoms6h (You'll also need packages like pristine-tar, git-dpm, git-buildpackage, and devscripts.) In any case, now I do the following: $ git init toxdpm $ cd toxdpm $ .../import-dscs.py tox Right, so now its time to switch to the new upstream. git-dpm doesn't have a --uscan option, so: $ uscan $ git-dpm import-new-upstream --rebase ../tox_1.7.2.orig.tar.gz There will be merge conflicts, so you need to resolve those manually, calling `git add ` and `git rebase --continue` after each resolution. You may not know exactly how to resolve each conflict, but I don't think the details matter. Once everything's resolved: $ git-dpm update-patches leaves you on the master branch. Now *before* you try to build the source package, you must add a new d/changelog entry, otherwise git-buildpackage will build the old version, and the quilt stack won't apply cleanly. So: $ dch -v1.7.2-1 $ git-buildpackage --git-ignore-new --git-export=WC -S -us -uc (or commit and just `git-buildpackage -S -us -uc`) That gets you a source package, but the binary package FTBFS because one additional test cannot be run during the build process (there's a DEP-8 test for full coverage). Now though, you *must* commit or stash the d/changelog change. Here's where things get a little frustrating. What I need to do is patch a file to add a line that skips a test. I'd like to append that to the 0003- patch but it's not the top of the quilt stack (there's an 0004- patch). But this doesn't seem possible, and `git-dpm apply-patch` doesn't seem to do this. Besides, it's very inconvenient to name the full d/p/*.patch file you want to apply. What I really want is the equivalent of `quilt push`. The best it seems you can do is: $ git-dpm checkout-patched $ git-dpm update-patches The file is patched, but now I have an d/p/0005- file instead of a modified 0003- patch file. Sigh. git-buildpackage && sbuild && adt-run and everything passes. We're done. == gbp pq == gbp-pq seems *much* more finicky about how to get a new upstream with updated patches. If you do not do things in exactly the right order, your new upstream merge will be totally broken. Or, at least they were for me. I had a ton of trial-and-error before I got this incantation correct. git-buildpackage has the bootstrapping step built-in: $ gbp import-dscs --debsnap tox $ gbp pq import $ gbp pq export This renames the d/patches files, and you *must* do this before you try to update to a new upstream your you'll be screwed. If you do it out of order, `gbp pq import` and `gbp pq rebase` will either do nothing, do the wrong thing (e.g. delete d/patches with no renames), or fail (you'll get patch application conflict errors with no way left to resolve them). $ git add debian/patches/ $ git stat On branch master Changes to be committed: (use "git reset HEAD ..." to unstage) renamed:debian/patches/intersphinx_mapping.patch -> debian/patches/0001-intersphinx_mapping.patch renamed:debian/patches/hack-requires.patch -> debian/patches/0002-hack-requires.patch renamed:debian/patches/build-time-tests.patch -> debian/patches/0003-build-time-tests.patch renamed:debian/patches/de-google-ify.patch -> debian/patches/0004-de-google-ify.patch modified: debian/patches/series $ git commit -a -m"patch import/export" Now get the new upstream. $ gbp import-orig --uscan $ dch -v1.7.2-1 "New upstream release." $ git commit -a -m"New upstream release." Now we should be able to rebase the patches onto 1.7.2: $ gbp pq rebase (I largely resolve the merge conflicts in the same way as with git-dpm.) $ gbp pq export $ git
