Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Barry Warsaw]

On Aug 31, 2016, at 11:31 PM, IOhannes m zmölnig (Debian/GNU) wrote:

>isn't this what `gbp pull` is supposed to do?

Indeed.  Either this is new-ish or I missed it, but it does exactly what I
want.  Thanks!

-Barry


pgpvGDUnhftFZ.pgp
Description: OpenPGP digital signature

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Sergio Durigan Junior]

On Friday, August 12 2016, Barry Warsaw wrote:

> On Aug 12, 2016, at 05:50 PM, Sergio Durigan Junior wrote:
>
>>I understand this is a matter of personal taste, but I beg to differ.  I
>>have been using git-buildpackage for most of my non-Python packages and,
>>despite really small nits here and there, I think it is an awesome tool.
>
> I think there are multiple ways in which git-buildpackage is used.  E.g. I use
> `gbp buildpackage` even for git-dpm maintained packages, to build the source
> package, etc.  (It even has a nicer `clone` command that makes it easy to
> actually get tracking branches; just wish it had something like
> `pull-and-update-all-branches` since it's a bit of a PITA to update the
> pristine-tar branch.)

Yeah, I use 'gbp dch' and 'gbp buildpackage' when dealing with git-dpm
packaging, too.

> It's just the few things that git-dpm adds on top of gbp that we're discussing
> getting rid of, like importing a new upstream, managing the quilt patch set,
> tagging.

Right, that's what I thought, and those things are exactly what make me
unhappy about using git-dpm :-).

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/


signature.asc
Description: PGP signature

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Barry Warsaw]

On Aug 15, 2016, at 11:44 PM, Thomas Goirand wrote:

>If we decide to use gbp-pq, in fact, we're deciding to not decide, and anyone
>can use PoQ (my choice, for example). Indeed, the way to store the patches is
>PoQ, and you then "gbp-pq import", modify, then "gbp-pq export" and store the
>packaging branch like this (ie: like a PoQ branch). So, basically, we'll be
>back to what everyone else is doing (ie: 99% of git maintained packages in
>Debian as much as I saw).

That sounds perfect then!

>IMO, that's required if we decide to continue using pristine-tar (which
>I don't think is a good idea, but let's not discuss that, as there seem
>to be a consensus for it).

So at a minimum:

gbp.conf
[import-orig]
pristine-tar = True
gbp.conf

Looking at my ~/.gbp.conf file, I have a bunch of other entries, some of which
are probably not appropriate for team settings.  I'd suggest `sign-tags=True`
though.

*Maybe* also these as conveniences:

[import-orig]
postimport = gbp dch -N%(version)s
import-msg = New upstream release.

What else?

Cheers,
-Barry


pgp_uaUuGilbj.pgp
Description: OpenPGP digital signature

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Thomas Goirand]

On 08/10/2016 10:41 PM, Barry Warsaw wrote:
> * IOW, if
>   you choose to use gbp-pq, am I forced to do so when I modify the same repo?
>   Or if you choose to use PoQ (plain 'ol quilt), will that affect how others
>   can co-maintain the package in git?

That's the point. If we decide to use gbp-pq, in fact, we're deciding to
not decide, and anyone can use PoQ (my choice, for example). Indeed, the
way to store the patches is PoQ, and you then "gbp-pq import", modify,
then "gbp-pq export" and store the packaging branch like this (ie: like
a PoQ branch). So, basically, we'll be back to what everyone else is
doing (ie: 99% of git maintained packages in Debian as much as I saw).

On 08/11/2016 01:12 AM, Simon McVittie wrote:
> no other special metadata present in git (you can optionally commit a
>   debian/gbp.conf, and I would recommend it, but it isn't required)

IMO, that's required if we decide to continue using pristine-tar (which
I don't think is a good idea, but let's not discuss that, as there seem
to be a consensus for it).

On 08/11/2016 01:12 AM, Simon McVittie wrote:
> Not good for gbp-pq (it works OK, but an import/export round-trip will
> mangle the metadata if you don't take steps to preserve it):
>
> Author: Donald Duck 
> Description: Reticulate splines correctly
>  This regressed in 2.0.
>  .
>  In particular, this broke embiggening.
> Last-update: Fri, 01 Apr 2016 12:34:00 +0100
> Origin: vendor, Debian
> Forwarded: http://bugs.example.org/123
> ---
> [diff goes here]
>
> Regards,
> S

If this is what gbp pq does, then that's the *right* thing (I don't
remember, probably because it didn't destroy my Debian headers, which I
carefully crafted by hand...).

Cheers,

Thomas

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Barry Warsaw]

On Aug 12, 2016, at 05:50 PM, Sergio Durigan Junior wrote:

>I understand this is a matter of personal taste, but I beg to differ.  I
>have been using git-buildpackage for most of my non-Python packages and,
>despite really small nits here and there, I think it is an awesome tool.

I think there are multiple ways in which git-buildpackage is used.  E.g. I use
`gbp buildpackage` even for git-dpm maintained packages, to build the source
package, etc.  (It even has a nicer `clone` command that makes it easy to
actually get tracking branches; just wish it had something like
`pull-and-update-all-branches` since it's a bit of a PITA to update the
pristine-tar branch.)

It's just the few things that git-dpm adds on top of gbp that we're discussing
getting rid of, like importing a new upstream, managing the quilt patch set,
tagging.

Cheers,
-Barry


pgp7ppU1EAjvh.pgp
Description: OpenPGP digital signature

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Sergio Durigan Junior]

On Wednesday, August 10 2016, Nikolaus Rath wrote:

> I don't believe that switching from git-dpm to git-buildpackage is going
> to make things easier, it'll just be trading one set of problems for
> another.

I understand this is a matter of personal taste, but I beg to differ.  I
have been using git-buildpackage for most of my non-Python packages and,
despite really small nits here and there, I think it is an awesome tool.

git-dpm, OTOH, has several limitations (as already mentioned by others,
it's not trivial to merge changes to local patches, and it can be quite
complicated to import a new upstream version into the repository).
I have had to package some Python packages these last weeks, and every
time I had to deal with git-dpm I almost always stumbled upon some
idiosyncrasy that got in the way of my work.

Long story short: I am totally favorable to make the switch to
git-buildpackage (in fact, I recently raised this topic on IRC but the
conversation eventually died).

Cheers,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/


signature.asc
Description: PGP signature

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Brian May]

Raphael Hertzog  writes:

> Anyway, +1 from me to switch to git-buildpackage and in fact among the
> python-django maintainers we are close to decide to switch back to
> git-buildpackage because it's really horrible to use when you want to
> merge across branches of different releases.

Have people here seen the following thread on debian-devel?

Subject: "[ANNOUNCE] git-series: track changes to a patch series over
time"

Might be worth looking at this thread, even if the conclusion is that
this is not a good solution (actually I don't really understand it and
haven't time to understand it).
-- 
Brian May 

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Simon McVittie]

On Thu, 11 Aug 2016 at 09:29:13 -0400, Barry Warsaw wrote:
> On Aug 11, 2016, at 12:12 AM, Simon McVittie wrote:
> >where all Debian-specific pseudo-headers appear at the end of the diff
> >(next to the Signed-off-by if any),
> 
> Did you mean to say "end of the diff headers"?

Yes, that's what I should have written. It needs to look like the
Signed-off-by on a conventional git patch submission.

> the DEP-3 headers are *before* the actual diff, separated from the diff
> headers by a blank line.

Separated by a line containing exactly "---" (which is mandated by DEP-3
and is one of several syntaxes accepted by git-am), but yes. Here's a
typical real-world example:
https://anonscm.debian.org/cgit/collab-maint/flatpak.git/tree/debian/patches/debian/Try-gtk-3.0-version-of-the-icon-cache-utility-first.patch

S

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Barry Warsaw]

Thanks for the follow up Simon.  One question...

On Aug 11, 2016, at 12:12 AM, Simon McVittie wrote:

>gbp pq works best if all repository users stick to the dialect of DEP-3
>where all Debian-specific pseudo-headers appear at the end of the diff
>(next to the Signed-off-by if any),

Did you mean to say "end of the diff headers"?  I ask because in your "Good
for gbp-pq" example:

>From: Donald Duck 
>Date: Fri, 01 Apr 2016 12:34:00 +0100
>Subject: Reticulate splines correctly
>
>This regressed in 2.0.
>
>In particular, this broke embiggening.
>
>Origin: vendor, Debian
>Forwarded: http://bugs.example.org/123
>---
>[diff goes here]

the DEP-3 headers are *before* the actual diff, separated from the diff
headers by a blank line.

Cheers,
-Barry

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Simon McVittie]

On Wed, 10 Aug 2016 at 16:41:40 -0400, Barry Warsaw wrote:
> * With git-dpm we *had* to enforce the tool choice because git-dpm's artifacts
>   had to be preserved. If we ditch git-dpm, is that still the case?  IOW, if
>   you choose to use gbp-pq, am I forced to do so when I modify the same repo?

You do not have to choose gbp-pq. You do have to use some tool that
copes with:

* a git repository with patches unapplied but present in debian/patches/
* no other special metadata present in git (you can optionally commit a
  debian/gbp.conf, and I would recommend it, but it isn't required)

In particular this rules out dgit (which wants a patches-applied tree)
and git-dpm (which wants a patches-applied tree with its own metadata).

In practice this means you can build with either gbp buildpackage, or plain
dpkg-buildpackage/debuild; and you can manage the patches either with gbp pq,
with quilt, or (in simple cases) by running git format-patch in an
upstream-tracking repository, dropping the results into debian/patches/
and modifying debian/patches/series with a text editor.

gbp pq works best if all repository users stick to the dialect of DEP-3
where all Debian-specific pseudo-headers appear at the end of the diff
(next to the Signed-off-by if any), so that it looks a lot like git
format-patch output (canonically with the leading From_ line and the
trailing signature omitted, although if they're present in input it
will of course cope). This is basically also what git-dpm generates,
so it should be familiar to DPMT people already. Good for gbp-pq:

From: Donald Duck 
Date: Fri, 01 Apr 2016 12:34:00 +0100
Subject: Reticulate splines correctly

This regressed in 2.0.

In particular, this broke embiggening.

Origin: vendor, Debian
Forwarded: http://bugs.example.org/123
---
[diff goes here]

Not good for gbp-pq (it works OK, but an import/export round-trip will
mangle the metadata if you don't take steps to preserve it):

Author: Donald Duck 
Description: Reticulate splines correctly
 This regressed in 2.0.
 .
 In particular, this broke embiggening.
Last-update: Fri, 01 Apr 2016 12:34:00 +0100
Origin: vendor, Debian
Forwarded: http://bugs.example.org/123
---
[diff goes here]

Regards,
S

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Raphael Hertzog]

On Wed, 10 Aug 2016, Nikolaus Rath wrote:
> I don't believe that switching from git-dpm to git-buildpackage is going
> to make things easier, it'll just be trading one set of problems for
> another.

I don't agree on this. I have been a happy git-buildpackage user for all
my packages. The problem with git-dpm is that the patch series can't be easily
fixed after a merge when it generates conflicts. It's too intricately tied
to git-dpm's own fake merge logic with commit it recorded in
debian/.git-dpm and it's very painful to bring the repository in a state
where git-dpm is happy.

With git-buildpackage, the merge and the update of the patch series are
distinct operations that can be done at different times. Since patches are
unapplied, the merge usually works fine and the patch series
can be easily rebased afterwards with your tool of choice.

Anyway, +1 from me to switch to git-buildpackage and in fact among the
python-django maintainers we are close to decide to switch back to
git-buildpackage because it's really horrible to use when you want to
merge across branches of different releases.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Barry Warsaw]

On Aug 10, 2016, at 11:53 AM, Nikolaus Rath wrote:

>I think the only way to make this less painful is to get rid of the idea
>of managing patches in a VCS and use something like gitpkg. This has the
>drawback source package is now *generated* from the Git repository
>(i.e., you can't do git clone + debuild), but it makes maintaining the
>Git repository much less painful. Judging from all the attempts I've
>seen so far, trying to put patches (i.e., the output of a VCS) under
>version-control is just a doomed endeavor.
>
>I don't believe that switching from git-dpm to git-buildpackage is going
>to make things easier, it'll just be trading one set of problems for
>another.

Two thoughts:

* We probably need to do some actual experimentation on actual packages, so
  we should plan for allowing that on a limited basis, with the caveat that
  once a new workflow is chosen, we'll make all team packages consistent
  again.

* With git-dpm we *had* to enforce the tool choice because git-dpm's artifacts
  had to be preserved.  If we ditch git-dpm, is that still the case?  IOW, if
  you choose to use gbp-pq, am I forced to do so when I modify the same repo?
  Or if you choose to use PoQ (plain 'ol quilt), will that affect how others
  can co-maintain the package in git?

We only need to mandate workflows and conventions where the effects and
artifacts are visible after a package is updated.  Think of it like the choice
of editor - no one cares whether you use vim, emacs, gedit, or whatever to
modify the files, because its effects are only local to you (for the most
part).

Cheers,
-Barry

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT

[Nikolaus Rath]

On Aug 10 2016, Thomas Goirand  wrote:
> As I only heard complains about git-dpm, maybe someone would like to
> express his joy using it, and explain why they think it's a nice tool.
> But is there such person? It seems git-dpm only brings frustration.

In my opinion, git-dpm solves the problem of keeping a full history of
Debian source packages (i.e., patches applied and debian/patches/
populated) as well as possible. This means that it is often painful.

I think the only way to make this less painful is to get rid of the idea
of managing patches in a VCS and use something like gitpkg. This has the
drawback source package is now *generated* from the Git repository
(i.e., you can't do git clone + debuild), but it makes maintaining the
Git repository much less painful. Judging from all the attempts I've
seen so far, trying to put patches (i.e., the output of a VCS) under
version-control is just a doomed endeavor.


I don't believe that switching from git-dpm to git-buildpackage is going
to make things easier, it'll just be trading one set of problems for
another.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT (was: PAPT Git)

[Barry Warsaw]

On Aug 10, 2016, at 08:49 PM, Brian May wrote:

>Most of the time it works pretty well... It looked good compared with
>the alternatives available at the time we made the decision.
>
>However this is irrelevant IMHO if it isn't being mantained.

Yep.  git-dpm was the best of breed at the time we were switching from svn,
and several developers had good experiences with it.  When things are fairly
simple, so is git-dpm, and when it Just Works, it's easy to use.  When it's
*not* -- or when you hit any of the bugs previously mentioned -- then you're
out of luck.

git-dpm is no longer maintained so those bugs won't get fixed.  And gbp-pq has
improved a lot since then.

Moving PAPT to git without git-dpm will gain team experience with that
toolset.  IIRC we figured it was as easy as `rm debian/.git-dpm` to switch,
and we should test that on a few candidate packages.  More important is to
update the documentation:

https://wiki.debian.org/Python/GitPackaging

so we're all on the same page.

Cheers,
-Barry

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT (was: PAPT Git)

[Brian May]

Thomas Goirand  writes:

> git-dpm also fails to tag upstream/foo automatically when importing a
> new version. I've been told to use "git-dpm tag", but that's not
> obvious. My own experience managing debian/patches quilt patches
> manually or through gbp pq is actually much much nicer.

The problem with git-dpm tag is that this tags the Debian release too -
sometimes you want to tag the upstream version and aren't ready to
release a Debian version just yet.

> Plus it appears git-dpm is unmaintained and even sometimes buggy

That is the big problem. Nobody maintaining it or responding to bug
reports.

> As I only heard complains about git-dpm, maybe someone would like to
> express his joy using it, and explain why they think it's a nice tool.
> But is there such person? It seems git-dpm only brings frustration.

Most of the time it works pretty well... It looked good compared with
the alternatives available at the time we made the decision.

However this is irrelevant IMHO if it isn't being mantained.
-- 
Brian May 

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT (was: PAPT Git)

[W. Martin Borgert]

On 2016-08-10 10:18, Thomas Goirand wrote:
> Instead of
> accepting the merge, and resolving conflicts later on, git-dpm goes into
> the rebase conflict mode of Git, and it's often not obvious what to do
> there. Messing-up everything, and restart from scratch (and then iterate
> until done properly) isn't uncommon.

Been there, lost hours :~(

> As I only heard complains about git-dpm, maybe someone would like to
> express his joy using it, and explain why they think it's a nice tool.
> But is there such person? It seems git-dpm only brings frustration.

Well, in most cases I did not have any problems with it.
Points I like and would prefer not to change:
 - no need to use quilt
 - no special build command, just plain dpkg-bp or whatever

The idea to try something else in PAPT is very welcome from my
side, no matter what tool.

Re: using git-dpm or plain git-buildpackage in PAPT and DPMT (was: PAPT Git)

[Thomas Goirand]

On 08/10/2016 09:21 AM, Sandro Tosi wrote:
> On Tue, Aug 9, 2016 at 11:50 PM, W. Martin Borgert  wrote:
>> On 2016-08-09 23:28, Daniel Stender wrote:
>>> On this occasion ... let it be me to start the discussion: let's get into 
>>> Git
>>> also for Python Apps soon.
>>
>> A common VCS for both DPMT and PAPT would be nice, indeed.
> 
> should we try and use plain git-buildpackage instead of git-dpm?

Piotr avoided the discussion during Debconf 16. IMO for good, as it
would have probably taken a full session.

I wrote a draft about my thought about it. Let me past it here.

The biggest defect of git-dpm, to me, is that it is often very painful
to merge a new upstream release in the packaging branch. Instead of
accepting the merge, and resolving conflicts later on, git-dpm goes into
the rebase conflict mode of Git, and it's often not obvious what to do
there. Messing-up everything, and restart from scratch (and then iterate
until done properly) isn't uncommon. Sometimes, all of this time is
spent just for a patch that needs to be removed removed.

With git-dpm, many operations aren't obvious, and it seems there's not
even an answer for stuff like this:
https://bugs.debian.org/810580
https://bugs.debian.org/801667
https://bugs.debian.org/801666

which stayed opened with no reply for months.

Sometimes, things are just very annoying:
https://bugs.debian.org/763569
https://bugs.debian.org/813302
https://bugs.debian.org/768382
https://bugs.debian.org/801649

It is also annoying that git-dpm doesn't produce a clean git history. It
ends up this way:

commit 241398abe0c476f39f0b4d62fb5227e4d0a52778
Merge: c72e5ea 880b748
Author: Thomas Goirand 
Date:   Tue Aug 2 08:40:37 2016 +

merge patched into master

commit 880b74889bbba237fe1bb17d992e1cbbb462be6b
Author: Thomas Goirand 
Date:   Tue Aug 2 08:39:49 2016 +

django-1.10-fix_runtests_lack_of_TEMPLATES

when really, this should be a single commit, not a merge operation. This
also doesn't include the debian/changelog comments, which also should be
in the same commit.

Even more: the generated patch headers aren't compliant with the format
1.0 as defined in the DEP. Meaning that we have to rewrite them. But
worse: git-dpm rewrites the headers loosing precious information like
the Origin field or bug numbers.

git-dpm also fails to tag upstream/foo automatically when importing a
new version. I've been told to use "git-dpm tag", but that's not
obvious. My own experience managing debian/patches quilt patches
manually or through gbp pq is actually much much nicer.

Plus it appears git-dpm is unmaintained and even sometimes buggy
(according to what I heard from others in Debconf: I didn't experienced
crashes myself). The non-mandatory use of "gbp pq" is nicer approach:
those who prefer manual quilt handling can do it.

As I only heard complains about git-dpm, maybe someone would like to
express his joy using it, and explain why they think it's a nice tool.
But is there such person? It seems git-dpm only brings frustration.

Cheers,

Thomas Goirand (zigo)