Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Tollef Fog Heen]

Last draft, incorporating the suggestions from Ian and Didier.

=== DRAFT Resolution ===

Vendor-specific patch series are a feature of dpkg that can be used to
apply a different series of quilt patches when the source package is
unpacked on different systems.  Since Debian source packages are usually
treated as a pure transport format (like tar), this property can cause
confusion and frustration for users.  Examples could be if only the
series file for one vendor is updated, or a source package is unpacked
on one system and then transferred to a system with a different vendor
for debugging.

The Committee recognises that there is a need for packages to behave
differently when built on different distributions, but this should be
done by using differing source packages, or as part of the build
process using current and future practices such as patches with
conditional behaviour or patching of files during the build rather than
at source unpacking time.

Since this feature is used by several packages today, we need a
reasonable transition period.  They will be considered buggy from when
this resolution is accepted, but it will not be considered severe enough
to warrant immediate removal from Debian.  After Buster is released, the
presence of a vendor-specific patch series will be a violation of a MUST
directive in Debian policy.

The Committee therefore resolves that:

1. Any use of dpkg's vendor-specific patch series feature is a bug for
   packages in the Debian archive (including contrib and non-free).

   This should be implemented in Debian Policy by declaring that a
   package SHOULD NOT contain a non-default series file.

2. After Buster is released, use of the vendor-specific patch series
   feature is forbidden in the Debian archive.

   This should be implemented in Debian Policy by declaring that a
   package MUST NOT contain a non-default series file.

=== End DRAFT Resolution ===

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Ian Jackson]

Tollef Fog Heen writes ("Bug#904302: Whether vendor-specific patch series 
should be permitted in the archive"):
> Second draft:
...
> The Committee recognises that there is a need for packages to behave
> differently when built on different distributions, but this should be
> done by using different source packages, or as part of the build
> process, using current and future practices such as patches with
> conditional behaviour, patching of files during the build, rather than
> at source unpacking time.

This is all good but can I suggest using the phrase `differing source
packages' rather than `different' ?  `Different' might mean source
packages with different source package name.  `Differing' more clearly
refers to source packages of the same name but which are different to
each other.

Thanks,
Ian.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Didier 'OdyX' Raboud]

Nice, thanks.

A minor nitpicks inline …

Le mardi, 23 octobre 2018, 21.28:40 h CEST Tollef Fog Heen a écrit :
> === DRAFT Resolution ===
> 
> Vendor-specific patch series are a feature of dpkg that can be used to
> apply a different series of quilt patches when the source package is
> unpacked on different systems.  Since Debian source packages are usually
> treated as a pure transport format (like tar), this property can cause
> confusion and frustration for users.  Examples could be if only the
> series file for one vendor is updated, or a source package is unpacked
> on one system and then transferred to a system with a different vendor
> for debugging.
> 
> The Committee recognises that there is a need for packages to behave
> differently when built on different distributions, but this should be
> done by using different source packages, or as part of the build
> process, using current and future practices such as patches with
> conditional behaviour, patching of files during the build, rather than
---^

use "or", as the list of examples has only two items?

> at source unpacking time.
> 
> Since this feature is used by several packages today, we need a
> reasonable transition period.  They will be considered buggy from when
> this resolution is accepted, but it will not be considered severe enough
> to warrant immediate removal from Debian.  After Buster is released, the
> presence of a vender-specific patch series will be a violation of a MUST
^

vendor

> directive in Debian policy.
> 
> The Committee therefore resolves that:
> 
> 1. Any use of dpkg's vendor-specific patch series feature is a bug for
>packages in the Debian archive (including contrib and non-free).
> 
>(…)
> 
> 2. After Buster is released, use of the vendor-specific patch series
>feature is forbidden in the Debian archive.
> 
>(…)

I'm not sure how these are to be interpreted with regards to "the whole Debian 
archive" vs "unstable". I'd adopt a more Policy-like language like:
> 2. After Buster is released, packages must not use vendor-specific patch
>series.

… but it resonates with the second paragraphs. I can live with the existing 
phrasing.

Cheers,
OdyX

signature.asc
Description: This is a digitally signed message part.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Tollef Fog Heen]

]] Tollef Fog Heen 

> ]] Tollef Fog Heen 
>
> That turned out to be in the rather more distant future than I
> intended.  Apologies about that.

… and again.

Diff from previous one:

- Included separate source packages as an alternative to build time
  patching.
- Fixed typo.

I have not included the language about release criticalness of bugs,
due to OdyX' comment about that being the domain of the release team,
not policy.

I hope to call for a vote on this by the end of the week.

Second draft:

=== DRAFT Resolution ===

Vendor-specific patch series are a feature of dpkg that can be used to
apply a different series of quilt patches when the source package is
unpacked on different systems.  Since Debian source packages are usually
treated as a pure transport format (like tar), this property can cause
confusion and frustration for users.  Examples could be if only the
series file for one vendor is updated, or a source package is unpacked
on one system and then transferred to a system with a different vendor
for debugging.

The Committee recognises that there is a need for packages to behave
differently when built on different distributions, but this should be
done by using different source packages, or as part of the build
process, using current and future practices such as patches with
conditional behaviour, patching of files during the build, rather than
at source unpacking time.

Since this feature is used by several packages today, we need a
reasonable transition period.  They will be considered buggy from when
this resolution is accepted, but it will not be considered severe enough
to warrant immediate removal from Debian.  After Buster is released, the
presence of a vender-specific patch series will be a violation of a MUST
directive in Debian policy.

The Committee therefore resolves that:

1. Any use of dpkg's vendor-specific patch series feature is a bug for
   packages in the Debian archive (including contrib and non-free).

   This should be implemented in Debian Policy by declaring that a
   package SHOULD NOT contain a non-default series file.

2. After Buster is released, use of the vendor-specific patch series
   feature is forbidden in the Debian archive.

   This should be implemented in Debian Policy by declaring that a
   package MUST NOT contain a non-default series file.

=== End DRAFT Resolution ===

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Adrian Bunk]

On Fri, Oct 05, 2018 at 08:49:47AM +0200, Philip Hands wrote:
> Adrian Bunk  writes:
> 
> > On Thu, Oct 04, 2018 at 08:21:07PM +0200, Philip Hands wrote:
> >>...
> >> IMO policy should recomend the use of separate source packages as the
> >> prefered solution to the problem that vendor-specific patch series were
> >> supposed to address.
> >
> > In this case please make an explicit decision on whether build-time 
> > patching is the recommended replacement for vendor-specific patch 
> > series, or what kinds of build-time patching will no longer be 
> > permitted.
> >
> > The current situation in the archive is:
> > - 18 packages with vendor-specific patch series
> > - an unknown number of packages (e.g. src:gcc-8) that are doing 
> >   vendor-specific build-time patching and/or patching based on
> >   other factors like architecture
> > - > 100 packages that are doing patching and/or configuration
> >   based on dpkg-vendor
> > - an unknown number of packages (e.g. src:gcc-8) that are doing patching 
> >   and/or configuration based on other tools like lsb_release
> >
> > It is not clear at all which of the above exactly you want to have 
> > removed from the archive and moved as permanent deltas downstream.
> 
> I think it's entirely up to the maintainers of the package (as long as
> they avoid vendor-specific patch series in future).
> 
> However if someone reads the prohibition against vendor-specific patch
> series, and is left wondering what is the best way to deal with this
> situation, then it would probably be helpful to give them a hint.
> 
> The methods you highlight all seem to suffer from the problem that if a
> downstream needs to make a vendor specific change, they need to do an
> odd dance where they may have to introduce a delta in order to get the
> vendor version out in a timely manner, then they need to get that into
> the debian source, and perhaps prompt a no-change release of the Debian
> package in order to be able to pick up the change and drop the delta.

This sounds like a very negative description of the standard
Open Source mantra "try to get all your changes upstream".

> It makes much more sense to me to have branches for the debian and
> downstream patches side-by-side in one's favourite source control system,
> and just build and release whatever one needs, whenever one needs it.
>...

Sometimes Debian and downstream maintainer are the same person.

Usually they are not, and then your suggestion won't work.

One common case is Ubuntu requiring different configuration of some 
packages, to avoid dependencies of security supported packages in
Ubuntu on packages that are not security supported in Ubuntu.

The fundamental reason for the existence of Raspbian is a vendor 
specific difference in all gcc packages. There are also other
packages in Debian that require different configuration for
the lower baseline.

The official source in the archive for a Debian package is the source 
tarball, which implies that any general solution used by downstreams
has to be based on the sources in these official tarballs.

> Cheers, Phil.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Didier 'OdyX' Raboud]

Le mercredi, 19 septembre 2018, 14.39:23 h CEST Philip Hands a écrit :
> How about this instead:
> 
>   The Committee therefore resolves that:
> 
>   1. Any use of dpkg's vendor-specific patch series feature is a bug for
>  packages in the Debian archive (including contrib and non-free),
>  however existing use of this feature in packages should not be
>  considered release critical until after the release of Buster.

I don't think this has been mentionned yet:

The TC needs to be careful when outlining what is "release-critical" or not: 
AFAIK this is part of what's delegated to the Release Team [0]. So unless the 
Release Team defers that specific question to us, I'd recommend to only talk 
in terms of Policy vocabulary. That doesn't imply we cannot align our 
recommended timelines with Debian suite release dates, of course.

That said, specifically regarding Philip's text, the "should not" is only 
emitting a $6.1.5-style opinion towards the RT, which is probably fine.

Cheers,

OdyX

[0] https://lists.debian.org/debian-devel-announce/2014/05/msg8.html

signature.asc
Description: This is a digitally signed message part.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive [and 1 more messages]

[Sam Hartman]

> "Wouter" == Wouter Verhelst  writes:


Wouter> But in the general case, I feel that downstream packaging
Wouter> changes belong downstream, not in Debian; therefore it is
Wouter> best to recommend that, in the general case, packages in
Wouter> Debian do not switch on dpkg-vendor.

Right.  Where as I believe it's not that clear cut and would rather
simply outline a bullet list of tradeoffs for maintainers to consider.

I'd be OK with a general case recommendation though; it's just not how
I'd do it if I were writing text.

And for the record, there are times when I've chosen to ship debian
directories upstream because it was the right thing in that case.
And I've dropped the upstream directory when circumstances changed.
There though, I agree with you that there is a dominant answer: don't
ship debian directories upstream.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive [and 1 more messages]

[Wouter Verhelst]

On Tue, Oct 09, 2018 at 10:14:44AM -0400, Sam Hartman wrote:
> > "Wouter" == Wouter Verhelst  writes:
> 
> Wouter> On Fri, Oct 05, 2018 at 08:40:03AM -0400, Sam Hartman wrote:
> >> That said, even there there are tradeoffs.  As an example, Ubuntu
> >> tries to use unmodified Debian source packages where possible.
> >> In some cases I think that the maintenance advantages of doing
> >> this and the slight but real political pressure it creates to
> >> push changes upstream to Debian may justify switching on
> >> dpkg-vendor.
> 
> Wouter> I disagree with that, because it forgets why you're pushing
> Wouter> things to Debian.
> 
> Wouter> The point of pushing things upstream is so that you as well
> Wouter> as upstream end up being the same, and the maintenance
> Wouter> difference disappears. By switching on dpkg-vendor, you're
> Wouter> *not* the same; instead, you're hiding your difference. This
> Wouter> is not generally helpful; it simply moves the maintenance
> Wouter> burden from Ubuntu to Debian (where it simply does not
> Wouter> belong).
> 
> I think that we're agreed that evaluating the maintenance burden is
> exactly the right criteria.

Yes.

> Imagine a case where the same folks are maintaining a package for
> multiple distributions and where the difference is small but important.
> In such a case I think our users and the free software community might
> best be served by a single repository and switching on something a lot
> like dpkg-vendor.

Sure. For one thing though, it's incorrect to assume that the same group
of people will maintain a given package for all eternity, and that the
time when they *do* keep maintaining that package in Debian is the same
as for any number of derivative distributions.

This is all explained in
.
While that wiki page talks about the Debian-upstream relationship, many
of the same (kind of) arguments apply equally well to a
downstream-Debian relationship.

> Imagine a case where  it's a different set of people doing the work for
> Debian than the distribution that wants the change.  The Debian
> maintainers are not  in a good position to test the change and have no
> desire to do so.  There, switching on vendor seems like the wrong
> option.
> 
> We're a group of volunteers; we encourage cross-project collaboration
> and working together.  I  believe that the primary consideration should
> be what reduces the burden on those doing the work.  There are secondary
> considerations of course.

Sure.

Note, I'm not saying that we should outlaw the practice. There are
considerations for doing (or not doing) certain things; an experienced
Debian maintainer can certainly decide that in one particular case,
those considerations do not apply, and then it should be fine to ignore
the advice and do what is right.

But in the general case, I feel that downstream packaging changes belong
downstream, not in Debian; therefore it is best to recommend that, in
the general case, packages in Debian do not switch on dpkg-vendor.

-- 
Could you people please use IRC like normal people?!?

  -- Amaya Rodrigo Sastre, trying to quiet down the buzz in the DebConf 2008
 Hacklab

Bug#904302: Whether vendor-specific patch series should be permitted in the archive [and 1 more messages]

[Sam Hartman]

> "Wouter" == Wouter Verhelst  writes:

Wouter> On Fri, Oct 05, 2018 at 08:40:03AM -0400, Sam Hartman wrote:
>> That said, even there there are tradeoffs.  As an example, Ubuntu
>> tries to use unmodified Debian source packages where possible.
>> In some cases I think that the maintenance advantages of doing
>> this and the slight but real political pressure it creates to
>> push changes upstream to Debian may justify switching on
>> dpkg-vendor.

Wouter> I disagree with that, because it forgets why you're pushing
Wouter> things to Debian.

Wouter> The point of pushing things upstream is so that you as well
Wouter> as upstream end up being the same, and the maintenance
Wouter> difference disappears. By switching on dpkg-vendor, you're
Wouter> *not* the same; instead, you're hiding your difference. This
Wouter> is not generally helpful; it simply moves the maintenance
Wouter> burden from Ubuntu to Debian (where it simply does not
Wouter> belong).

I think that we're agreed that evaluating the maintenance burden is
exactly the right criteria.


Imagine a case where the same folks are maintaining a package for
multiple distributions and where the difference is small but important.
In such a case I think our users and the free software community might
best be served by a single repository and switching on something a lot
like dpkg-vendor.

Imagine a case where  it's a different set of people doing the work for
Debian than the distribution that wants the change.  The Debian
maintainers are not  in a good position to test the change and have no
desire to do so.  There, switching on vendor seems like the wrong
option.

We're a group of volunteers; we encourage cross-project collaboration
and working together.  I  believe that the primary consideration should
be what reduces the burden on those doing the work.  There are secondary
considerations of course.

--Sam

Bug#904302: Whether vendor-specific patch series should be permitted in the archive [and 1 more messages]

[Ian Jackson]

Sam Hartman writes ("Bug#904302: Whether vendor-specific patch series should be 
permitted in the archive [and 1 more messages]"):
> So imagine that Ubuntu and several other downstreams care more about
> security and hardening than they do about backward compatibility and
> they choose to change a number of gcc and other tool defaults in support
> of that.  I realize my example is not entirely hypothetical, but I want
> to emphasize I have not researched to get all the details right, because
> it doesn't matter.
> 
> Especially if multiple downstreams or individual users who build from
> source might want the change, I think carrying the delta in  the Debian
> source package can be valuable.  It needs to be balanced against a lot
> of other concerns.

I agree that carrying a switch-on-able delta in the Debian package
would be a good thing there.  So, the meat of the change should
definitely go to Debian or maybe even to upstream as a
--with-extra-hardening configure option or some such.

This should be enabled via DEB_BUILD_OPTIONS, or a commented-out line
in the rules file, or by reading /etc/compilers/hardening-enabled at
build- or runtime, or something.  Not by looking at `the vendor'.

A scenario why this is needed can be seen from this scenario:

Suppose someone wants to try to maintain a distro which is like
Debian, but which takes some subset of packages from Raspbian.

The obvious way to do this is to import most packages from Debian and
the other packages from Raspbian, and to fix up the bugs which occur
at the interface.

If packages in Debian and Raspbian use dpkg-vendor --is raspbian to
decide whether to do the Raspbian thing, then there is no way to make
this work because there is no correct answer: the answer to whether a
package should do the Raspbian thing depends not only on which distro
we are building or running on, but on which package it is.

In practice if I were maintaining that distro I would be tempted do
some kind of hideous hack to make the output of dpkg-vendor depend on
the name of package we were building.  Because how else will I do it ?
Playing whack-a-mole with dpkg-vendor calls would be impractical.

If the Raspbian-specific features are enabled by carrying a changed
source package in Raspbian, then things will mostly DTRT and generally
problems will occur only when the delta-enablement is done via some
kind of indirection, and the indirection messily crosses the `cutoff'
between the Raspbian-originated and Debian-originated packages.

Ie I only have to manually fix the problems that irreducible, not ones
introduced by ill-advised calls to dpkg-vendor.

> And yes, in many cases dgit is the answer.  That said, if I were
> maintaining the same package both for Debian and for the downstream I
> work on, I might well value having a single source tree enough to use
> dpkg-vendor or lsb-release or something to switch.

In that case, that is convenient for you but it is wrong for your
downstreams and users.  We should be discouraging such tradeoffs.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive [and 1 more messages]

[Sam Hartman]

Actually directly switching on vendor seems fairly bad.

However, to the extent that downstream changes can be encapsulated into
options/deltas that someone might want, I think it may often be
reasonable to carry the delta in Debian.

So imagine that Ubuntu and several other downstreams care more about
security and hardening than they do about backward compatibility and
they choose to change a number of gcc and other tool defaults in support
of that.  I realize my example is not entirely hypothetical, but I want
to emphasize I have not researched to get all the details right, because
it doesn't matter.

Especially if multiple downstreams or individual users who build from
source might want the change, I think carrying the delta in  the Debian
source package can be valuable.  It needs to be balanced against a lot
of other concerns.

However, I do tend to agree with Ian that actually turning on that delta
in a specific vendor environment may be best carrief as a
vendor-specific source package.

That said, even there there are tradeoffs.
As an example, Ubuntu tries to use unmodified Debian source packages
where possible.  In some cases I think that the maintenance advantages
of doing this and the slight but real political pressure it creates to
push changes upstream to Debian may justify switching on dpkg-vendor.

I think my point here is that there's a lot of complexity, and I'm not
even convinced it would be desirable to recommend against using
mechanisms like dpkg-vendor.
I think what we can hopefully all agree on is that the dpkg vendor patch
series as bad.
Perhaps before we go and recommend something specific we can actually
write up some of these tradeoffs and give people the information they
need to make informed decisions.

And yes, in many cases dgit is the answer.  That said, if I were
maintaining the same package both for Debian and for the downstream I
work on, I might well value having a single source tree enough to use
dpkg-vendor or lsb-release or something to switch.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive [and 1 more messages]

[Ian Jackson]

Adrian Bunk writes ("Bug#904302: Whether vendor-specific patch series should be 
permitted in the archive"):
> On Thu, Oct 04, 2018 at 08:21:07PM +0200, Philip Hands wrote:
> > IMO policy should recomend the use of separate source packages as the
> > prefered solution to the problem that vendor-specific patch series were
> > supposed to address.
> 
> In this case please make an explicit decision on whether build-time 
> patching is the recommended replacement for vendor-specific patch 
> series, or what kinds of build-time patching will no longer be 
> permitted.

Surely the TC can recommend X without outlawing Y.

> It is not clear at all which of the above exactly you want to have 
> removed from the archive and moved as permanent deltas downstream.

Speaking personally, I think that packages should almost never look at
dpkg-vendor (or equivalent, eg looking at lsb_release).  Any switching
on vendor is probably a bug.  The reasons why switching on vendor is
probably wrong have been rehearsed earlier in this discussion.

But I am not asking the TC to outlaw the use of dpkg-vendor et al
because:

 * The dpkg vendor series feature is uniquely weird and bad.  (Both in
   principle, and because of what are arguably lesser design bugs.)

 * Unlike the dpkg vendor series feature, the use of dpkg-vendor (or
   equivalent) at build- or run-time is not directly in the way of my
   project to improve our source code management processes.

 * I think an effort to outlaw switching based on vendor would
   generate a much bigger fight which is not worth having.

 * I hope that my programme of making it easier to handle divergence
   properly, by having actually different source code, and carrying
   that delta indefinitely, will tempt people away from these kind of
   vendor-switching approaches.

Another way to look at this is that in general, where possible, I
think transitions to new things should be done by making the new thing
attractive rather than by going around forbidding or breaking the old
thing.

Sadly the vendor series feature is too broken, and causes too many
problems for others, for that.  The problems of switching on vendor
are real but much less severe so taking the carrot rather than stick
approach is much more sensible.

But of course a body like the TC should certainly recommend good
practices rather than troublesome ones.

> The TC was asked to make a decision, and a decision turning a clear 
> situation into a blurry "it is permitted but kinda recommended against" 
> would only create future conflicts.

We have a lot of things in Debian where one approach is recommended,
but other approaches are permitted or tolerated.  This does not in
general seem to result in conflicts.

> A 1:1 vendor-specific patch series -> build-time patching change
> would be a mostly technical change. As already said this could
> even be implemented before buster.

Because I think dpkg-vendor is the wrong answer, I don't want the TC
to recommend that.

Philip Hands writes ("Bug#904302: Whether vendor-specific patch series should 
be permitted in the archive"):
> However if someone reads the prohibition against vendor-specific patch
> series, and is left wondering what is the best way to deal with this
> situation, then it would probably be helpful to give them a hint.

Quite.

Ian.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Philip Hands]

Adrian Bunk  writes:

> On Thu, Oct 04, 2018 at 08:21:07PM +0200, Philip Hands wrote:
>>...
>> IMO policy should recomend the use of separate source packages as the
>> prefered solution to the problem that vendor-specific patch series were
>> supposed to address.
>
> In this case please make an explicit decision on whether build-time 
> patching is the recommended replacement for vendor-specific patch 
> series, or what kinds of build-time patching will no longer be 
> permitted.
>
> The current situation in the archive is:
> - 18 packages with vendor-specific patch series
> - an unknown number of packages (e.g. src:gcc-8) that are doing 
>   vendor-specific build-time patching and/or patching based on
>   other factors like architecture
> - > 100 packages that are doing patching and/or configuration
>   based on dpkg-vendor
> - an unknown number of packages (e.g. src:gcc-8) that are doing patching 
>   and/or configuration based on other tools like lsb_release
>
> It is not clear at all which of the above exactly you want to have 
> removed from the archive and moved as permanent deltas downstream.

I think it's entirely up to the maintainers of the package (as long as
they avoid vendor-specific patch series in future).

However if someone reads the prohibition against vendor-specific patch
series, and is left wondering what is the best way to deal with this
situation, then it would probably be helpful to give them a hint.

The methods you highlight all seem to suffer from the problem that if a
downstream needs to make a vendor specific change, they need to do an
odd dance where they may have to introduce a delta in order to get the
vendor version out in a timely manner, then they need to get that into
the debian source, and perhaps prompt a no-change release of the Debian
package in order to be able to pick up the change and drop the delta.

It makes much more sense to me to have branches for the debian and
downstream patches side-by-side in one's favourite source control system,
and just build and release whatever one needs, whenever one needs it.

BTW Do we have any way of determining how many packages already deal
with vendor-specific changes in this way?

I'll admit that I've not had to deal with such packages, so feel free to
explain to me why my perception of the situation is utterly deranged.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Adrian Bunk]

On Thu, Oct 04, 2018 at 08:21:07PM +0200, Philip Hands wrote:
>...
> IMO policy should recomend the use of separate source packages as the
> prefered solution to the problem that vendor-specific patch series were
> supposed to address.

In this case please make an explicit decision on whether build-time 
patching is the recommended replacement for vendor-specific patch 
series, or what kinds of build-time patching will no longer be 
permitted.

The current situation in the archive is:
- 18 packages with vendor-specific patch series
- an unknown number of packages (e.g. src:gcc-8) that are doing 
  vendor-specific build-time patching and/or patching based on
  other factors like architecture
- > 100 packages that are doing patching and/or configuration
  based on dpkg-vendor
- an unknown number of packages (e.g. src:gcc-8) that are doing patching 
  and/or configuration based on other tools like lsb_release

It is not clear at all which of the above exactly you want to have 
removed from the archive and moved as permanent deltas downstream.

The status quo is that everything is permitted,
which is a pretty clear situation.

The TC was asked to make a decision, and a decision turning a clear 
situation into a blurry "it is permitted but kinda recommended against" 
would only create future conflicts.

A 1:1 vendor-specific patch series -> build-time patching change
would be a mostly technical change. As already said this could
even be implemented before buster.

If the TC wants to additionally change the status quo on the high-level 
question whether Debian wants permanent downstream deltas maintained in 
Debian or downstream, it should make an explicit decision on that.

> Cheers, Phil.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Ian Jackson]

Philip Hands writes ("Re: Bug#904302: Whether vendor-specific patch series 
should be permitted in the archive"):
> IMO policy should recomend the use of separate source packages as the
> prefered solution to the problem that vendor-specific patch series were
> supposed to address.

That would be great.  (I suspect that it is rather controversial,
though, even though I think it is obviously correct...)

If there is consensus in the TC that this is the best approach,
putting something about it in the TC resolution as a recommendation
would have about as much weight as putting it in policy, and avoid
jurisdictional questions.

Ian.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Philip Hands]

Ian Jackson  writes:

> Adrian Bunk writes ("Bug#904302: Whether vendor-specific patch series should 
> be permitted in the archive"):
>> On Wed, Sep 19, 2018 at 02:39:23PM +0200, Philip Hands wrote:
>> >   The Committee therefore resolves that:
>> > 
>> >   1. Any use of dpkg's vendor-specific patch series feature is a bug for
>> >  packages in the Debian archive (including contrib and non-free),
>> 
>> This misses an important part of the previous proposal:
>
> I think Phil was just intending to leave the recitals part alone, and
> proposing only a change to the operative part - not to delete the
> recitals.

Correct -- sorry if that wasn't clear.

>>   The Committee recognises that there is a need for packages to behave
>>   differently when built on different distributions, but this should be
>>   done as part of the build process, using current and future practices
>>   such as patches with conditional behaviour, patching of files during the
>>   build, rather than at source unpacking time.
>
> However, now that we are talking about the recitals I would like to
> suggest that the recitals should include *maintaining different source
> packages in different distributions* as one of the suggested options.
>
> IMO it is far superior to patches which are conditional (at runtime or
> at build-time) on dpkg-vendor and I would not like to see that
> perpetuated.

As you say, a separate source package seems like the right way to deal
with this situation.

IMO policy should recomend the use of separate source packages as the
prefered solution to the problem that vendor-specific patch series were
supposed to address.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[David Bremner]

Adrian Bunk  writes:


> My understanding of the TC proposal so far is that this would
> recommend a 1:1 conversion from vendor-specific patch series
> to build-time patching. And as I said, you could even get rid
> of the "after buster" part if someone has conversions for all
> of the 18 affected packages ready.

My hope is that the final wording will recommend (demand?) the
conversion from vendor-specific patch series to _something else_,
without taking a position on what that something else should be. The
wording proposed in the last TC meeting (but not yet sent to the bug I
think) did mention build-time patching as an option, but weakened the
initial wording's apparent endorsement of build-time patching as the
recommended method. Before discussing too much further, it might be
helpful to have the revised wording on the bug. If I remember correctly
Tollef was going to post those.

d

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Adrian Bunk]

On Thu, Oct 04, 2018 at 01:32:08PM +0100, Ian Jackson wrote:
> Adrian Bunk writes ("Bug#904302: Whether vendor-specific patch series should 
> be permitted in the archive"):
> > On Wed, Sep 19, 2018 at 02:39:23PM +0200, Philip Hands wrote:
> > >   The Committee therefore resolves that:
> > > 
> > >   1. Any use of dpkg's vendor-specific patch series feature is a bug for
> > >  packages in the Debian archive (including contrib and non-free),
> > 
> > This misses an important part of the previous proposal:
> 
> I think Phil was just intending to leave the recitals part alone, and
> proposing only a change to the operative part - not to delete the
> recitals.
> 
> >   The Committee recognises that there is a need for packages to behave
> >   differently when built on different distributions, but this should be
> >   done as part of the build process, using current and future practices
> >   such as patches with conditional behaviour, patching of files during the
> >   build, rather than at source unpacking time.
> 
> However, now that we are talking about the recitals I would like to
> suggest that the recitals should include *maintaining different source
> packages in different distributions* as one of the suggested options.

This is not really applicable here, we are only talking about cases 
where Debian maintainer and downstream distribution have in the past
explicitely moved a downstream delta into the package in Debian.

> IMO it is far superior to patches which are conditional (at runtime or
> at build-time) on dpkg-vendor and I would not like to see that
> perpetuated.

My understanding of the TC proposal so far is that this would
recommend a 1:1 conversion from vendor-specific patch series
to build-time patching. And as I said, you could even get rid
of the "after buster" part if someone has conversions for all
of the 18 affected packages ready.

My understanding of the TC proposal so far is that this would
recommend a 1:1 conversion from vendor-specific patch series 
to build-time patching. And as I said, you could even get rid
of the "after buster" part if someone has conversions for all
of the 18 affected packages ready.

If you want to have the status quo changed, you should IMHO ask
the TC for an explicit decision on that.

The disussion so far has not been about that topic, and adding
a last-minute insertion questioning the status quo into the
decision would not be warranted.

An ambiguous TC decision or policy wording recommending to push such 
changes back downstream would also risk conflicts inside Debian if 
different people would start arguing with that for whatever side
they personally prefer.

> Ian.

cu
Adrian

[1] assuming there is a good reason for doing so

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Ian Jackson]

Adrian Bunk writes ("Bug#904302: Whether vendor-specific patch series should be 
permitted in the archive"):
> On Wed, Sep 19, 2018 at 02:39:23PM +0200, Philip Hands wrote:
> >   The Committee therefore resolves that:
> > 
> >   1. Any use of dpkg's vendor-specific patch series feature is a bug for
> >  packages in the Debian archive (including contrib and non-free),
> 
> This misses an important part of the previous proposal:

I think Phil was just intending to leave the recitals part alone, and
proposing only a change to the operative part - not to delete the
recitals.

>   The Committee recognises that there is a need for packages to behave
>   differently when built on different distributions, but this should be
>   done as part of the build process, using current and future practices
>   such as patches with conditional behaviour, patching of files during the
>   build, rather than at source unpacking time.

However, now that we are talking about the recitals I would like to
suggest that the recitals should include *maintaining different source
packages in different distributions* as one of the suggested options.

IMO it is far superior to patches which are conditional (at runtime or
at build-time) on dpkg-vendor and I would not like to see that
perpetuated.

Ian.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Adrian Bunk]

On Wed, Sep 19, 2018 at 02:39:23PM +0200, Philip Hands wrote:
>...
>   The Committee therefore resolves that:
> 
>   1. Any use of dpkg's vendor-specific patch series feature is a bug for
>  packages in the Debian archive (including contrib and non-free),

This misses an important part of the previous proposal:

  The Committee recognises that there is a need for packages to behave
  differently when built on different distributions, but this should be
  done as part of the build process, using current and future practices
  such as patches with conditional behaviour, patching of files during the
  build, rather than at source unpacking time.

Otherwise it might sound as if any kind of vendor-specific patching 
would be no longer be allowed in the Debian archive.

>  however existing use of this feature in packages should not be
>  considered release critical until after the release of Buster.
>...

We might disagree whether or not the whole change is a good idea, but if
it is done I do not see any good reason for waiting until after the
release of buster.

Policy should in any case provide guidance what replacement for 
vendor-specific patching during the build is recommended instead.

With only 18 packages, submitting 18 RC bugs with patches switching them 
to the replacement should then be doable immediately.

> Cheers, Phil.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Ian Jackson]

Sean Whitton writes ("Bug#904302: Whether vendor-specific patch series should 
be permitted in the archive"):
> I would be grateful if you would micromanage just enough that there
> isn't anything controversial left for people to disagree about :)

That seems like a generally good rule for TC decisions.  It avoids
another aspect of the same issue coming back to the TC; and it avoids
setting in stone things that haven't been thought through in the TC
context (and which are best answered by whatever the usual process
is),

In this case I don't think the details (of the transition away from
use of this feature, or the policy wording) are controversial; except
that the timetable, and the consequences at various times for a
package that still has a vendor series, might well be.

No-one seems to have proposed anything but "bug, RC after buster", but
of course opponents of the change are focusing on that and if the TC
just says "these are a bad thing" then an opponent of the change might
well reasonably say "OK I agree this is a bug but it is not RC" or "I
intend to fix this in buster+2".

So, borrowing Phil's text and editing slightly:

  1. Presence of any dpkg vendor-specific patch series is a bug for
 packages in the Debian archive (including contrib and non-free).
 Such a bug should be considered release critical, but not until
 after the release of Buster.

The consequences for what to write in policy, want to do in lintian,
how the release team should handle these bugs, etc., all follow
clearly from that text, except for implementaion details that can be
thrashed out in the relevant fora.

I changed "use of [the] feature" to "presence of [the] series" to
avoid the possibility that someone would disingenuously argue that a
series.ubuntu file, in a package in Debian, is not "use" of the
feature.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Sean Whitton]

Hello,

On Tue 02 Oct 2018 at 07:48PM +0200, Tollef Fog Heen wrote:

> Given the entire decision has been delegated to us, I think we should,
> yes.  If we'd just been asked to decide on a matter of technical policy,
> that would have been slightly different.

I think that the wording of my initial mail was ambiguous with regard to
which one of these was being requested.

I would be grateful if you would micromanage just enough that there
isn't anything controversial left for people to disagree about :)

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Tollef Fog Heen]

]] Philip Hands 

> On reflection this seems like we're straying into micro-management.
> Should we really be determining the detail of how this is done in
> policy?

Given the entire decision has been delegated to us, I think we should,
yes.  If we'd just been asked to decide on a matter of technical policy,
that would have been slightly different.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Ian Jackson]

Philip Hands writes ("Bug#904302: Whether vendor-specific patch series should 
be permitted in the archive"):
> Possibly also with something like this?:
> 
>  Post-Buster this should be implemented in Debian Policy by
>  declaring that a package MUST NOT contain a non-default series
>  file.
> 
>  The approach adopted to allow existing usage is left to the
>  discretion of the Policy Maintainers.

Nit-picking, you might want to say

   The approach adopted to tolerating existing usage before then
   is left to the discretion of the Policy Maintainers.

Ian.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Philip Hands]

Tollef Fog Heen  writes:

> ]] Philip Hands 
>
>> Tollef Fog Heen  writes:
>> 
>> >This should be implemented in Debian Policy by declaring that a a
>>^^^
>> You've this doubled 'a' on two occasions in this text.
>
> I'll fix that, thanks for spotting it.
>
>> Presumaly we would not want to see new packages adopting the use of
>> vendor-specific patch series prior to Buster.
>> 
>> Do we need to make the "SHOULD NOT" conditional on the package already
>> having a vendor-specific patch series at the time of this resolution?
>
> I think that just adds needless complexity and assumes that
> maintainers will want to add bugs to their package.  I really hope
> that's not the case, so I don't think it's worthwhile to add extra
> language for it.

Well, I was thinking that we could simply state the MUST NOT as the
general case straight away, but with a limited exception for packages
that already contain the bug now, where that exception expires with the
release of Buster.

Your approach seems to require a timely update to policy post-buster,
whereas if there's a conditional in policy there would be no urgency to
removing it, so it can be done at the convenience of the policy
maintainers.

On reflection this seems like we're straying into micro-management.
Should we really be determining the detail of how this is done in
policy?

How about this instead:

  The Committee therefore resolves that:

  1. Any use of dpkg's vendor-specific patch series feature is a bug for
 packages in the Debian archive (including contrib and non-free),
 however existing use of this feature in packages should not be
 considered release critical until after the release of Buster.

Possibly also with something like this?:

 Post-Buster this should be implemented in Debian Policy by
 declaring that a package MUST NOT contain a non-default series
 file.

 The approach adopted to allow existing usage is left to the
 discretion of the Policy Maintainers.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Tollef Fog Heen]

]] Philip Hands 

> Tollef Fog Heen  writes:
> 
> >This should be implemented in Debian Policy by declaring that a a
>^^^
> You've this doubled 'a' on two occasions in this text.

I'll fix that, thanks for spotting it.

> Presumaly we would not want to see new packages adopting the use of
> vendor-specific patch series prior to Buster.
> 
> Do we need to make the "SHOULD NOT" conditional on the package already
> having a vendor-specific patch series at the time of this resolution?

I think that just adds needless complexity and assumes that maintainers
will want to add bugs to their package.  I really hope that's not the
case, so I don't think it's worthwhile to add extra language for it.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Philip Hands]

Tollef Fog Heen  writes:

>This should be implemented in Debian Policy by declaring that a a
   ^^^
You've this doubled 'a' on two occasions in this text.

Presumaly we would not want to see new packages adopting the use of
vendor-specific patch series prior to Buster.

Do we need to make the "SHOULD NOT" conditional on the package already
having a vendor-specific patch series at the time of this resolution?

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Sean Whitton]

Hello,

On Sat 15 Sep 2018 at 07:06PM +0200, Tollef Fog Heen wrote:

> A first draft is below, feedback on wording and content appreciated.

LGTM, thank you.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Wouter Verhelst]

On Sat, Sep 15, 2018 at 04:14:57PM -0300, David Bremner wrote:
> Tollef Fog Heen  writes:
> 
> >
> > The Committee recognises that there is a need for packages to behave
> > differently when built on different distributions, but this should be
> > done as part of the build process, using current and future practices
> > such as patches with conditional behaviour, patching of files during the
> > build, rather than at source unpacking time.
> 
> Does it need to be said that producing different source packages for
> different distributions is also perfectly acceptable?

I think that "different distributions" is by definition outside the authority
of the TC, and therefore it shouldn't even be mentioned in the resolution ;-)

-- 
Could you people please use IRC like normal people?!?

  -- Amaya Rodrigo Sastre, trying to quiet down the buzz in the DebConf 2008
 Hacklab

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[David Bremner]

Tollef Fog Heen  writes:

>
> The Committee recognises that there is a need for packages to behave
> differently when built on different distributions, but this should be
> done as part of the build process, using current and future practices
> such as patches with conditional behaviour, patching of files during the
> build, rather than at source unpacking time.

Does it need to be said that producing different source packages for
different distributions is also perfectly acceptable? A narrow reading
of this paragraph might give the impression that it is not. One option
would be say something like

  ... on different distributions, but this difference should not be
  done at source unpacking time but e.g. as part of the build
  process...

Arguably this concern is moot because everyone (TM) understands that a
package being built refers to going from a source package to one or more
binary packages.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Tollef Fog Heen]

]] Tollef Fog Heen 

> ]] Sean Whitton 
> 
> > The concrete question that I am asking the committee to decide, in my
> > capacity as a Policy delegate, is whether or not vendor-specific patch
> > series should be permitted in the Debian archive.
> 
> It's now been five days since I mailed the various package
> maintainers. I intend to write up a resolution and then call for a vote
> in the not-too-distant future, so if there is anything we have not
> covered in the discussion so far, please chime in sooner rather than
> later

That turned out to be in the rather more distant future than I
intended.  Apologies about that.

A first draft is below, feedback on wording and content appreciated.

=== DRAFT Resolution ===

Vendor-specific patch series are a feature of dpkg that can be used to
apply a different series of quilt patches when the source package is
unpacked on different systems.  Since Debian source packages are usually
treated as a pure transport format (like tar), this property can cause
confusion and frustration for users.  Examples could be if only the
series file for one vendor is updated, or a source package is unpacked
on one system and then transferred to a system with a different vendor
for debugging.

The Committee recognises that there is a need for packages to behave
differently when built on different distributions, but this should be
done as part of the build process, using current and future practices
such as patches with conditional behaviour, patching of files during the
build, rather than at source unpacking time.

Since this feature is used by several packages today, we need a
reasonable transition period.  They will be considered buggy from when
this resolution is accepted, but it will not be considered severe enough
to warrant immediate removal from Debian.  After Buster is released, use
of a vender-specific patch series will be a violation of a MUST
directive in Debian policy.

The Committee therefore resolves that:

1. Any use of dpkg's vendor-specific patch series feature is a bug for
   packages in the Debian archive (including contrib and non-free).

   This should be implemented in Debian Policy by declaring that a a
   package SHOULD NOT contain a non-default series file.

2. After Buster is released, use of the vendor-specific patch series
   feature is forbidden in the Debian archive.

   This should be implemented in Debian Policy by declaring that a a
   package MUST NOT contain a non-default series file.

=== End DRAFT Resolution ===

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Sean Whitton]

Hello Marga,

On Wed 15 Aug 2018 at 11:35AM +0200, Margarita Manterola wrote:

> Apart from this, the concern that has been raised about making packages
> instabuggy is valid. I would like our decision to include that this
> should
> be SHOULD first, giving maintainers a window of time to fix their
> packages
> and only become a MUST once those packages have been fixed (or something
> along those lines, maybe after buster is released).

I'd like to suggest giving a window of time.  Otherwise the policy
process alone won't really have the authority to switch from
should->must.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Margarita Manterola]

On 2018-08-09 21:22, Tollef Fog Heen wrote:

It's now been five days since I mailed the various package
maintainers. I intend to write up a resolution and then call for a vote
in the not-too-distant future, so if there is anything we have not
covered in the discussion so far, please chime in sooner rather than
later


I don't have much to add, but I want to state my opinion.

I agree with what has been said that the effect of the vendor series is
surprising and causes confusion when people first encounter this 
feature.

The reason for this (as has been mentioned) is that it's unexpected that
something would unpack differently in different environments while it's
expected that things will build differently in different environments.

It would be nice if we could carry out a more generic decision based on
this: that source packages should unpack the same regardless of the
derivative on which they are unpacked.  However, I don't think we have
the authority for such a ruling, as we can only rule about Debian, not
about derivatives. However, I believe we can still recommend it without
authority.

Apart from this, the concern that has been raised about making packages
instabuggy is valid. I would like our decision to include that this 
should
be SHOULD first, giving maintainers a window of time to fix their 
packages

and only become a MUST once those packages have been fixed (or something
along those lines, maybe after buster is released).

--
Regards,
Marga

Re: Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Ian Jackson]

(switching from the bug to debian-ctte & secretary@)

Sean Whitton writes ("Bug#904302: Whether vendor-specific patch series should 
be permitted in the archive"):
> On Sun 29 Jul 2018 at 07:52AM +0200, Tollef Fog Heen wrote:
> > I believe his request might also be considered under §6.1.1, since we're
> > being asked about a policy change.  (After talking to Sean in person, he
> > said he intended it under §6.1.3, not §6.1.1, though.)
> 
> I think technically it's §6.1.3 because according to the policy team
> delegation, we decide what goes into the policy manual.

Insofar as the policy delegation claims to delegate to the policy
editors the final decisions on what goes into policy (rather than
merely the routine task of editing the document) [1], it is ultra
vires.  The DPL cannot delegate a power they do not have.

Or to put it another way: even if the policy editors did not refer a
question to the TC, the TC could effectively overrule the policy
editors by using its power in 6.1.1.  Obviously this certainly ought
only to be considered after attempts to solve the problem another way
have been fully explored (6.3.6).

> But it certainly falls under at least one of §6.1.1 and §6.1.3, and not
> §6.1.4.

Obviously I agree with this.

Thanks,
Ian.

[1] I don't read the delegation that way, but for this purpose the
wording of the delegation doesn't matter.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Ian Jackson]

Colin Watson writes ("Bug#904302: Whether vendor-specific patch series should 
be permitted in the archive"):
> ...  My opinion from experience with this feature is that those
> derivative maintainers would have an easier time if they used
> patches with conditional behaviour (or maintained a local branch, of
> course, although that was part of what source package management
> features like this were trying to reduce).

Obviously I agree with the conclusion, but I would like to propose a
different way of looking at this:

The Debian archive and derivative archives, and the .dsc source
package formats, including "3.0 (quilt)" currently including the
vendor series feature, *are a version control system*. [1]

The vendor series feature *is a way to represent a branch*.
We should consider its merits in those terms, by comparing it to other
technology we have for handling branches.

In the current system [1], the obvious competing way to represent a
branch is for the derivative to actually change the source package.
If vendor series are forbidden, the same effect can always be achieved
that way.

Compared to changing the source package, the vendor series appears to
offer a reduction of maintainer friction.  This is a relatively minor
benefit.  (Note that resolution of any merge conflicts during rebase,
or whatever, is still necessary; the only benefit is a small degree of
automation and a slight reduction in different files etc. on
maintainer systems.)  However, that convenience comes at too high a
price.  In particular, the invisibility of the vendor series - that
very lack of friction - means that the vendor series is often wrong.

Conversely, derivatives must already have means for merging or
rebasing their own delta-represented-as-changes-to-the-source-package,
so as to track Debian.  (Vendor series cannot, for example, represent
changes to files in debian/.)  Those mechanisms are often fairly well
developed, and if they are insufficient they need to be improved [2].
And people can inspect the differences by comparing source packages.

So I think it is nearly always better, even without considering global
effects, to represent any needed derivative delta as a changes source
package, than to represent it as a vendor series.[3]

But forbidding vendor series has the global benefit that no-one who
deals with source packages from Debian derivatives has to write code
to deal with, and spend mental energy on, yet another way to represent
the delta.

Thanks,
Ian.

[1] By modern standards, an appallingly bad one.  I would argue that
it all made sense in 1993 (when the best alternative was CVS) but
nowadays we have much much better tools.  This is why I am engaged in
a project to provide something much better than .dsc, based on git.

[2] Tracking Debian is IMO still too hard.  I am working on that.

[3] That does not mean I think that changing the source package is
always or even mostly the best answer; often other approaches will be
better still.  It just means that vendor series are worse than
changing the source package.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Tollef Fog Heen]

]] Sean Whitton 

> The concrete question that I am asking the committee to decide, in my
> capacity as a Policy delegate, is whether or not vendor-specific patch
> series should be permitted in the Debian archive.

It's now been five days since I mailed the various package
maintainers. I intend to write up a resolution and then call for a vote
in the not-too-distant future, so if there is anything we have not
covered in the discussion so far, please chime in sooner rather than
later

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Paul Gevers]

Hi,

[As the main liferea uploader and the one that implemented this there.]

On Thu, 2 Aug 2018 09:46:28 +0100 Simon McVittie  wrote:
> liferea, a RSS feed reader, uses Ubuntu RSS feeds by default when unpacked
> on Ubuntu. This is a patch to XML configuration files, so would not be
> trivial to do any other way while using the same source package for
> Ubuntu and Debian (the best idea I have would be to use xsltproc or
> xmlstarlet or something, at which point you have two problems).

I don't object an alternative way of doing this, however, I am not going
to do the work. The reason why I did this was to remove the (packaging)
delta between Debian and Ubuntu because the package was often behind in
Ubuntu because of large (semi-) Ubuntu specific changes. Luckily the
technical code has been un-delta-ed some time ago, but if this is
forbidden I'll just drop the patch unless I get a patch that can be
up-streamed for this (I expect upstream to be helpful here).

Paul



signature.asc
Description: OpenPGP digital signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Wouter Verhelst]

On Thu, Aug 02, 2018 at 09:46:28AM +0100, Simon McVittie wrote:
> mate-terminal and tilix, both terminals, have been adapted to Ubuntu
> having patched vte to stay with pcre instead of moving to pcre2.
> mate-terminal could easily use cpp; tilix is written in D, and I don't
> know whether that has a preprocessor.

It does not, but it does have a mechanism to have a somewhat limited
subset of D code be evaluated at compile time (which is actually far
more powerful than a preprocessor)

It also has an explicit mechanism for conditional compilation, which
would apply here; https://dlang.org/spec/version.html has the details on
how that works.

-- 
Could you people please use IRC like normal people?!?

  -- Amaya Rodrigo Sastre, trying to quiet down the buzz in the DebConf 2008
 Hacklab

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Simon McVittie]

On Mon, 23 Jul 2018 at 09:22:17 +0800, Sean Whitton wrote:
> There are currently at least 18 source packages which use
> vendor-specific series files.  I have not been able to determine an
> upper bound.

Here is a survey of packages that do this, based on this search from
Stuart Prescott

$ apt-file search --index-names dsc -x ^/debian/patches/.*\\.series

and my parallel attempt:
https://codesearch.debian.net/search?q=%28diff%7Cpatch%29+path%3Adebian%2Fpatches%2F%28.*%29%5C.series

qjackctl and zlib (in the codesearch) are false positives caused by
older versions existing in the search index.

deluge switches on AppIndicator support by default but only in Ubuntu,
presumably for Unity's benefit. This is in imperative Python code, so it
could easily be runtime-conditional. I suspect this might be better
implemented by detecting a Unity (or AppIndicator-supporting) session
somehow, because people can run non-Unity desktops on Ubuntu, and can
run AppIndicator-supporting desktops on Debian.

fail2ban and libfreenect patch the contents of debian/ (!) to carry
out a semi-automated backport for neurodebian. I think this might be
more appropriately done by merging into a backports git branch, like I
did for flatpak in jessie. Both of these packages also omit the rest
of the patches from neurodebian-backport.series, so either they are
actually using some other mechanism that is not the 3.0 (quilt) vendor
patch series feature, or their backports are (incorrectly?) omitting
the Debian patch series.

filezilla, a file manager, changes how file sizes are displayed to follow
 when unpacked on Ubuntu. This
could easily be done with cpp and a compile-time conditional in the
patch if desired.

hexchat, smuxi and xchat, IRC clients, use Ubuntu IRC servers and
channels by default when unpacked on Ubuntu. Again, this could be done
with cpp (for (he)?xchat) or a runtime conditional in C# code (for smuxi)
if desired. smuxi seems to omit one Debian patch when unpacked on
Ubuntu, which is probably a bug (the patch doesn't seem Debian-specific).

libxfce4util uses X-Ubuntu-Gettext-Domain for l10n when unpacked on
Ubuntu. An analogous patch in glib2.0 is applied unconditionally, even
in Debian. I'm not sure which way is better. This is in C code and so
could easily be done with cpp if desired.

liferea, a RSS feed reader, uses Ubuntu RSS feeds by default when unpacked
on Ubuntu. This is a patch to XML configuration files, so would not be
trivial to do any other way while using the same source package for
Ubuntu and Debian (the best idea I have would be to use xsltproc or
xmlstarlet or something, at which point you have two problems).

lilo adjusts user-visible vendor/version strings. This is C code and
could easily be done with cpp, with the bonus that it would work for
any other vendor; I sent a patch to #896081, although it fails to build
(I don't understand why it fails, but I think it might be orthogonal to
my changes).

mate-power-manager hides a preference that, on Ubuntu, is handled by
indicator-power. Like deluge, this seems to be based on an assumption
that Ubuntu is strongly correlated with a desktop environment that uses
particular indicator widgets. This is a patch to XML, so not trivially
replaceable, although you could use xmlstarlet or something.

mate-terminal and tilix, both terminals, have been adapted to Ubuntu
having patched vte to stay with pcre instead of moving to pcre2.
mate-terminal could easily use cpp; tilix is written in D, and I don't
know whether that has a preprocessor. I think both of these would be
better off looking for indications that the vte has been patched in
this way, instead of checking for Ubuntu; otherwise, when Ubuntu
eventually adds pcre2 to main and stops patching vte, they will get
the opposite bug.

mixxx, a DJ-mixing GUI, has a .desktop file that attempts to run under
pasuspender so that it can play audio with direct ALSA, presumably for
smaller/more predictable latency or some other benefit of a simpler audio
path. There is a Debian patch to undo this and just run normally, and an
Ubuntu patch series to undo the Debian patch and use pasupender, with the
comment "Debian-specific as Ubuntu uses PulseAudio by default" in 2010.
The observant will note that the majority of Debian desktops now use
PulseAudio by default, too, so this reasoning might not be valid any more.

numix-gtk-theme uses an Ubuntu-preferred icon theme by patching a
declarative data file, with the comment that Ubuntu developers don't want
this GTK theme to depend on a corresponding icon theme. This looks like
it might indicate a missing (Debian) dependency, tbh... This change is
not in imperative code, but would be easy to make with sed if desired.

packagekit changes various repository and help URLs to be appropriate
for Debian or Ubuntu. This is in declarative data files, so cannot be
done with cpp, although it would probably be straightforward to do with
sed.

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Guillem Jover]

Hi Gunnar,

On Mon, 2018-07-30 at 00:14:23 -0500, Gunnar Wolf wrote:
> Still, I would like to have Guillem's opinion.

Sorry, but I'd rather not participate in the processes I very strongly
disagree with, for a body I don't think should even exist.

Thanks,
Guillem

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Sean Whitton]

Hello,

On Mon 30 Jul 2018 at 02:20PM +1000, Stuart Prescott wrote:

> The normal mantra is: "policy is to document standard practice", "policy is
> not a stick to beat maintainers with", "policy changes should (normally) not
> make packages instabuggy". The current approach is not consistent with
> Debian's normal processes for dealing with Policy; vendor.series *is* standard
> practice and packages would be made instabuggy for carrying such files if
> policy were used as a stick as requested.
>
> (I'm not claiming that this is not a misfeature or that as a project we
> shouldn't be helping maintainers and derivatives to get rid of vendor.series,
> just that pretending that it's only a policy issue and then using policy as a
> back door to overrule maintainers is completely the wrong way of doing it and
> undermines both Policy and TC.)

We're not really in the normal Policy case anymore, because normally
people talk it out and reach consensus.  For this bug that simply has
not happened.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Tollef Fog Heen]

]] Gunnar Wolf 

> ...Although if we make a policy change in this, it will require
> cooperation of the dpkg developers if we don't want to go into 6.1.4
> (which we don't).

No, it won't, it's a policy change on what's allowed in the archive, not
what features dpkg is allowed to implement.

I imagine the language we'd want to use is along the lines of «Packages
must not use dpkg's vendor-specific patch series functionality».

> I am explicitly cc:ing Guillem here. Guillem, please take a look at
> this bug and give us your PoV!

Thanks!

Cheers,
-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Gunnar Wolf]

Sean Whitton dijo [Mon, Jul 30, 2018 at 12:01:35PM +0800]:
> > ...Although if we make a policy change in this, it will require
> > cooperation of the dpkg developers if we don't want to go into 6.1.4
> > (which we don't).
> 
> Sorry, I don't follow.
> 
> Surely policy can disallow things in packages even though those things
> have support in dpkg.  For example, dpkg lets packages connect to the
> Internet to download and embed dependencies, but we forbid that.
> 
> Whether or not the feature is removed from dpkg seems orthogonal to the
> T.C. bug I filed.

You are right. Sorry for the noise.

Still, I would like to have Guillem's opinion.


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Stuart Prescott]

On Sunday, 29 July 2018 22:14:23 AEST Gunnar Wolf wrote:
> Sean Whitton dijo [Mon, Jul 30, 2018 at 09:30:04AM +0800]:
> > > I believe his request might also be considered under §6.1.1, since we're
> > > being asked about a policy change.  (After talking to Sean in person, he
> > > said he intended it under §6.1.3, not §6.1.1, though.)
> > 
> > I think technically it's §6.1.3 because according to the policy team
> > delegation, we decide what goes into the policy manual.
> > 
> > But it certainly falls under at least one of §6.1.1 and §6.1.3, and not
> > §6.1.4.
> 
> ...Although if we make a policy change in this, it will require
> cooperation of the dpkg developers if we don't want to go into 6.1.4
> (which we don't).
> 
> I am explicitly cc:ing Guillem here. Guillem, please take a look at
> this bug and give us your PoV!

It's not just the dpkg maintainers, it is also all the maintainers who are 
using this:

$ apt-file search --index-names dsc -x ^/debian/patches/.*\\.series
deluge: /debian/patches/ubuntu.series
fail2ban: /debian/patches/neurodebian-backport.series
filezilla: /debian/patches/debian.series
filezilla: /debian/patches/ubuntu.series
hexchat: /debian/patches/ubuntu.series
libfreenect: /debian/patches/neurodebian-backport.series
libxfce4util: /debian/patches/ubuntu.series
liferea: /debian/patches/ubuntu.series
lilo: /debian/patches/ubuntu.series
mate-power-manager: /debian/patches/ubuntu.series
mate-terminal: /debian/patches/ubuntu.series
mixxx: /debian/patches/ubuntu.series
numix-gtk-theme: /debian/patches/ubuntu.series
packagekit: /debian/patches/ubuntu.series
smuxi: /debian/patches/ubuntu.series
tilix: /debian/patches/ubuntu.series
xchat: /debian/patches/ubuntu.series
xfce4-smartbookmark-plugin: /debian/patches/ubuntu.series

The normal mantra is: "policy is to document standard practice", "policy is 
not a stick to beat maintainers with", "policy changes should (normally) not 
make packages instabuggy". The current approach is not consistent with 
Debian's normal processes for dealing with Policy; vendor.series *is* standard 
practice and packages would be made instabuggy for carrying such files if 
policy were used as a stick as requested.

(I'm not claiming that this is not a misfeature or that as a project we 
shouldn't be helping maintainers and derivatives to get rid of vendor.series, 
just that pretending that it's only a policy issue and then using policy as a 
back door to overrule maintainers is completely the wrong way of doing it and 
undermines both Policy and TC.)

Stuart



dd-list for the above:

Adrien Cunin 
   filezilla

Andrew Starr-Bochicchio 
   deluge (U)

Arne Bernin 
   libfreenect (U)

Cristian Greco 
   deluge

David Michael Smith 
   liferea

Debian Desktop Theme Team 
   numix-gtk-theme

Debian GNOME Maintainers 
   tilix

Debian Multimedia Maintainers 
   mixxx

Debian Xfce Maintainers 
   libxfce4util
   xfce4-smartbookmark-plugin

Debian+Ubuntu MATE Packaging Team 
   mate-power-manager
   mate-terminal
   numix-gtk-theme (U)

Emilio Pozuelo Monfort 
   liferea (U)

Evgeni Golov 
   xfce4-smartbookmark-plugin (U)

Free Ekanayaka 
   mixxx (U)

Gianfranco Costamagna 
   xchat

Jeremy Bicha 
   numix-gtk-theme (U)
   tilix (U)

Joachim Wiedorn 
   lilo

John Paul Adrian Glaubitz 
   mate-power-manager (U)
   mate-terminal (U)

Julian Andres Klode 
   packagekit (U)

Lionel Le Folgoc 
   libxfce4util (U)
   xfce4-smartbookmark-plugin (U)

Mark Renouf 
   libfreenect (U)

Martin Wimpress 
   mate-terminal (U)

Matteo F. Vescovi 
   mixxx (U)

Matthias Klumpp 
   packagekit
   tilix (U)

Mattia Rizzolo 
   hexchat

Mike Gabriel 
   mate-power-manager (U)
   mate-terminal (U)
   numix-gtk-theme (U)

Mirco Bauer 
   smuxi

Nicolas Bourdaud 
   libfreenect

Paul Brossier 
   mixxx (U)

Paul Gevers 
   liferea (U)

Petr Baudis 
   mate-power-manager (U)

Stefano Karapetsas 
   mate-power-manager (U)
   mate-terminal (U)

Vangelis Mouhtsis 
   mate-power-manager (U)
   mate-terminal (U)

Victor Seva 
   smuxi (U)

Wences Arana 
   mate-terminal (U)

Yaroslav Halchenko 
   fail2ban
   libfreenect (U)

Yves-Alexis Perez 
   libxfce4util (U)
   xfce4-smartbookmark-plugin (U)




-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Sean Whitton]

Hello,

On Sun 29 Jul 2018 at 10:14PM -0500, Gunnar Wolf wrote:

> Sean Whitton dijo [Mon, Jul 30, 2018 at 09:30:04AM +0800]:
>> > I believe his request might also be considered under §6.1.1, since we're
>> > being asked about a policy change.  (After talking to Sean in person, he
>> > said he intended it under §6.1.3, not §6.1.1, though.)
>>
>> I think technically it's §6.1.3 because according to the policy team
>> delegation, we decide what goes into the policy manual.
>>
>> But it certainly falls under at least one of §6.1.1 and §6.1.3, and not
>> §6.1.4.
>
> ...Although if we make a policy change in this, it will require
> cooperation of the dpkg developers if we don't want to go into 6.1.4
> (which we don't).

Sorry, I don't follow.

Surely policy can disallow things in packages even though those things
have support in dpkg.  For example, dpkg lets packages connect to the
Internet to download and embed dependencies, but we forbid that.

Whether or not the feature is removed from dpkg seems orthogonal to the
T.C. bug I filed.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Gunnar Wolf]

Sean Whitton dijo [Mon, Jul 30, 2018 at 09:30:04AM +0800]:
> > I believe his request might also be considered under §6.1.1, since we're
> > being asked about a policy change.  (After talking to Sean in person, he
> > said he intended it under §6.1.3, not §6.1.1, though.)
> 
> I think technically it's §6.1.3 because according to the policy team
> delegation, we decide what goes into the policy manual.
> 
> But it certainly falls under at least one of §6.1.1 and §6.1.3, and not
> §6.1.4.

...Although if we make a policy change in this, it will require
cooperation of the dpkg developers if we don't want to go into 6.1.4
(which we don't).

I am explicitly cc:ing Guillem here. Guillem, please take a look at
this bug and give us your PoV!


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Sean Whitton]

Hello,

On Sun 29 Jul 2018 at 07:52AM +0200, Tollef Fog Heen wrote:

> I believe his request might also be considered under §6.1.1, since we're
> being asked about a policy change.  (After talking to Sean in person, he
> said he intended it under §6.1.3, not §6.1.1, though.)

I think technically it's §6.1.3 because according to the policy team
delegation, we decide what goes into the policy manual.

But it certainly falls under at least one of §6.1.1 and §6.1.3, and not
§6.1.4.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Raphael Hertzog]

Hi,

On Sat, 28 Jul 2018, Colin Watson wrote:
> > Debian should not seek to prevent maintainers doing something that
> > they have agreed to do in collaboration with downstreams.
> 
> My memory of the origin of this feature is that the dpkg developer who
> originated it asked me if it might help with upstreaming changes from
> Ubuntu to Debian, I said something along the lines of "uh, maybe, I
> guess, it sounds like that would have some problems?", and then they
> went ahead and did it.  I don't *think* it was originally a request that
> came from derivatives, and I don't remember the sort of collaborative
> design that I think you're suggesting here.

As the person who implemented this vendor-specific patch series, I
have to recognize that it was a bad idea. It was easy to implement
and at that time looked like something possibly useful to encourage more
collaboration between Debian an Ubuntu.

But I agree with most of the critics that I have read and I believe
that we should forbid its use in Debian.

Now I'm no longer involved in dpkg, I don't know if Guillem is willing
to deprecate the feature. But at least it should be forbidden at the
policy level.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Tollef Fog Heen]

]] Stuart Prescott 

> Essentially, this is a request that the TC overrule both Debian
> maintainers *and* derivative maintainers in what they have agreed as a
> workflow that obviously works for them. Today, Debian decides to not
> allow debian/patches/vendor.series, then tomorrow, a derivative
> patches dpkg-source and Debian is asked to decide whether
> debian/patches/vendor-dammit-i-want-these-patches-applied.series is
> allowed in the source package.

I can see where you are coming from, but I disagree with your
conclusions.

Hypothetically, if a downstream distribution implemented support
vendor.series without there being any support in Debian for it, I don't
think we would disallow vendor.series in Debian.  (I think it would be a
bad idea for a downstream to patch such functionality into dpkg, but
there are many bad ideas that should not be forbidden.)

> (Footnote: Sean has referred this under §6.1.3, requesting a decision but 
> since this is overriding a (set of) maintainer(s) that includes those using 
> vendor.series and the dpkg maintainers, I assume §6.1.4 applies.)

I believe his request might also be considered under §6.1.1, since we're
being asked about a policy change.  (After talking to Sean in person, he
said he intended it under §6.1.3, not §6.1.1, though.)

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Colin Watson]

On Sat, Jul 28, 2018 at 02:18:51AM +1000, Stuart Prescott wrote:
> Debian does not micromanage its maintainers and Debian most certainly does 
> not tell derivatives what to do; however, that is precisely what this 
> proposal is requesting.

Is it?  What I see is more like a derivative (or at least some of its
developers; Ubuntu has not had a formal discussion about this and I'm
sure we don't have unanimity) asking that Debian update its source
package standards to forbid a feature that was implemented with the best
of intentions to make that derivative's life easier, but which has in
fact turned out to have the reverse effect.

It's technically true that Ubuntu could individually decide to patch out
this feature in both dpkg and all of the source packages that use it,
and that we haven't not done so.  That's mainly because it would be a
fair amount of work, not to mention that the patches would be of a
really strange shape (removing ubuntu.series in an Ubuntu patch ...).
Furthermore, since the feature was originally there mainly to improve
collaboration with Ubuntu, or at least that's mostly how it's ended up
being used, if Ubuntu doesn't think it's working well in practice then
Debian processes are *exactly* the correct way to discuss it.  The
alternative is a bizarre ping-pong game of patch management behaviour
that would be very confusing for everyone outside the small group of
people up to speed with it, and that would do nobody any good in the
long run.

> Debian should not seek to prevent maintainers doing something that
> they have agreed to do in collaboration with downstreams.

My memory of the origin of this feature is that the dpkg developer who
originated it asked me if it might help with upstreaming changes from
Ubuntu to Debian, I said something along the lines of "uh, maybe, I
guess, it sounds like that would have some problems?", and then they
went ahead and did it.  I don't *think* it was originally a request that
came from derivatives, and I don't remember the sort of collaborative
design that I think you're suggesting here.

I admit that we in Ubuntu didn't push back as hard as in retrospect we
should have done, and some people were positive about the idea; but at
the time, there were a lot of ideas floating around about how to make
the process of upstreaming patches better, and while many of them were
helpful it would have been hard to manage a 100% hit rate.  (dpkg-vendor
and related features were implemented around the same time, and I think
those were a good idea and have been helpful.)

That said, it was some time back and I haven't been able to find any
actual transcripts of the conversation, so I'm relying on fallible human
memory and will gladly apologise if I'm wrong.  Still, this sort of
design decision should still be open to being revisited when it turns
out to have serious problems in practice.

> One might find vendor.series to be a poor technical solution, but then
> those who find it abhorrent might seek to find a better one, rather
> than attempt to resort to administrative power.

I think Steve and I, at least, have both made it clear what we think
would be better: briefly, apply patches unconditionally but if necessary
give them conditional behaviour that can be controlled at build time.
While this occasionally means making the patch a bit more complicated,
it simplifies management of the patch *series*, and avoids a whole
category of problems that maintainers make in practice.  (Certainly I've
been saying essentially this for years when people ask about this kind
of thing on #ubuntu-devel.)

However, features built into the source package format are tempting for
maintainers to use, and so we end up playing whack-a-mole.  The only
good fix for that is to remove the feature.  It doesn't require a
replacement at the level of the source package format, but it would
require ensuring that packages stop using the feature first, and that's
the kind of interoperability issue that requires a change in policy.
Even if policy doesn't jump straight to forbidding the use of
vendor-specific series, it would make things very much easier if it were
to formally deprecate them.

To me, this is similar to the DEB_AUTO_UPDATE_DEBIAN_CONTROL feature in
CDBS.  That was also introduced with good intentions to try to make
maintainers' lives easier, and it was tempting enough that for a while
quite a few people used it; but it proved to have deleterious
side-effects and so was deprecated.  (I think this was only ever
deprecated at the level of Lintian and the ftpmaster REJECT-FAQ rather
than actually in policy, but the basic idea seems similar.)

> Essentially, this is a request that the TC overrule both Debian maintainers 
> *and* derivative maintainers in what they have agreed as a workflow that 
> obviously works for them.

Again, this is a misstatement of what derivative maintainers have in
fact decided, except perhaps by omission.

Some individual package maintainers who care about 

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Stuart Prescott]

> The concrete question that I am asking the committee to decide, in my
> capacity as a Policy delegate, is whether or not vendor-specific patch
> series should be permitted in the Debian archive.

I can see that a user might find it surprising that a source package is 
unpacked differently on different systems and I agree that it is better that 
distro-specific patches bubble up the chain towards upstreams. I agree that 
maintainers should avoid setting elephant traps.

It is not, however, within Debian's remit to decide what dpkg in a 
derivative distribution does. It is already within the power of the 
derivative to accept or reject such patches and they have chosen to accept 
these patches. If a derivative does not wish to apply such patches, then 
they should simply have dpkg-source not apply them or remove the series file 
at package import stage. This is the derivative's choice, not Debian's.

Debian does not micromanage its maintainers and Debian most certainly does 
not tell derivatives what to do; however, that is precisely what this 
proposal is requesting. Debian should not seek to prevent maintainers doing 
something that they have agreed to do in collaboration with downstreams. One 
might find vendor.series to be a poor technical solution, but then those who 
find it abhorrent might seek to find a better one, rather than attempt to 
resort to administrative power.

Essentially, this is a request that the TC overrule both Debian maintainers 
*and* derivative maintainers in what they have agreed as a workflow that 
obviously works for them. Today, Debian decides to not allow 
debian/patches/vendor.series, then tomorrow, a derivative patches dpkg-
source and Debian is asked to decide whether debian/patches/vendor-dammit-i-
want-these-patches-applied.series is allowed in the source package.

The TC has power under §6.1.4 to overrule a developer. It has no power to 
overrule a derivative but that is what is really what is being requested.

(Footnote: Sean has referred this under §6.1.3, requesting a decision but 
since this is overriding a (set of) maintainer(s) that includes those using 
vendor.series and the dpkg maintainers, I assume §6.1.4 applies.)

Stuart


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Tollef Fog Heen]

]] Sean Whitton 

> The concrete question that I am asking the committee to decide, in my
> capacity as a Policy delegate, is whether or not vendor-specific patch
> series should be permitted in the Debian archive.
> 
> There is a broader question of whether source packages should be allowed
> to unpack differently on different systems through other means, such as
> patch systems implemented in debian/rules (this could be done using
> dpkg-vendor(1)).  But given that 3.0 (quilt) is now dominant, you might
> leave this broader question aside.

I agree with Ian, Steve, and Colin that it is rather surprising for
source packages to unpack differently on a different OS.  Source
packages are (at least to most people) a transport format; not entirely
unlike tar, and those do in general not have the property that the
unpacking OS matters.  My understanding is also that vendor series are
mostly used by Ubuntu and the experience that Colin and Steve have is
that they are mostly used for the wrong reasons, and cause at least as
many problems as they solve.

I think we should disallow vendor-specific patch series in Debian, and
ask the dpkg developers to kindly deprecate the functionality given the
mentioned concerns.

As for the wider question about patch systems operating through
debian/rules, I'm a lot more comfortable allowing that since you then
run code to get to the patches-applied source code.  (I think it's a bad
idea with patches-unapplied packages, but there are use cases for it,
and not all bad ideas should be outlawed.)

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Gunnar Wolf]

Sean Whitton dijo [Mon, Jul 23, 2018 at 09:22:17AM +0800]:
> As a Debian Policy delegate I hereby request that the Technical
> Committee decide whether a proposal that has been submitted to modify
> the Debian Policy Manual should be accepted:
> 
> #850156 [n|  |  ] [dpkg-dev, debian-policy] Please firmly deprecate 
> vendor-specific series files
> 
> I am making this request because we have not been able to establish
> consensus, but I think this bug is one that we should not just leave
> sitting open against debian-policy: if the proposal is eventually
> accepted, then leaving the bug open means more vendor-specific series
> files in the archive that then have to be removed.
> 
> There are currently at least 18 source packages which use
> vendor-specific series files.  I have not been able to determine an
> upper bound.
> (...)

Hi,

FWIW, I did a first reading only of the issue, and I find it quite
agreeable. I think it boils down to the principle of least surprise.

There are many alternative and less-obvious ways where a package can
be programmed to act differently depending on where it is being built,
but having it act at source-unpackaging time effectively hides things
and potentially leads to longer head-scratching periods. #ifdef-like
mechanisms are ugly and might carry some reliability issues, but I
think they are preferable as they are so very explicit.

So... I'd like to have some more discussion on this, particularly I'd
like Mike to explain in a nutshell his views. I have not yet read
#850156 (which I really should before stating a definitive viewpoint).


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Bdale Garbee]

Sean Whitton  writes:

> The concrete question that I am asking the committee to decide, in my
> capacity as a Policy delegate, is whether or not vendor-specific patch
> series should be permitted in the Debian archive.

While I am no longer a member of the TC, and my analysis of the
situation consists only of reading this email, I would like to convey a 
strongly negative immediate reaction to the idea of allowing
vendor-specific patch series to remain in the Debian archive.  

I find Steve's argument that they represent an unhelpful middle ground 
particularly persuasive.

Regards,

Bdale


signature.asc
Description: PGP signature

Bug#904302: Whether vendor-specific patch series should be permitted in the archive

[Sean Whitton]

Package: tech-ctte
X-debbugs-cc: debian-pol...@lists.debian.org, sunwea...@debian.org, 
ijack...@chiark.greenend.org.uk, s...@debian.org, vor...@debian.org
Control: block 850156 by -1

As a Debian Policy delegate I hereby request that the Technical
Committee decide whether a proposal that has been submitted to modify
the Debian Policy Manual should be accepted:

#850156 [n|  |  ] [dpkg-dev, debian-policy] Please firmly deprecate 
vendor-specific series files

I am making this request because we have not been able to establish
consensus, but I think this bug is one that we should not just leave
sitting open against debian-policy: if the proposal is eventually
accepted, then leaving the bug open means more vendor-specific series
files in the archive that then have to be removed.

There are currently at least 18 source packages which use
vendor-specific series files.  I have not been able to determine an
upper bound.

Before continuing to summarise the issue, I should state a conflict of
interest.  I am one of the people working on dgit, and Ian Jackson's
motivation for proposing this change is because packages with
vendor-specific series files cannot be used with dgit.  I share Ian's
desire to make dgit work with as many packages as possible, and this
might have biased the following summary.  I am, however, confident in my
judgement that this proposal should be brought before the committee.

Vendor-specific patch series are a feature of dpkg that can be used to
apply a different series of quilt patches when the source package is
unpacked on different systems.  So for example, there could be an Ubuntu
patch series alongside the default patch series, and then the results of

% dpkg-source -x foo.dsc

would be different depending on whether you are running Ubuntu or
another system (e.g. Debian).

Source packages are intended as a compressed representation of unpacked
source packages, for transportation.  But thanks to vendor-specific
patch series, it is not the case that the composition of packing and
unpacking a source package is guaranteed to be the identity function.[1]
This is very odd for a transportation format.

The practical consequences fall into two categories.  Firstly, it makes
it difficult to develop tools like dgit, which must assume that the
compressed representation works like other compressed representations.
And secondly, it can be quite confusing to users working with the
compressed representation directly.  These practical consequences are
the reasons why I think we should get this bug

For example, someone might want to use a Debian system to investigate a
bug on an Ubuntu system.  They might begin by downloading some source
packages from the Ubuntu mirrors.  Since they obtained them from Ubuntu,
they will form the reasonable expectation that unpacking these source
packages will get them the code running on the Ubuntu system they are
debugging.  But if the source packages use vendor-specific patch series,
they might get something quite different -- which, until they realise,
might completely block them from resolving the bug.  As Ian puts it,
"The version of the package you get should depend on where you got the
package from, not where you are looking at it."

On the other hand, Mike Gabriel objects to the removal of
vendor-specific patch series because they are a way of reducing the
maintainance burden for packages that need to be slightly different in
Debian and in derivatives like Ubuntu:

> My main context when working for Debian derivates is: get everything
> into Debian, bind the derivatives' devs' (wo)man(or other) power to
> Debian and allow them to achieve their goals for their derivative
> distro at the same time. Maintaining several slightly different
> src:package versions in Debian and derivative X, Y and Z costs a lot
> of time.
>
> The vendor.series file is a tiny helper tool, that eases people's day
> if working in a context I described above.
>
> With Ubuntu, where the vendor.series (i.e. ubuntu.series) file is used
> here and there in my team contexts, you sometimes encounter Ubuntu
> patches in third party package (which you don't have impact on) that
> break a certain behaviour in a vanilla Debian package. Thus, having
> the mechanism to easily patch the Ubuntu build of your package is very
> handy.

A concrete example from Mike: https://bugs.debian.org/850156#60

It has been argued that vendor series are simply a more convenient way
of doing the equivalent of "#ifdef ubuntu".  Ian and I don't agree.
#ifdefs are visible in the actual source code, where users will look to
understand the behaviour.  Doing this kind of work in the source code or
the build system is much more conventional, than doing it in the source
code delivery mechanism.  In particular, it doesn't violate the
idempotency of packing and unpacking the source package.  #ifdefs like
this do not cause any trouble for tools like dgit.

A final argument in favour of Ian's proposal that seems worth