Bug#904302: Why outlawing vendor-specific patch series would be a bad idea
]] Adrian Bunk > Hi, > > looking at something where I worked on the upstream implementation ages ago: > https://sources.debian.org/src/liferea/1.12.4-1/debian/patches/ubuntu-example-feeds.patch/ > > It is a common problem that users should be able to get started quickly > after installing a program. > > When liferea is started by a user for the first time, the default feedlist > in the locale of the user gets installed as feedlist for the user. > > It is clear why a derivative, especially a brand-aware one like Ubuntu, > wants to change this feedlist. > > And it is also clear why this change cannot be applied in Debian. > > One obvious solution if vendor-specific series files get outlawed in > Debian would be to switch from ubuntu.series to manual patching in > debian/rules based on dpkg-vendor(1). Or it would mean that Ubuntu would carry a XubuntuY version and do manual (or automatic, based on whatever tooling they have available) merges from Debian, marking it clearly as a different work. [...] > The whole underlying notion that there would be one source tree that > gets built is also flawed. This is not true in all cases, and can > never be. We are not talking about what's built. We're talking about what's unpacked. It's well understood that what's unpacked is not always what's built; build processes can (and do) all kinds of transformations and is understood to be executable code. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are
Bug#904302: Why outlawing vendor-specific patch series would be a bad idea
On Fri, Aug 17, 2018 at 09:49:00AM +0200, Tollef Fog Heen wrote: > ]] Adrian Bunk > > > Hi, > > > > looking at something where I worked on the upstream implementation ages ago: > > https://sources.debian.org/src/liferea/1.12.4-1/debian/patches/ubuntu-example-feeds.patch/ > > > > It is a common problem that users should be able to get started quickly > > after installing a program. > > > > When liferea is started by a user for the first time, the default feedlist > > in the locale of the user gets installed as feedlist for the user. > > > > It is clear why a derivative, especially a brand-aware one like Ubuntu, > > wants to change this feedlist. > > > > And it is also clear why this change cannot be applied in Debian. > > > > One obvious solution if vendor-specific series files get outlawed in > > Debian would be to switch from ubuntu.series to manual patching in > > debian/rules based on dpkg-vendor(1). > > Or it would mean that Ubuntu would carry a XubuntuY version and do > manual (or automatic, based on whatever tooling they have available) > merges from Debian, marking it clearly as a different work. That's why I wrote: > On the more general point, there is no actual rationale given why the > TC should discuss outlawing vendor-specific patch series without also > discussing manual debian/rules patching in 3.0 (quilt) packages based > on dpkg-vendor(1) or other mechanisms. If vendor-specific patching (and configuration?) should move from Debian to deltas carried permanently by derivatives, then this should be discussed and decided for the general case. Outlawing a non-buggy patching implementation will in practice only result in the same patching being done with per-package patching implementations. These are more likely to be buggy, and make it impossible to easily search for all packages doing it. > [...] > > > The whole underlying notion that there would be one source tree that > > gets built is also flawed. This is not true in all cases, and can > > never be. > > We are not talking about what's built. We're talking about what's > unpacked. It's well understood that what's unpacked is not always > what's built; build processes can (and do) all kinds of transformations > and is understood to be executable code. Is it really well understood? A large part of the email that opened this bug only makes sense under the assumption that a TC decision will result in what is being unpacked becoming what will be built. More exactly, the part from "And secondly" up to and including the section discussing "#ifdef ubuntu". For arguments like "will get them the code running on the Ubuntu system" or "#ifdefs are visible in the actual source code" it doesn't matter whether the different patching based on the distribution will be done by dpkg or debian/rules, and the same applies to debian/rules patching based on the release or the architecture. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed
Bug#904302: Why outlawing vendor-specific patch series would be a bad idea
On Fri, Aug 17, 2018 at 09:49:00AM +0200, Tollef Fog Heen wrote: > > One obvious solution if vendor-specific series files get outlawed in > > Debian would be to switch from ubuntu.series to manual patching in > > debian/rules based on dpkg-vendor(1). > > Or it would mean that Ubuntu would carry a XubuntuY version and do > manual (or automatic, based on whatever tooling they have available) > merges from Debian, marking it clearly as a different work. I would like you to consider - and I think this is part of what Adrian is raising too¹ - this kind of case where the Debian maintainer *wants* to support particular derivatives in their source package in Debian and is willing to test it properly. Having this facility avoids the need for any kind of source package delta resolution process needing to take place², which might add arbitrary delays between a new package being uploaded to unstable and becoming available in the derivative's unstable suite. It means that the Debian uploader does not need to become - or to find - a derivative uploader to perform this resolution. And it avoids maintainers having to cook up their own solution if they want to do it at build time without tool support. FAOD I am not challenging that the drawbacks of vendor-specific patch series as outlined in this bug exist - just saying that Adrian's point has some merit in that having this *kind* of support (perhaps not this specific implementation) easily available is useful. TBH I'm not sure what I'd ask -ctte to do with this argument. If you do decide to outlaw the vendor-specific series, maybe advice (6.1.5) to relevant developers to consider designing a way to better support derivative specific patches within Debian. Cheers for handling this issue, -- Iain Lane [ i...@orangesquash.org.uk ] Debian Developer [ la...@debian.org ] Ubuntu Developer [ la...@ubuntu.com ] ¹ sorry for putting words in your mouth if this isn't what you meant :-) ² in Ubuntu's case this is not automatic; a human uploader needs to at the very least review the output of merge-o-matic). signature.asc Description: PGP signature
Bug#904302: Why outlawing vendor-specific patch series would be a bad idea
Iain Lane writes ("Bug#904302: Why outlawing vendor-specific patch series would
be a bad idea"):
> On Fri, Aug 17, 2018 at 09:49:00AM +0200, Tollef Fog Heen wrote:
> > > One obvious solution if vendor-specific series files get outlawed in
> > > Debian would be to switch from ubuntu.series to manual patching in
> > > debian/rules based on dpkg-vendor(1).
> >
> > Or it would mean that Ubuntu would carry a XubuntuY version and do
> > manual (or automatic, based on whatever tooling they have available)
> > merges from Debian, marking it clearly as a different work.
Quite.
If an Ubuntu user downloads the liferea source code from Debian, to
see what Debian's version is like, they should *not* get the Ubuntu
branding and default feed changes.
What this example shows is that, when a derivative wants a program to
behave differently for these kind of reasons, this should generally be
done by *actually changing the source code*.
Ubuntu already has machinery for automatically merging from Debian.
That machinery should be used. And improved, if necessary.
> I would like you to consider - and I think this is part of what Adrian
> is raising too¹ - this kind of case where the Debian maintainer *wants*
> to support particular derivatives in their source package in Debian and
> is willing to test it properly.
The matter might be different for changes which are to deal with the
presence or absence of particular things in the derivative. In that
case, the change can simply be made to Debian. But it should be
conditional not on dpkg-vendor, but on whatever the actual difference
is.
In general, I think package builds should not pay any attention to
dpkg-vendor. Can I please add that request to the TC's deliberations ?
> Having this facility avoids the need for any kind of source package
> delta resolution process needing to take place
This is, of course, false.
The vendor series file *is a source package delta resolution process*.
It's just a terrible one.
> which might add arbitrary delays between a new package being
> uploaded to unstable and becoming available in the derivative's
> unstable suite.
If new updates are not be merged automatically, by a robot, in a
timely way, then that is a tooling problem.
Note that if there is a vendor series, *other* changes made in Debian
to the upstream files in the package will be silently ignored.
This is incredibly counterintuitive and undesriable.
> TBH I'm not sure what I'd ask -ctte to do with this argument. If you do
> decide to outlaw the vendor-specific series, maybe advice (6.1.5) to
> relevant developers to consider designing a way to better support
> derivative specific patches within Debian.
"Derivative-specific patches" covers two kinds of changes:
1. Changes which are direct differences of policy or preference in the
derivative, such as changes to the default liferea feeds, or changes
to which package(s) are available.
2. Changes which are consequential on other changes. Eg, changes to
cope with different (build-)dependencies.
No changes of type (1) should never appear in Debian source packages,
for the reasons I have explained. Therefore the mechanism you are
suggesting "relevant developers" should design ought not to exist.
Instead, if carrying patches downstream is too hard, that should be
fixed. And indeed I and others are working on that.
Changes of type (2) should be ideally be dealt with by detecting the
situation at build-time where possible, but that should be done by
looking at which of the alternative build dependencies is installer,
or whatever, not by checking dpkg-vendor.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Bug#904302: Whether vendor-specific patch series should be permitted in the archive
Hello Marga, On Wed 15 Aug 2018 at 11:35AM +0200, Margarita Manterola wrote: > Apart from this, the concern that has been raised about making packages > instabuggy is valid. I would like our decision to include that this > should > be SHOULD first, giving maintainers a window of time to fix their > packages > and only become a MUST once those packages have been fixed (or something > along those lines, maybe after buster is released). I'd like to suggest giving a window of time. Otherwise the policy process alone won't really have the authority to switch from should->must. -- Sean Whitton signature.asc Description: PGP signature
Bug#904302: Why outlawing vendor-specific patch series would be a bad idea
On Fri, Aug 17, 2018 at 08:58:49AM -0700, Sean Whitton wrote: > Hello, > > On Fri 17 Aug 2018 at 12:01AM +0300, Adrian Bunk wrote: > > > On Mon, Jul 23, 2018 at 09:22:17AM +0800, Sean Whitton wrote: > >> For example, someone might want to use a Debian system to investigate a > >> bug on an Ubuntu system. They might begin by downloading some source > >> packages from the Ubuntu mirrors. Since they obtained them from Ubuntu, > >> they will form the reasonable expectation that unpacking these source > >> packages will get them the code running on the Ubuntu system they are > >> debugging. > > > > This is not a "reasonable expectation", this is a bogus assumption. > > And users should be clearly told that investigating Ubuntu problems > > on a Debian system is not a good idea - the supported way for that > > is a chroot (or some more sophisticated virtualization solution). > > People don't expect that running dpkg-buildpackage on a Debian system > would get them the binary package running on an Ubuntu system. That's > different from the expectation that the source they get is the source > running on their Ubuntu system. The main misconception is that there would always be *the* source. Steps you might have before the compilation starts: 1. dpkg unpacks upstream sources 2. dpkg applies patches 3. debian/rules unpacks upstream tarballs as part of the build 4. debian/rules applies patches based on distribution 5. debian/rules applies patches based on release 6. debian/rules applies patches based on architecture What is "the source running on their Ubuntu system" for src:gcc-8? This package skips steps 1 and 2, but does all of steps 3-6. After step 2 you have 3 upstream tarballs plus a debian/ with a pretty sophisticated custom machinery - that's not useful for seeing what gets built. If you want to look after step 6 at the source "running on your system", what you will see depends on your: - distribution: Debian or Ubuntu? - release: stretch or buster/sid? - architecture: amd64 or arm64? > Sean Whitton cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed
Bug#904302: Why outlawing vendor-specific patch series would be a bad idea
Hello, On Fri 17 Aug 2018 at 12:32PM +0100, Ian Jackson wrote: > In general, I think package builds should not pay any attention to > dpkg-vendor. Can I please add that request to the TC's deliberations ? This is about package builds, not the unpacking of source packages, so is surely a completely separate issue (which we should discuss at the level of Debian Policy, not refer directly to the T.C.). -- Sean Whitton signature.asc Description: PGP signature
Bug#904302: Why outlawing vendor-specific patch series would be a bad idea
Hello, On Fri 17 Aug 2018 at 12:01AM +0300, Adrian Bunk wrote: > On Mon, Jul 23, 2018 at 09:22:17AM +0800, Sean Whitton wrote: >> For example, someone might want to use a Debian system to investigate a >> bug on an Ubuntu system. They might begin by downloading some source >> packages from the Ubuntu mirrors. Since they obtained them from Ubuntu, >> they will form the reasonable expectation that unpacking these source >> packages will get them the code running on the Ubuntu system they are >> debugging. > > This is not a "reasonable expectation", this is a bogus assumption. > And users should be clearly told that investigating Ubuntu problems > on a Debian system is not a good idea - the supported way for that > is a chroot (or some more sophisticated virtualization solution). People don't expect that running dpkg-buildpackage on a Debian system would get them the binary package running on an Ubuntu system. That's different from the expectation that the source they get is the source running on their Ubuntu system. -- Sean Whitton signature.asc Description: PGP signature
Bug#904302: Why outlawing vendor-specific patch series would be a bad idea
Adrian Bunk writes ("Bug#904302: Why outlawing vendor-specific patch series
would be a bad idea"):
> The main misconception is that there would always be *the* source.
>
> Steps you might have before the compilation starts:
> 1. dpkg unpacks upstream sources
> 2. dpkg applies patches
> 3. debian/rules unpacks upstream tarballs as part of the build
> 4. debian/rules applies patches based on distribution
> 5. debian/rules applies patches based on release
> 6. debian/rules applies patches based on architecture
I disagree that (4) should ever be relevant. There should never be
any patchese applied conditionally based on dpkg-vendor, for the
reasons I explained very recently in response to the liferea example./
We don't ever do (5), do we ? Please tell me we don't. We can have
different source code in our different releases.
I can see that (6) might be needed in some exceptional cases but
normally there is a better way.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Bug#904302: Why outlawing vendor-specific patch series would be a bad idea
On Fri, Aug 17, 2018 at 07:33:02PM +0100, Ian Jackson wrote:
> Adrian Bunk writes ("Bug#904302: Why outlawing vendor-specific patch series
> would be a bad idea"):
> > The main misconception is that there would always be *the* source.
> >
> > Steps you might have before the compilation starts:
> > 1. dpkg unpacks upstream sources
> > 2. dpkg applies patches
> > 3. debian/rules unpacks upstream tarballs as part of the build
> > 4. debian/rules applies patches based on distribution
> > 5. debian/rules applies patches based on release
> > 6. debian/rules applies patches based on architecture
>
> I disagree that (4) should ever be relevant. There should never be
> any patchese applied conditionally based on dpkg-vendor, for the
> reasons I explained very recently in response to the liferea example./
>
> We don't ever do (5), do we ? Please tell me we don't. We can have
> different source code in our different releases.
For packages like src:firefox-esr the same source code might
be maintained to support releases ranging from oldoldstable to
unstable - 52.8.0esr-1~deb7u1 is a security update for wheezy
that is technically a backport of a package in buster/sid.
I do not know whether firefox-esr does patching based on release right
now, but this is a case where one package is being used to provide
security support for up to 4 different Debian releases at the same time.
I would also not be surprised if some package would do different
patching based on release for easy rebuilding in stable-backports,
this would sound like a natural solution to me.
> I can see that (6) might be needed in some exceptional cases but
> normally there is a better way.
As I said, src:gcc-8 does all of steps 3-6.
Much of the relevant code is in
https://sources.debian.org/src/gcc-8/8.2.0-4/debian/rules.patch/
Things are even more interesting due to this being a debian/ shared
between Debian and Ubuntu which are using different upstream sources.[1]
> Ian.
cu
Adrian
[1] GFDL
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
