Bug#1007717: marked as done (Native source package format with non-native version)

[Debian Bug Tracking System]

Your message dated Mon, 27 Jun 2022 09:23:25 -0700
with message-id <87v8smkrcy@athena.silentflame.com>
and subject line Bug#1007717: Native source package format with non-native 
version
has caused the Debian Bug report #1007717,
regarding Native source package format with non-native version
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)


-- 
1007717: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1007717
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: tech-ctte

(Sorry for the resend, this should have gone to the BTS the first
time; have fixed a slip in the wording.)

Please:

Part I - belss continued use of 1.0 native format, for now at least:

 1. Declare explicitly that there is nothing wrong with a package with
a native format, but a non-native version number.

 2. Request that the dpkg maintainer relax the restriction which
prevents the use of 3.0 native with Debian revision.

 3. Consequently, declare that the recent MBF on this topic ought not
to have been filed against packages where simply changing the
source format does not currently work.  That would include at
least 1.0 native packages with Debian revisions.

Part II - bless continued use of 1.0-with-diff, for now at least:

 4. Declare that sometimes the use of 1.0-with-diff can be the best
tradeoff between different considerations.  In particular,
because 1.0 is the only format which botH:
 (a) Optimises bandwidth and storage by reusing the upstream 
 data when it hasn't changed.
 (b) Avoids polluting the working tree (package source code)
 with [patches], which cause trouble especially with
 git-based workflows.

 5. Consequently, declare that the recent MBF on this topic ought not
to have been filed against 1.0 with diff packages, at least
without some further filter.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
--- End Message ---
--- Begin Message ---
With three first preference votes for A and five first preference votes
for C, the outcome is no longer in doubt.  Therefore, using its powers
under constitution 6.1.5, the Technical Committee issues the following
advice:

  1. It is not a bug of any severity for a package with a non-native
 version number to use a native source package format.

  2. Thus, we think that dpkg shouldn't issue warnings, or otherwise
 complain, when a non-native version number is used w/ 3.0 (native).

  3. We suggest that the wontfix tag on #737634 be reconsidered.

  4. We believe that there are indeed circumstances in which
 1.0-with-diff is the best choice for a particular source package,
 including, but not limited to, git-first packaging workflows.
 However, we recommend discontinuing use of 1.0-with-diff in other
 circumstances, to simplify the contents of the archive.

 This is because there is currently no other source format which is
 such that avoid both (i) uploading the whole source, including
 upstream, for every upload; and (ii) having to maintain
 debian/patches/ inside the package tree.

  5. We decline to comment on the recent source package format MBF.

-- 
Sean Whitton--- End Message ---

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Sean Whitton]

Hello,

On Wed 08 Jun 2022 at 09:07pm +02, Helmut Grohne wrote:

> I find it interesting that you seem to equate git-first with dgit. My
> mental model separated those concepts and considered git-first workflows
> on salsa as well. And once you equate them, you can derive a lot of
> conclusions. In my previous mail, I stated that dgit provides the sort
> of uniformity I was looking for quite many aspects. I'm less sure that
> all git-first users are dgit users though.
>
> I think I take more issue with non-dgit git-first workflows than with
> dgit ones, because dgit is so well documented and is a workflow that is
> already shared by a noticeable fraction of the archive.

Didn't mean to equate them, sorry.  We can state the following:

dgit should have support for all git-first workflows.  I am not sure
whether there are any git-first workflows in use on salsa for which you
can't 'dgit push-source' atm.  If there are, those are dgit bugs.

> What is not entirely clear to me is why dgit requires the 1.0-with-diff
> format features. It seems to me that it quite happily deals with a
> number of 3.0 (quilt) packages already. I suppose that you'll now
> explain to me how some git trees are not representable as 3.0 (quilt)
> packages, but do those exist in practice or are those mostly
> pathological?

Sorry, didn't mean to suggest that dgit requires 1.0-with-diff.  It does not.

>> The goal is to attack this problem on two fronts:
>>
>> 1. reduce the need for uniformity by making it possible for people to
>>get their cross-archive work done using 'dgit clone'
>
> I've seen this argument multiple times already and indeed dgit solves
> part of the uniformity issues. However, dgit history often lacks
> maintainer history (and in that way is little better than apt source)

Right, (1) is about making it easy for people to be using 'dgit
push-source' such that the maintainer history is there when you 'dgit clone'.

> and it also lacks a collaboration feature that would save me from
> sending a .debdiff to the bts. Possibly, such a collaboration feature
> is out-of-scope for dgit, but then maybe it is not the kind of tool
> solving the problem of streamlining cross-archive work.

Not out of scope, we have some notes for 'dgit nmudiff' in the BTS.
Would be cool to collaborate with you on finishing that up.

> Either way, my understanding is that unless maintainers switch to
> using dgit primarily, we just gain a secondary workflow here.

No -- (1) is all about ensuring that maintainers don't have to change
their workflows aside from how they do the actual upload.

>> 2. develop git-first workflows that solve all the existing usecases.
>
> Can i rephrase that as follows? Implement so many features into dgit
> that its workflow covers the needs of existing workflows and hope for
> maintainers to migrate to dgit.
>
> It feels a bit like https://xkcd.com/927/ (14 competing standards ->
> 15). On the other hand, dgit is remarkably successful already.

I don't think it can be rephrased that way.

Firstly, (1) is about dgit, but (2) is about tools like git-debrebase(1)
and workflows such as dgit-maint-merge(7).

More substantially, (1) is about attaching maintainer histories to the
dgit view, and (2) is about developing workflows which enables the
maintainer and dgit views to be identical.  That's what I mean when I
say that (1) and (2) are attacking the problem on two fronts.

(2) is particularly important for submitting NMU changes to the
maintainer -- it enables using salsa merge requests, for example.

> I had hoped that we could kinda trim the available workflows eventually.
> It seems like you want to let them all die slowly and concurrently.
>
> [...]
>
> I don't think there is "the git-first work".  Instead there is lots of
> concurrent git-first workflows that are all somewhat similar and yet
> subtly different. Those subtle differences is what I would like to get
> rid of.
>
> Now we've turned a discussion about source package formats into a
> discussion about workflows and git. So when I reason about uniformity, I
> effectively want those idiosyncratic workflows to go away. If dgit
> requires 1.0-with-diff for now, then maybe we could phrase it as an
> exception that is specific to dgit and limited until a better solution
> (such as the format proposed by Ian) is mature. If there are more
> git-first workflows beyond dgit that we want to support, maybe we can
> require declaring a working Vcs-Git for 1.0-with-diff uploads?

My own appraisal of the situation is that we do not know enough yet to
be thinking about cutting back on features and workflows.  But I
certainly agree that it would be better to have a better solution, like
Ian's sketch.

I think Ian has already suggested adding text to our resolution
recommending the development of such a source format.  To be honest I
don't think it's really necessary as everyone agrees it would be better
to have it, but if you think it should be said explicitly then let's 

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Lucas Nussbaum]

On 08/06/22 at 21:07 +0200, Helmut Grohne wrote:
> I think I take more issue with non-dgit git-first workflows than with
> dgit ones, because dgit is so well documented and is a workflow that is
> already shared by a noticeable fraction of the archive.

I'm curious: how do you measure dgit usage?

You cannot just look at the list of repositories at
https://browse.dgit.debian.org/, since that just says that at some point
in the past, someone used dgit to work on that package, right?
For example, the keepass2 package has a repo there, but it is completely
outdated compared to what is in unstable.

Lucas

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Lucas Nussbaum]

On 08/06/22 at 21:07 +0200, Helmut Grohne wrote:
> Now we've turned a discussion about source package formats into a
> discussion about workflows and git. So when I reason about uniformity, I
> effectively want those idiosyncratic workflows to go away. If dgit
> requires 1.0-with-diff for now, then maybe we could phrase it as an
> exception that is specific to dgit and limited until a better solution
> (such as the format proposed by Ian) is mature. If there are more
> git-first workflows beyond dgit that we want to support, maybe we can
> require declaring a working Vcs-Git for 1.0-with-diff uploads?

I think that the workflow used by the Debian X team is such a git-first
workflow that is not using dgit. That workflow combines Debian-specific
patches managed by quilt, and commits cherry-picked from upstream
directly applied to the source in git (outside of quilt).
See 

There are 89 packages maintained by Debian X among the 607 packages in
testing still using 1.0.

Lucas

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Helmut Grohne]

Hi Sean,

On Tue, Jun 07, 2022 at 04:35:24PM -0700, Sean Whitton wrote:
> I disagree with you that this is primarily about package ownership, and
> I think that we agree on more than you realise we do :)

Hmm. It's not that obvious. While it would be possible to remove the
choice of workflow from strong package ownership, the way we practice
package ownership presently grants that freedom. Therefore I think they
still are closely related.

> The git-first people are not making a trade-off between the maintainer's
> convenience and the convenience of others who want to work with the
> package.  The most important reason for wanting both (i) git-first
> workflows and (ii) uploads done with dgit, is to make it easier for
> people other than the maintainer to work with the package.  So, your
> priorities are quite in agreement with those of Ian, Sam, Russ and I.

I find it interesting that you seem to equate git-first with dgit. My
mental model separated those concepts and considered git-first workflows
on salsa as well. And once you equate them, you can derive a lot of
conclusions. In my previous mail, I stated that dgit provides the sort
of uniformity I was looking for quite many aspects. I'm less sure that
all git-first users are dgit users though.

> In other words, I would like to make weaker package ownership more
> possible, in a project with Debian's history, by improving our tools for
> dealing with packages for which you're not primary maintainer.

I do see how this is a dgit goal. It just seems to me that dgit bolts a
secondary workflow onto packages that maintainers are free to ignore
entirely. Some choose to use that dgit workflow exclusively and others
don't. In a sense, the problem with dgit is that it leaves too much
freedom to maintainers (and in a project like Debian, it really has to
do exactly that or it would run into straight opposition).

> What we disagree about is the extent to which the current tooling
> amounts to a failed experiment, such that we should give up on it and
> fall back to 'apt-get source'.  Many people strongly prefer 'dgit clone'
> to 'apt-get source' already, and the number of people switching to
> upload with 'dgit [--gbp] push-source' is steadily rising.  Against this
> backdrop, some of us are interested in git-first workflows for reducing
> the distance between the output of 'debcheckout' and 'dgit clone'.

Indeed, dgit kinda faces a chicken & egg problem and it seems that it is
quite good at solving it: The usefulness of dgit grows with its
adoption. I have looked into replacing my apt source workflow with dgit
clone a number of times already and will continue trying.

> It's completely reasonable that you're more sceptical about the
> prospects of solving the outstanding issues in this programme than
> others are, but surely we can agree that this is an ongoing piece of
> work rather than something we're sure we can reject?  And if keeping an
> old source package format around is needed for that work to continue,
> then that's a compelling reason to keep it around.

I think I take more issue with non-dgit git-first workflows than with
dgit ones, because dgit is so well documented and is a workflow that is
already shared by a noticeable fraction of the archive.

What is not entirely clear to me is why dgit requires the 1.0-with-diff
format features. It seems to me that it quite happily deals with a
number of 3.0 (quilt) packages already. I suppose that you'll now
explain to me how some git trees are not representable as 3.0 (quilt)
packages, but do those exist in practice or are those mostly
pathological?

> > How would you go about reducing them to a sane number?
> 
> The goal is to attack this problem on two fronts:
> 
> 1. reduce the need for uniformity by making it possible for people to
>get their cross-archive work done using 'dgit clone'

I've seen this argument multiple times already and indeed dgit solves
part of the uniformity issues. However, dgit history often lacks
maintainer history (and in that way is little better than apt source)
and it also lacks a collaboration feature that would save me from
sending a .debdiff to the bts. Possibly, such a collaboration feature is
out-of-scope for dgit, but then maybe it is not the kind of tool solving
the problem of streamlining cross-archive work.

Either way, my understanding is that unless maintainers switch to using
dgit primarily, we just gain a secondary workflow here.

> 2. develop git-first workflows that solve all the existing usecases.

Can i rephrase that as follows? Implement so many features into dgit
that its workflow covers the needs of existing workflows and hope for
maintainers to migrate to dgit.

It feels a bit like https://xkcd.com/927/ (14 competing standards ->
15). On the other hand, dgit is remarkably successful already.

I had hoped that we could kinda trim the available workflows eventually.
It seems like you want to let them all die slowly and concurrently.

> > You can rephrase

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Sean Whitton]

Hello,

On Wed 08 Jun 2022 at 12:06pm +02, Julian Andres Klode wrote:

> On Tue, Jun 07, 2022 at 08:19:29PM +0200, Helmut Grohne wrote:
>> Please keep in mind that this is about trade-offs. It is a question of
>> how we value "package ownership". If we favour the strong ownership
>> approach that Debian used for a long time, then yes accommodating the
>> needs of maintainers is paramount. If we want to lessen the concept of
>> ownership, embrace drive-by contributions and decentralize maintenance,
>> then we need a stronger focus on uniformity. I suppose that my own
>> opinion on this matter is fairly obvious at this point.
>
> This is also a significant issue for downstreams. With my Ubuntu hat
> on, if I have to touch packages downstream, I loathe it everytime I
> get a non-descript blob of all the changes.

This is not inherent to all of the workflows we are discussing here,
just one of them.  Indeed, this is one of the primary motivations for
one of the others.

-- 
Sean Whitton

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Julian Andres Klode]

On Tue, Jun 07, 2022 at 08:19:29PM +0200, Helmut Grohne wrote:
> Please keep in mind that this is about trade-offs. It is a question of
> how we value "package ownership". If we favour the strong ownership
> approach that Debian used for a long time, then yes accommodating the
> needs of maintainers is paramount. If we want to lessen the concept of
> ownership, embrace drive-by contributions and decentralize maintenance,
> then we need a stronger focus on uniformity. I suppose that my own
> opinion on this matter is fairly obvious at this point.

This is also a significant issue for downstreams. With my Ubuntu hat
on, if I have to touch packages downstream, I loathe it everytime I
get a non-descript blob of all the changes.

I suspect this is the same for other downstream distributions that
want to modify packages. We cannot cater to everyone's individual
packaging repository approach, and downstream repositories if they
exist are separate anyhow.

Using 1.0 instead of 3.0 (quilt) is just being a hostile upstream,
like Red Hat is with dumping all their kernel patches together.

-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer  i speak de, en

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Sean Whitton]

Hello,

On Tue 07 Jun 2022 at 11:26am +01, Ian Jackson wrote:

> In this case I would like to suggest that progress would be better
> served by trying to unblock a better source format that (i) has some
> kind of delta representation (ii) does not put a
> needing-to-be-maintained copy of the delta metadata inside the working
> tree.
>
> There are several proposals for how to do that which are obviously
> possible to implement.  If the TC wants to unblock that, you could
> look at them and pick one.
>
> (And, I quextion whether .dsc format is the right place for Debian to
> pursue uniformity.  .dsc is a legacy format we are maintaining because
> our git transition is stalled.)

I think that this would be out-of-scope for this bug as presented to us:
we'd need someone to present those options for us to choose between.
Let's stick with the current resolution text for a bug that's probably
been open too long already.

-- 
Sean Whitton

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Sean Whitton]

Hello,

On Tue 07 Jun 2022 at 08:19pm +02, Helmut Grohne wrote:

> Please keep in mind that this is about trade-offs. It is a question of
> how we value "package ownership". If we favour the strong ownership
> approach that Debian used for a long time, then yes accommodating the
> needs of maintainers is paramount. If we want to lessen the concept of
> ownership, embrace drive-by contributions and decentralize maintenance,
> then we need a stronger focus on uniformity. I suppose that my own
> opinion on this matter is fairly obvious at this point.

I disagree with you that this is primarily about package ownership, and
I think that we agree on more than you realise we do :)

The git-first people are not making a trade-off between the maintainer's
convenience and the convenience of others who want to work with the
package.  The most important reason for wanting both (i) git-first
workflows and (ii) uploads done with dgit, is to make it easier for
people other than the maintainer to work with the package.  So, your
priorities are quite in agreement with those of Ian, Sam, Russ and I.

In other words, I would like to make weaker package ownership more
possible, in a project with Debian's history, by improving our tools for
dealing with packages for which you're not primary maintainer.

What we disagree about is the extent to which the current tooling
amounts to a failed experiment, such that we should give up on it and
fall back to 'apt-get source'.  Many people strongly prefer 'dgit clone'
to 'apt-get source' already, and the number of people switching to
upload with 'dgit [--gbp] push-source' is steadily rising.  Against this
backdrop, some of us are interested in git-first workflows for reducing
the distance between the output of 'debcheckout' and 'dgit clone'.

It's completely reasonable that you're more sceptical about the
prospects of solving the outstanding issues in this programme than
others are, but surely we can agree that this is an ongoing piece of
work rather than something we're sure we can reject?  And if keeping an
old source package format around is needed for that work to continue,
then that's a compelling reason to keep it around.

> How would you go about reducing them to a sane number?

The goal is to attack this problem on two fronts:

1. reduce the need for uniformity by making it possible for people to
   get their cross-archive work done using 'dgit clone'

2. develop git-first workflows that solve all the existing usecases.

> You can rephrase it as reducing complexity if that helps. It's not the
> one additional source package format that is too much. It's that we
> continue using all the failed experiments while adding new ones and when
> some Lucas comes around and tries to clean up the mess he gets referred
> to us.

As I said, I don't think it's fair to characterise the git-first work as
a failed experiment.  It's ongoing work.  And the source package format
is just a way to keep it going, at the present moment.

> I have anecdotal evidence that people find Debian's processes too
> complex. Unless you closely work withing a particular packaging team
> (with unified processes), onboarding people into Debian takes very long
> compared to other projects.

Right, hence why we want 'dgit clone' to be as useful as possible.

-- 
Sean Whitton

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Sean Whitton]

Hello,

On Tue 07 Jun 2022 at 09:31am +02, Lucas Nussbaum wrote:

> A middle ground between (4) and (4b) could be to discourage the use of
> 1.0-with-diff in circumstances where it is not justified. Something
> like:
>
> 4c. We believe that there are indeed circumstances in which
> 1.0-with-diff is the best choice for a particular source package,
> including, but not limited to, git-first packaging workflows.
> However, we recommend discontinuing use of 1.0-with-diff in other
> circumstances to gain more uniformity.
>
> That opens the path to an archive where the only remaining 1.0 packages
> are the ones where there's a reason to use 1.0. (as opposed to
> currently, where we have a mix of packages using 1.0 for a good reason,
> and packages using 1.0 because nobody cared to update them to modern
> practices).

This would be a good ballot option to include, thank you.

-- 
Sean Whitton

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Helmut Grohne]

Hi Sean,

On Mon, Jun 06, 2022 at 11:08:48PM -0700, Sean Whitton wrote:
> I think this argument needs to be made more precise -- we should be
> clearer about why this particular un-uniformity is bad.  I don't think
> the issue for new contributors is persuasive enough, as new contributors
> can mostly ignore source packages.  It's not like, e.g., debhelper
> vs. cdbs.  I haven't yet seen an argument that the lack of uniformity is
> doing anyone's work any harm comparable to the harm done to things like
> what Ian and Sam want to do.

It seems to me that Ian et al works from the premise that source
packages are an export format and that git is the truth. In a sense,
that is correct as that's what most maintainers work with.
Unfortunately, packaging repositories have very diverse workflows, lack
discoverability and miss a simple trust root. A number of people have
tried to reconcile the various workflows, but that helps only so much.
On the other hand, source packages offer a very simple trust root and
uniform interface. If I am to modify an arbitrary package (with no prior
knowledge about it), I have some options to go about it.

I can use debcheckout. For a few packages, there is no repository
declared. Yet others point to non-existent servers. Still others point
to an obsolete repository that has been moved. After checking it out, I
have a git tree of something. It may come without upstream sources (e.g.
DHG) or with upstream sources. Patches may be applied or may be not. It
may be for the version I'm looking at or including unreleased changes or
pointing at some other suite. The representation of patches in git is
another story. The source repository may provide the ability to create
pull requests. Maybe the maintainer looks at them or maybe I'll have to
send a mail to the bts. I have no chance of figuring out this mess
without investing significant amount of time.

Then I can use dgit. It is way more uniform in a lot of aspects. Its
trust root resides with the CA cartel. The resulting tree may or may not
be representing the version I'm interested in. The history may be useful
or autogenerated for my checkout as initial checkin. The Debian
maintainer may be completely ignoring this representation. Most likely,
I won't be able to send a pull request.

And then, I can apt source the package. This is really simple and has an
easy to understand trust root. It also is reproducible. Unless using a
1.0 source format, debian/source/format tells me what kind of patch
system to expect and gladly there aren't too many options. I can
concentrate on the substance of my change without being distracted by
the packaging workflow details.

Please keep in mind that this is about trade-offs. It is a question of
how we value "package ownership". If we favour the strong ownership
approach that Debian used for a long time, then yes accommodating the
needs of maintainers is paramount. If we want to lessen the concept of
ownership, embrace drive-by contributions and decentralize maintenance,
then we need a stronger focus on uniformity. I suppose that my own
opinion on this matter is fairly obvious at this point.

Annoyingly enough, 3.0 (quilt) can be abused as well by deleting the
series file and managing patch application inside debian/rules. We do
have packages doing just that.

> I agree, it's not about the benefits of the source format, we do indeed
> understand all the trade-offs by now.  It's that certain ideas and
> workflows *which are not really about source packages* are made
> inconvenient or impossible if we remove this option.  In other words, it
> needs to be replaced before it can be deprecated.

I admit being somewhat ignorant towards workflows, because there are
just too many for me to be bothered to learn them all. In any case, I do
take issue with the premise that workflows need to continue to work.
We're deleting packages from unstable and stop supporting unique use
cases all the time. In most cases, we could continue supporting them for
longer, but we choose not to. I hope we all agree that there are too
many competing workflows. How would you go about reducing them to a sane
number?

> Thanks for coming up with the text.  I'd say that as uniformity is not
> good in itself, it would be good to have more concrete reasons for
> wanting uniformity in this case documented in this bug (not necessarily
> in the resolution text) before we add it to the ballot.

You can rephrase it as reducing complexity if that helps. It's not the
one additional source package format that is too much. It's that we
continue using all the failed experiments while adding new ones and when
some Lucas comes around and tries to clean up the mess he gets referred
to us.

I have anecdotal evidence that people find Debian's processes too
complex. Unless you closely work withing a particular packaging team
(with unified processes), onboarding people into Debian takes very long
compared to other projects.

Helmut

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Ian Jackson]

Christoph Berg writes ("Re: Bug#1007717: Draft resolution for "Native source 
package format with non-native version""):
> Re: Lucas Nussbaum
> > 4c. We believe that there are indeed circumstances in which
> > 1.0-with-diff is the best choice for a particular source package,
> > including, but not limited to, git-first packaging workflows.
> > However, we recommend discontinuing use of 1.0-with-diff in other
> > circumstances to gain more uniformity.
...
> The bit I was missing is something like "we would welcome changes to
> the 3.0 format to make it usable for the remaining cases where 1.0 is
> still better today". Did anyone investigate if that would be feasible?

A format that solves this problem well is entirely feasible on a
technical level, and not even particularly difficult.

Contenders for the details that have been mentioned in this thread[1]
are 3.0-with-git-diff (#1007781) and Lucas's suggestion for
incremental tarballs in this conversation
(https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1007717#185).

That could certainly fit within "changes to 3.0" since 3.0 already
encompasses a very wide variety of formats.

The real diifficulty isn't technical.  It's that it isn't worth
anyone's time *implementing* it because of the strong possibility that
its support and deployment would be blocked.

Perhaps TC would be willing to say something like this

   We hope for the development of a version of the 3.0 format which is
   usable for the remaining cases where 1.0 is still better today.

   The "3.0 with git diff" proposal in #1007781 seems to us to be a
   good option, and we encourage its implementation.  If the
   implementation quality is good, we feel it should be supported
   within Debian (subject to an appropriate transition plan).

I think 3.0-with-git-diff is the most promising candidate because it
fits into the existing ftpmaster workflow exactly as 1.0-with-diff
does now; this isn't so true of Lucas's incremental tarballs, and it
is even less true of my earlier 3.0-with-rsync-delta.

If the TC wants to give its blessing to one of the other format
proposals then that would be fine too from my point of view, but my
personal feeling is that it would be setting the TC up for a fight
with ftpmaster.  From what I know of the ftpmaster workflow I think
even Lucas's incremental tarball proposal would be a retrograde step
for them.

Ian.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Matthew Vernon]

On 07/06/2022 07:08, Sean Whitton wrote:


I agree, it's not about the benefits of the source format, we do indeed
understand all the trade-offs by now.  It's that certain ideas and
workflows *which are not really about source packages* are made
inconvenient or impossible if we remove this option.  In other words, it
needs to be replaced before it can be deprecated.


Mmm. And I'd be pretty reluctant to talk about deprecation until the 
replacement is good to go.



What would you think about adding an alternative option 4?

4b. We believe that there are indeed circumstances in which
 1.0-with-diff is the best choice for a particular source package.
 Given that the number of packages for which this is relevant is
 fairly small, we recommend discontinuing use of 1.0-with-diff to
 gain more uniformity.


Thanks for coming up with the text.  I'd say that as uniformity is not
good in itself, it would be good to have more concrete reasons for
wanting uniformity in this case documented in this bug (not necessarily
in the resolution text) before we add it to the ballot.


I would certainly vote against 4b as drafted; I would need considerable 
persuasion that "more uniformity" here is a concrete benefit.


Regards,

Matthew

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Ian Jackson]

Helmut Grohne writes ("Re: Bug#1007717: Draft resolution for "Native source 
package format with non-native version""):
> What would you think about adding an alternative option 4?
> 
> 4b. We believe that there are indeed circumstances in which
> 1.0-with-diff is the best choice for a particular source package.
> Given that the number of packages for which this is relevant is
> fairly small, we recommend discontinuing use of 1.0-with-diff to
> gain more uniformity.

It seems to me that the lack of uniformity (and indeed technical
superiority) here is due entirely due to our inability to move forward
with a replacement for 1.0-with-diff that is actually superior for a
wider range of use cases.

I know that the TC has a history of favouring what looks like
"progress" over other factors (such as contributor happiness, software
diversity, and indeed inconvenient details).

In this case I would like to suggest that progress would be better
served by trying to unblock a better source format that (i) has some
kind of delta representation (ii) does not put a
needing-to-be-maintained copy of the delta metadata inside the working
tree.

There are several proposals for how to do that which are obviously
possible to implement.  If the TC wants to unblock that, you could
look at them and pick one.

(And, I quextion whether .dsc format is the right place for Debian to
pursue uniformity.  .dsc is a legacy format we are maintaining because
our git transition is stalled.)

Ian.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Christoph Berg]

Re: Lucas Nussbaum
> A middle ground between (4) and (4b) could be to discourage the use of
> 1.0-with-diff in circumstances where it is not justified. Something
> like:
> 
> 4c. We believe that there are indeed circumstances in which
> 1.0-with-diff is the best choice for a particular source package,
> including, but not limited to, git-first packaging workflows.
> However, we recommend discontinuing use of 1.0-with-diff in other
> circumstances to gain more uniformity.
> 
> That opens the path to an archive where the only remaining 1.0 packages
> are the ones where there's a reason to use 1.0. (as opposed to
> currently, where we have a mix of packages using 1.0 for a good reason,
> and packages using 1.0 because nobody cared to update them to modern
> practices).

The bit I was missing is something like "we would welcome changes to
the 3.0 format to make it usable for the remaining cases where 1.0 is
still better today". Did anyone investigate if that would be feasible?

Christoph

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Lucas Nussbaum]

On 07/06/22 at 07:43 +0200, Helmut Grohne wrote:
> Hallo,
> 
> On Tue, May 10, 2022 at 05:29:57PM -0700, Sean Whitton wrote:
> > DRAFT
> > 
> > Using its powers under constitution 6.1.5, the Technical Committee
> > issues the following advice:
> 
> I've given this some thought and feel uneasy about one item.
> 
> >   4. We believe that there are indeed circumstances in which
> >  1.0-with-diff is the best choice for a particular source package,
> >  including, but not limited to, git-first packaging workflows.
> 
> While I can agree with this item on a technical level, I think there is
> more to it than that and I am wondering whether it sends the "right"
> message.
> 
> Sometimes, things we do are technically possible and fill a niche well.
> Yet, we decide that it is no longer reasonable to continue supporting
> them and remove their support despite the feature being useful to some.
> Quite clearly, there is a trade-off involved here. Continuing to support
> 1.0-with-diff comes with a cost that reduces uniformity inside the
> archive. Evidently, this is what motivated Lucas to file the MBF
> initially. My experience is that lack of uniformity is a significant
> barrier to prospective contributors to Debian.
> 
> Exploring different technical approaches does have value as well, but I
> think we've had sufficient time to consider the various advantages and
> disadvantages of various source packages formats. On a whole, it seems
> to me that that the number of packages benefiting from 1.0-with-diff is
> relatively small.
> 
> What would you think about adding an alternative option 4?
> 
> 4b. We believe that there are indeed circumstances in which
> 1.0-with-diff is the best choice for a particular source package.
> Given that the number of packages for which this is relevant is
> fairly small, we recommend discontinuing use of 1.0-with-diff to
> gain more uniformity.

Hi,

A middle ground between (4) and (4b) could be to discourage the use of
1.0-with-diff in circumstances where it is not justified. Something
like:

4c. We believe that there are indeed circumstances in which
1.0-with-diff is the best choice for a particular source package,
including, but not limited to, git-first packaging workflows.
However, we recommend discontinuing use of 1.0-with-diff in other
circumstances to gain more uniformity.

That opens the path to an archive where the only remaining 1.0 packages
are the ones where there's a reason to use 1.0. (as opposed to
currently, where we have a mix of packages using 1.0 for a good reason,
and packages using 1.0 because nobody cared to update them to modern
practices).

Lucas

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Sean Whitton]

Hello,

On Tue 07 Jun 2022 at 07:43am +02, Helmut Grohne wrote:

> While I can agree with this item on a technical level, I think there is
> more to it than that and I am wondering whether it sends the "right"
> message.
>
> Sometimes, things we do are technically possible and fill a niche well.
> Yet, we decide that it is no longer reasonable to continue supporting
> them and remove their support despite the feature being useful to some.
> Quite clearly, there is a trade-off involved here. Continuing to support
> 1.0-with-diff comes with a cost that reduces uniformity inside the
> archive. Evidently, this is what motivated Lucas to file the MBF
> initially. My experience is that lack of uniformity is a significant
> barrier to prospective contributors to Debian.

I think this argument needs to be made more precise -- we should be
clearer about why this particular un-uniformity is bad.  I don't think
the issue for new contributors is persuasive enough, as new contributors
can mostly ignore source packages.  It's not like, e.g., debhelper
vs. cdbs.  I haven't yet seen an argument that the lack of uniformity is
doing anyone's work any harm comparable to the harm done to things like
what Ian and Sam want to do.

> Exploring different technical approaches does have value as well, but I
> think we've had sufficient time to consider the various advantages and
> disadvantages of various source packages formats. On a whole, it seems
> to me that that the number of packages benefiting from 1.0-with-diff is
> relatively small.

I agree, it's not about the benefits of the source format, we do indeed
understand all the trade-offs by now.  It's that certain ideas and
workflows *which are not really about source packages* are made
inconvenient or impossible if we remove this option.  In other words, it
needs to be replaced before it can be deprecated.

> What would you think about adding an alternative option 4?
>
> 4b. We believe that there are indeed circumstances in which
> 1.0-with-diff is the best choice for a particular source package.
> Given that the number of packages for which this is relevant is
> fairly small, we recommend discontinuing use of 1.0-with-diff to
> gain more uniformity.

Thanks for coming up with the text.  I'd say that as uniformity is not
good in itself, it would be good to have more concrete reasons for
wanting uniformity in this case documented in this bug (not necessarily
in the resolution text) before we add it to the ballot.

-- 
Sean Whitton

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Helmut Grohne]

Hallo,

On Tue, May 10, 2022 at 05:29:57PM -0700, Sean Whitton wrote:
> DRAFT
> 
> Using its powers under constitution 6.1.5, the Technical Committee
> issues the following advice:

I've given this some thought and feel uneasy about one item.

>   4. We believe that there are indeed circumstances in which
>  1.0-with-diff is the best choice for a particular source package,
>  including, but not limited to, git-first packaging workflows.

While I can agree with this item on a technical level, I think there is
more to it than that and I am wondering whether it sends the "right"
message.

Sometimes, things we do are technically possible and fill a niche well.
Yet, we decide that it is no longer reasonable to continue supporting
them and remove their support despite the feature being useful to some.
Quite clearly, there is a trade-off involved here. Continuing to support
1.0-with-diff comes with a cost that reduces uniformity inside the
archive. Evidently, this is what motivated Lucas to file the MBF
initially. My experience is that lack of uniformity is a significant
barrier to prospective contributors to Debian.

Exploring different technical approaches does have value as well, but I
think we've had sufficient time to consider the various advantages and
disadvantages of various source packages formats. On a whole, it seems
to me that that the number of packages benefiting from 1.0-with-diff is
relatively small.

What would you think about adding an alternative option 4?

4b. We believe that there are indeed circumstances in which
1.0-with-diff is the best choice for a particular source package.
Given that the number of packages for which this is relevant is
fairly small, we recommend discontinuing use of 1.0-with-diff to
gain more uniformity.

Helmut

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Ian Jackson]

Lucas Nussbaum writes ("Re: Bug#1007717: Draft resolution for "Native source 
package format with non-native version""):
> On 11/05/22 at 17:29 +0100, Ian Jackson wrote:
> > But I think that might not meet ftpmaster's review needs.  AIUI
> > ftpmaster like to review the diff qua diff, and a tarball isn't so
> > straightforward.  I had a similar idea to use an rsync batchfile as
> > the delta, which foundered on the same issue.
> 
> Note that it's not worse than using native formats, where ftpmasters get
> a single tarball anyway...

Indeed.  But I don't necessarily expect to be able to predict what
ftpmaster (or, for that matter, the dpkg maintainers) will like.

Thanks for exploring the design space!  What would be really good
would be to get agreement in principle from the key stakeholders...

Ian.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Lucas Nussbaum]

On 11/05/22 at 17:29 +0100, Ian Jackson wrote:
> Lucas Nussbaum writes ("Re: Bug#1007717: Draft resolution for "Native source 
> package format with non-native version""):
> > Out of curiosity, if 3.0 (native) supported multiple tarballs, wouldn't
> > it be a good solution?
> 
> Oh, I hadn't thought of that.
> 
> > The main limitation I see is that it would not allow to represent
> > efficiently small changes to large text files (which a git-based diff
> > would allow).
> 
> That may not be important.  My feeling is that it wouldn't be.
> 
> However, I think there are some other difficulties with this idea.
> 
> *Deletion* of files *is* important.  Something would have to be done
> to support that.  (Tarball repacking is an abominable workflow which
> we should only do when we must.)
> 
> It is important that packing and unpacking these things works roughly
> the same way that things work right now for the diffish formats.  Ie,
> for a package with two tarballs, the first tarball would have to omit
> the Debian revision from its filename (so that it needn't be
> re-uploaded), and the second tarball would have to *contain* it.
> dpkg-source -b would have to calculate the appropriate second tarball
> as a diff from the first.  (GNU tar has an incremental option that
> could perhaps be used.)
> 
> I think, having followed this line of throught, the result looks quite
> like a "3.0 (diff)" only the diff is in the form of an incremental
> tarball rather than a textual patch file.
> 
> This could definitely work.

FWIW, GNU tar's incremental mode supports file deletions.

Example:
-->8
# create a first archive
mkdir t
echo foo > t/f1
echo bar > t/f2
echo baz > t/f3
tar --listed-increment=snapshot-file -czvvvf archive.tgz t
# remove a file, create an incremental archive
rm t/f2
tar --listed-increment=snapshot-file -czvvvf archive.1.tgz t

# remove everything
rm -rf snapshot-file t
# extract incremental archives
tar Gvvvxf archive.tgz
tar Gvvvxf archive.1.tgz

# listing of archive.1.tgz
tar Gvvvtf archive.1.tgz
-->8

This is implemented using the GNU-specific D entry type. According to
https://www.freebsd.org/cgi/man.cgi?query=tar&apropos=0&sektion=5&format=html :

  D  This indicates a directory entry. Unlike the POSIX-stan-
 dard "5" typeflag, the header is followed by data records
 listing the names of files in this directory.  Each name
 is preceded by an ASCII "Y" if the file is stored in this
 archive or "N" if the file is not stored in this archive.
 Each name is terminated with a null, and an extra null
 marks the end of the name list.  The purpose of this en-
 try is to support incremental backups; a program restor-
 ing from such an archive may wish to delete files on disk
 that did not exist in the directory when the archive was
 made.

 Note that the "D" typeflag specifically violates POSIX,
 which requires that unrecognized typeflags be restored as
 normal files.  In this case, restoring the "D" entry as a
 file could interfere with subsequent creation of the
 like-named directory.

> But I think that might not meet ftpmaster's review needs.  AIUI
> ftpmaster like to review the diff qua diff, and a tarball isn't so
> straightforward.  I had a similar idea to use an rsync batchfile as
> the delta, which foundered on the same issue.

Note that it's not worse than using native formats, where ftpmasters get
a single tarball anyway...

Lucas

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Ian Jackson]

Lucas Nussbaum writes ("Re: Bug#1007717: Draft resolution for "Native source 
package format with non-native version""):
> Out of curiosity, if 3.0 (native) supported multiple tarballs, wouldn't
> it be a good solution?

Oh, I hadn't thought of that.

> The main limitation I see is that it would not allow to represent
> efficiently small changes to large text files (which a git-based diff
> would allow).

That may not be important.  My feeling is that it wouldn't be.

However, I think there are some other difficulties with this idea.

*Deletion* of files *is* important.  Something would have to be done
to support that.  (Tarball repacking is an abominable workflow which
we should only do when we must.)

It is important that packing and unpacking these things works roughly
the same way that things work right now for the diffish formats.  Ie,
for a package with two tarballs, the first tarball would have to omit
the Debian revision from its filename (so that it needn't be
re-uploaded), and the second tarball would have to *contain* it.
dpkg-source -b would have to calculate the appropriate second tarball
as a diff from the first.  (GNU tar has an incremental option that
could perhaps be used.)

I think, having followed this line of throught, the result looks quite
like a "3.0 (diff)" only the diff is in the form of an incremental
tarball rather than a textual patch file.

This could definitely work.

But I think that might not meet ftpmaster's review needs.  AIUI
ftpmaster like to review the diff qua diff, and a tarball isn't so
straightforward.  I had a similar idea to use an rsync batchfile as
the delta, which foundered on the same issue.

And I'm not sure that it will find better favour with dpkg upstream
than my "3.0 (git-diff)".

> I'm asking because if 3.0 (native) gets more generic by allowing
> non-native revisions, it might be an easier sell to introduce
> multi-tarballs support, than to introduce a completely different source
> format.

Mmmm.  I think there are many possibilities here which would suffice.
Right now, though, it's a bit hard to make progress without feedback
on what general direction would be most well received.

Ian.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Lucas Nussbaum]

Thanks for your answer.

On 11/05/22 at 12:38 +0100, Ian Jackson wrote:
> I would love for there to be something like 3.0-with-git-diff.  Indeed,
> I filed this wishlist bug to ask if contribution would be welcome:
>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1007781
> but have not had any response.
> 
> The code in dpkg-source for 1.0-with-diff is quite crusty particularly
> in respect of the implied behaviour wrt scanning your ".." for stuff.
> The *format* of 1.0-with-diff is quite reasonable, but it lacks
> support more kinds of delta.  That could be done as an extension to
> 1.0-with-diff, but I doubt that would be a popular direction.

Out of curiosity, if 3.0 (native) supported multiple tarballs, wouldn't
it be a good solution?

The main limitation I see is that it would not allow to represent
efficiently small changes to large text files (which a git-based diff
would allow).

I'm asking because if 3.0 (native) gets more generic by allowing
non-native revisions, it might be an easier sell to introduce
multi-tarballs support, than to introduce a completely different source
format.

Lucas

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Ian Jackson]

Lucas Nussbaum writes ("Re: Bug#1007717: Draft resolution for "Native source 
package format with non-native version""):
> If it was possible to use 3.0 (native) with non-native revisions, would
> there be remaining circumstances where 1.0-with-diff is the best choice?

Yes, unfortunately.

If you have a package whose orig source code is large, and the delta
is representable with 1.0-with-diff (which is quite likely), then you
don't want to be reuploading the whole tarball each time.  That's
wasteful of everyone's bandwidth and disk space.

So you want a representation that offers delta compression.  That is
offered by the non-native formats.  The non-native formats supported
by the archive are 1.0-with-diff and "3.0 (quilt)".  The latter has
the problem with debian/patches/ living inside the source tree, which
is quite undesirable especially for "git-first workflows" (as the
draft text puts it).

> If not, maybe the fact that this is the blocking issue should be made
> explicit in (4)?

   4. We believe that there are indeed circumstances in which
  1.0-with-diff is the best choice for a particular source package,
  including, but not limited to, git-first packaging workflows.
 +This is because there is currently no other source format which avoids
 +reuploading the whole source (including upstream) for each upload,
 +and also avoids having to maintain debian/patches/ inside the
 +package tree.

Or something.

> That would be a way to state: "either dpkg maints refuses to support 3.0
> (native) with non-native revs, and 1.0-with-diff must not be considered
> deprecated; or dpkg maints supports it, and it might be possible to
> deprecate 1.0".

I would love for there to be something like 3.0-with-git-diff.  Indeed,
I filed this wishlist bug to ask if contribution would be welcome:
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1007781
but have not had any response.

The code in dpkg-source for 1.0-with-diff is quite crusty particularly
in respect of the implied behaviour wrt scanning your ".." for stuff.
The *format* of 1.0-with-diff is quite reasonable, but it lacks
support more kinds of delta.  That could be done as an extension to
1.0-with-diff, but I doubt that would be a popular direction.

Ian.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Lucas Nussbaum]

On 10/05/22 at 17:29 -0700, Sean Whitton wrote:
> Hello,
> 
> At today's ctte meeting we considered whether we can start a vote on
> this, but two committee members were missing, and someone else at the
> meeting reported that they hadn't yet been able to spend enough time
> thinking through the issue to be ready to vote.
> 
> We came up with the following plan.  Below I've drafted a ballot.  Once
> each of those three individuals has let me know that they've had a
> chance to catch up, I'll start a vote.  The hope is that this can happen
> well in advance of our next meeting.  So, ctte members, if I don't
> already know that you're caught up, please write to me once you are.
> 
> ~
> 
> DRAFT
> 
> Using its powers under constitution 6.1.5, the Technical Committee
> issues the following advice:
> 
>   1. It is not a bug of any severity for a package with a non-native
>  version number to use a native source package format.
> 
>   2. Thus, we think that dpkg shouldn't issue warnings, or otherwise
>  complain, when a non-native version number is used w/ 3.0 (native).
> 
>   3. We suggest that the wontfix tag on #737634 be reconsidered.
> 
>   4. We believe that there are indeed circumstances in which
>  1.0-with-diff is the best choice for a particular source package,
>  including, but not limited to, git-first packaging workflows.
> 
>   5. We decline to comment on the recent source package format MBF.
> 
>  Option A -- issue items 1--5
> 
>  Option B -- issue items 1, 2, 3 and 5, but not 4
> 
>  Option N -- none of the above.
> 
> END DRAFT

Hi,

If it was possible to use 3.0 (native) with non-native revisions, would
there be remaining circumstances where 1.0-with-diff is the best choice?
If not, maybe the fact that this is the blocking issue should be made
explicit in (4)?

That would be a way to state: "either dpkg maints refuses to support 3.0
(native) with non-native revs, and 1.0-with-diff must not be considered
deprecated; or dpkg maints supports it, and it might be possible to
deprecate 1.0".

Lucas

Bug#1007717: Draft resolution for "Native source package format with non-native version"

[Sean Whitton]

Hello,

At today's ctte meeting we considered whether we can start a vote on
this, but two committee members were missing, and someone else at the
meeting reported that they hadn't yet been able to spend enough time
thinking through the issue to be ready to vote.

We came up with the following plan.  Below I've drafted a ballot.  Once
each of those three individuals has let me know that they've had a
chance to catch up, I'll start a vote.  The hope is that this can happen
well in advance of our next meeting.  So, ctte members, if I don't
already know that you're caught up, please write to me once you are.

~

DRAFT

Using its powers under constitution 6.1.5, the Technical Committee
issues the following advice:

  1. It is not a bug of any severity for a package with a non-native
 version number to use a native source package format.

  2. Thus, we think that dpkg shouldn't issue warnings, or otherwise
 complain, when a non-native version number is used w/ 3.0 (native).

  3. We suggest that the wontfix tag on #737634 be reconsidered.

  4. We believe that there are indeed circumstances in which
 1.0-with-diff is the best choice for a particular source package,
 including, but not limited to, git-first packaging workflows.

  5. We decline to comment on the recent source package format MBF.

 Option A -- issue items 1--5

 Option B -- issue items 1, 2, 3 and 5, but not 4

 Option N -- none of the above.

END DRAFT

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1007717: Native source package format with non-native version

[Sean Whitton]

Hello,

On Tue 29 Mar 2022 at 09:06PM +02, Lucas Nussbaum wrote:

> On 28/03/22 at 16:21 -0700, Sean Whitton wrote:
>>
>> I don't think it's the preferred method.  I believe most of the project
>> sees git histories are the preferred tool for achieving the goal you
>> state.
>>
>> If we had only source packages for this purpose, then yes, 3.0 (quilt)
>> plus DEP-3 would be preferred.  But we have git, too, and indeed dgit.
>> Repositories for packages contain both upstream history and Debian
>> packaging history, and we have powerful git subcommands for extracting
>> information.  In many cases there is a good argument to be made that the
>> git history is part of the preferred form of modification.
>
> I think there are three different use cases to consider:

I agree that we should consider them separately.

>   A. preferred form for regular contributions to the package (typically
>   by the package maintainer)
>
>   B. preferred form for occasional contributions to the package (typically
>   by an NMUer, or by the security team)
>
>   C. preferred form for reviewing the packaging and Debian-specific
>   changes.
>
>
> I fully agree that the git repository is the preferred form for A.
> However, for B and C, I think that our lingua franca is the source
> package, and thus a source package that doesn't make it hard to
> understand things such as Debian-specific patches. Of course that
> could change if we were able to standardize on a git workflow (or
> a small set of git workflows), but I don't see this happenning anytime
> soon.

What you get from 'dgit clone' is designed to serve (B) and (C) well,
and somewhat sidesteps precisely the problems created by our having
multiple git workflows.

Please consider trying out dgit-user(7) and/or dgit-nmu-simple(7) next
time you need to engage in (B) or (C).  It's really very nice :)

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1007717: Native source package format with non-native version

[Bastian Blank]

On Tue, Mar 15, 2022 at 04:29:17PM +, Ian Jackson wrote:
>  1. Declare explicitly that there is nothing wrong with a package with
> a native format, but a non-native version number.
>  2. Request that the dpkg maintainer relax the restriction which
> prevents the use of 3.0 native with Debian revision.

One additional point:
We currently have packages in the archive, which must be "3.0 (native)",
but are generated from other packages: all the secure boot signed
packages.

Example source package:
| Package: linux-signed-amd64
| Version: 5.17~rc8+1~exp1

is generated from binary package:
| Package: linux-image-amd64-signed-template
| Version: 5.17~rc8-1~exp1

Generating the source package needs to munge the version, just to
appease that version restriction.  This also means, adding dependencies
on versions is difficult, as both versions sort differently.  And bugs
are assigned to wrong versions, so version tracking does not work.

Regards,
Bastian

-- 
Kirk to Enterprise -- beam down yeoman Rand and a six-pack.

Bug#1007717: Native source package format with non-native version

[Lucas Nussbaum]

Hi Sean,

On 28/03/22 at 16:21 -0700, Sean Whitton wrote:
> Hello Lucas,
> 
> Many thanks for providing this summary of your position.  Let me just
> note a point of disagreement:
> 
> On Tue 15 Mar 2022 at 10:06PM +01, Lucas Nussbaum wrote:
> 
> > A point that I find important is the following: as a project, I think
> > that we care about facilitating the review of changes we make to
> > upstream source.
> 
> Certainly we do.
> 
> > I think that the preferred method (and widely accepted method) for
> > that is currently to use the 3.0 (quilt) format with DEP-3-documented
> > patches, on top of a tarball that contains the pristine upstream
> > source.
> >
> > The git packaging workflows that generate source packages using either
> > 1.0 native packages, or 1.0 non-native packages without patches, make it
> > harder to identify and review those changes, as they require browsing
> > the git repository, which sometimes is not properly documented using
> > Vcs-*.
> 
> I don't think it's the preferred method.  I believe most of the project
> sees git histories are the preferred tool for achieving the goal you
> state.
> 
> If we had only source packages for this purpose, then yes, 3.0 (quilt)
> plus DEP-3 would be preferred.  But we have git, too, and indeed dgit.
> Repositories for packages contain both upstream history and Debian
> packaging history, and we have powerful git subcommands for extracting
> information.  In many cases there is a good argument to be made that the
> git history is part of the preferred form of modification.

I think there are three different use cases to consider:

  A. preferred form for regular contributions to the package (typically
  by the package maintainer)

  B. preferred form for occasional contributions to the package (typically
  by an NMUer, or by the security team)

  C. preferred form for reviewing the packaging and Debian-specific
  changes.


I fully agree that the git repository is the preferred form for A.
However, for B and C, I think that our lingua franca is the source
package, and thus a source package that doesn't make it hard to
understand things such as Debian-specific patches. Of course that
could change if we were able to standardize on a git workflow (or
a small set of git workflows), but I don't see this happenning anytime
soon.

Lucas

Bug#1007717: Native source package format with non-native version

[Sean Whitton]

Hello Lucas,

Many thanks for providing this summary of your position.  Let me just
note a point of disagreement:

On Tue 15 Mar 2022 at 10:06PM +01, Lucas Nussbaum wrote:

> A point that I find important is the following: as a project, I think
> that we care about facilitating the review of changes we make to
> upstream source.

Certainly we do.

> I think that the preferred method (and widely accepted method) for
> that is currently to use the 3.0 (quilt) format with DEP-3-documented
> patches, on top of a tarball that contains the pristine upstream
> source.
>
> The git packaging workflows that generate source packages using either
> 1.0 native packages, or 1.0 non-native packages without patches, make it
> harder to identify and review those changes, as they require browsing
> the git repository, which sometimes is not properly documented using
> Vcs-*.

I don't think it's the preferred method.  I believe most of the project
sees git histories are the preferred tool for achieving the goal you
state.

If we had only source packages for this purpose, then yes, 3.0 (quilt)
plus DEP-3 would be preferred.  But we have git, too, and indeed dgit.
Repositories for packages contain both upstream history and Debian
packaging history, and we have powerful git subcommands for extracting
information.  In many cases there is a good argument to be made that the
git history is part of the preferred form of modification.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#1007717: Native source package format with non-native version

[Matthew Vernon]

On 17/03/2022 17:52, Russ Allbery wrote:

Helmut Grohne  writes:


Do you think it would be impossible to move forward on this matter in a
consensus-based way?


I don't know.  I have some reasons to be dubious, but it's possible that
I'm being excessively pessimistic.


I'm inclined to agree with Russ here; my impression is there are one or 
two long-standing areas of disagreement here, and that consensus hasn't 
been arrived at.



In the spirit of consensus: Do you agree that retrying this in a
consensus-based way is still possible?


If the relevant people required to implement a decision are willing to
tackle it that way, I am certainly willing to participate from the Policy
side.


For the avoidance of doubt, if that is the case, I am not going to 
suggest the TC gets in the way!


Regards,

Matthew

Bug#1007717: Native source package format with non-native version

[Lucas Nussbaum]

Hi Ian,

On 15/03/22 at 16:29 +, Ian Jackson wrote:
> Part I - belss continued use of 1.0 native format, for now at least:
> 
>  1. Declare explicitly that there is nothing wrong with a package with
> a native format, but a non-native version number.
> 
>  2. Request that the dpkg maintainer relax the restriction which
> prevents the use of 3.0 native with Debian revision.
> 
>  3. Consequently, declare that the recent MBF on this topic ought not
> to have been filed against packages where simply changing the
> source format does not currently work.  That would include at
> least 1.0 native packages with Debian revisions.
> 
> Part II - bless continued use of 1.0-with-diff, for now at least:
> 
>  4. Declare that sometimes the use of 1.0-with-diff can be the best
> tradeoff between different considerations.  In particular,
> because 1.0 is the only format which botH:
>  (a) Optimises bandwidth and storage by reusing the upstream 
>  data when it hasn't changed.
>  (b) Avoids polluting the working tree (package source code)
>  with [patches], which cause trouble especially with
>git-based workflows.
> 
>  5. Consequently, declare that the recent MBF on this topic ought not
> to have been filed against 1.0 with diff packages, at least
> without some further filter.

I did the MBF mentioned in (5) (about suggesting the move from the 1.0
format to one of the 3.0 formats), and agreed to pause those efforts in
.

However, it might be worth clarifying if the MBF in (3) is mine, or
Guillem's one (with usertag dpkg-mismatch-source-vs-version-format and
user debian-d...@lists.debian.org) about "Mismatched source format vs
source version" (an example bug is #1007088). I think it's mine, but I'm
not sure. It might also be both.

Lucas

Bug#1007717: Native source package format with non-native version

[Felix Lechner]

Hi,

On Thu, Mar 17, 2022 at 11:57 AM Russ Allbery  wrote:
>
> source package format

While everyone is receptive to new labels, I prefer "upload format" or
"archive format". Either one helps us to distinguish the intermediate
product from any workflow objects a maintainer may have.

> single tarball as a source package format

It is also not necessary to "define" unpatched sources in the archive
by how many tarballs they have, or any tarballs at all.

In fact, I wrote a manifest utility that could one day replace tarball
signatures with per-file hashes. The latter remain useful even when
sources are repackaged, because manifests can be cryptographically
daisy-chained in a "chain of custody." Of course, they would be more
often used for patched upload formats.

Kind regards
Felix Lechner

Bug#1007717: Native source package format with non-native version

[Russ Allbery]

Sam Hartman  writes:
>> "Russ" == Russ Allbery  writes:

> Russ> Switching terminology to completely leave behind the terms
> Russ> with ambiguous meanings isn't a bad idea, but if so we really
> Russ> need a term that captures "is a packaging of an upstream
> Russ> software package with a separate existence" versus "exists
> Russ> solely as a Debian package."  "with-revision" or
> Russ> "without-revision" doesn't feel to me like it does this.
> Russ> Native and non-native do, which is why I was sticking with
> Russ> them, but maybe we can come up with some other equally-good
> Russ> terminology.

> Why do we need that distinction?

I think we need some way to talk about the various things that change
about a package if it is a packaging of an upstream package.  Examples
include a debian/watch file, an upstream metadata file, an upstream
signing key (still useful with single-tarball formats if upstream uses
signed tags, etc.), and provenance information in debian/copyright.

Those things are not tied to the source package format.  We want
provenance information for the upstream source even if you're using a
single tarball as a source package format, but we don't need it if there
is no upstream.

> Looking at current policy, 5.6.12 talks about having a debian revision
> or not having a debian revision.

It seems less useful to me to frame this solely in terms of the version
number format, rather than having the version number format indicate
whether this is a packaging of an upstream source with an independent
existence.  That would imply that if you just omit the debian revision,
you don't need to record the provenance of the upstream source, which
doesn't seem correct.

-- 
Russ Allbery (r...@debian.org)  

Bug#1007717: Native source package format with non-native version

[Sam Hartman]

> "Russ" == Russ Allbery  writes:

Russ> Switching terminology to completely leave behind the terms
Russ> with ambiguous meanings isn't a bad idea, but if so we really
Russ> need a term that captures "is a packaging of an upstream
Russ> software package with a separate existence" versus "exists
Russ> solely as a Debian package."  "with-revision" or
Russ> "without-revision" doesn't feel to me like it does this.
Russ> Native and non-native do, which is why I was sticking with
Russ> them, but maybe we can come up with some other equally-good
Russ> terminology.

Why do we need that distinction?

Looking at current policy, 5.6.12 talks about having a debian revision
or not having a debian revision.

Other parts of policy talk about  what parts of the source package there
are.

Why do we need more than these two distinctions.

I think that current policy has mostly left behind the work native
(although their are a few uses still).
My suspicion is that avoiding native allowed us to get a broader
consensus in the policy process.

Why isn't what we have good enough?

Bug#1007717: Native source package format with non-native version

[Russ Allbery]

Helmut Grohne  writes:

> Do you think it would be impossible to move forward on this matter in a
> consensus-based way?

I don't know.  I have some reasons to be dubious, but it's possible that
I'm being excessively pessimistic.

> Yes, please. Though as is evidenced in the replies to your mail, I would
> try to avoid "native" and "non-native" as much as possible given the
> existing confusion. I suggest using something like with-revision vs
> without-revision and single-tarball (from your mail) vs
> patches-separated to transport the concepts.

Switching terminology to completely leave behind the terms with ambiguous
meanings isn't a bad idea, but if so we really need a term that captures
"is a packaging of an upstream software package with a separate existence"
versus "exists solely as a Debian package."  "with-revision" or
"without-revision" doesn't feel to me like it does this.  Native and
non-native do, which is why I was sticking with them, but maybe we can
come up with some other equally-good terminology.

> More and more, it seems to me that we are looking into design work as
> opposed to picking an existing option.

*I* was doing design work, for sure.  But I'm not a member of the TC.  :)
The point was to offer you a design to consider as part of submitting the
request to the TC.

> In the spirit of consensus: Do you agree that retrying this in a
> consensus-based way is still possible?

If the relevant people required to implement a decision are willing to
tackle it that way, I am certainly willing to participate from the Policy
side.

-- 
Russ Allbery (r...@debian.org)  

Bug#1007717: Native source package format with non-native version

[Sam Hartman]

> "Helmut" == Helmut Grohne  writes:

Helmut> Hi Russ,
Helmut> On Tue, Mar 15, 2022 at 09:22:09PM -0700, Russ Allbery wrote:
>> > Specifically, I'd like to ask the TC to come up with policy on
>> native > packages and debian revisions using its power under
>> 6.1.1.
>> 
>> As a Policy Editor, I support this request.

Helmut> As a TC member I admit disliking this. While there is
Helmut> disagreement on how things are supposed to work, the views
Helmut> don't seem to be that far apart. Urgency is effectively
Helmut> removed by Lucas agreeing to pause further deprecation
Helmut> work. Do you think it would be impossible to move forward on
Helmut> this matter in a consensus-based way?

I think this is like usrmerge.
I actually think there is a rough consensus, but there are one or two
parties who find that consensus unacceptable and who are in a position
to block the consensus.

As an example, the text in policy 5.6.12 has been significantly revised
since Russ and I started discussing this issue in 2014.
It's possible that policy could be improved, but honestly I think that
policy is already consistent with the hypothesis I presented to the TC.

I also already tried to seek consensus where this is possible as DPL,
and included a pointer to my consensus call on the issues where there
was consensus in the material I already presented to the TC.

This is a case where people have slowly been building consensus since
2014.  One of the key stakeholders has not effectively participated
constructively, and so some things haven't been possible.

We're asking the TC to do the last mile work--review the existing
consensus-forming discussions, figure out where we are, and wrap it up
just like you did with usrmerge.

I've even tried to give you the key discussions that happened on
debian-devel.
I suspect one of the policy editors could go pull that together for
discussions on debian-policy, but if not, I'd be happy to do that work
if requested.

So, no, I do think this is ready for the TC to finish things up now.

--Sam


signature.asc
Description: PGP signature

Bug#1007717: Native source package format with non-native version

[Lucas Nussbaum]

On 16/03/22 at 23:54 +, Wookey wrote:
> On 2022-03-16 15:29 +, Ian Jackson wrote:
> > In practice, the vast majority of packages are maintained in git on
> > salsa.  The maintainers use those git repositories as the PFM.
> 
> > but almost everyone is already treating git as primary.
> 
> Is this definitely true? For example: I know I'm not doing this. I did
> try, and I do have some git repos on salsa, but I've mostly given up
> with it all and stuck with uscan and tarballs and quilt (and my trusty
> 'packages' directory). It's much easier for me. (The salsa repos that
> exist for my packages are not canonical and often stale).
> 
> I'm sure Ian is right that there is a trend towards git from tarballs
> and dscs, but I just question whether we know it is 'the vast
> majority'? Are there really now very few maintainers using the
> 'classic tooling'? How do we know?

Hi,

If you look at https://trends.debian.net/#version-control-system, I
think it's fair to say that the vast majority of packages are now
maintained in git.

And if you look at the status of those repositories according to
vswatch, most of them are actively used:

udd=> select status, count(distinct source) from vcswatch group by status order 
by 2;
 status  | count 
-+---
 UNREL   |   119
 OLD |  1218
 ERROR   |  1328
 COMMITS |  3620
 NEW |  7127
 OK  | 17873

(the last three lines being 'normal' states.)

Lucas

Bug#1007717: Native source package format with non-native version

[Helmut Grohne]

Hi Russ,

On Tue, Mar 15, 2022 at 09:22:09PM -0700, Russ Allbery wrote:
> > Specifically, I'd like to ask the TC to come up with policy on native
> > packages and debian revisions using its power under 6.1.1.
> 
> As a Policy Editor, I support this request.

As a TC member I admit disliking this. While there is disagreement on
how things are supposed to work, the views don't seem to be that far
apart. Urgency is effectively removed by Lucas agreeing to pause further
deprecation work. Do you think it would be impossible to move forward on
this matter in a consensus-based way?

> I've been involved in a lot of these discussions over the years, and the
> tentative conclusion that I've come to is that we have unfortunately mixed
> several different concepts in how we talk about source packages.  This has
> not *technically* reduced the flexibility of the packaging format because
> there are various workarounds, but it's *conceptually* reduced the
> flexibility in a way that makes those workarounds look conceptually
> incorrect.  As a result, we have constant repetitions of similar arguments
> stemming from well-meaning attempts to clean up the apparent
> inconsistencies (including some I have started).

While I did point out the conflation of matters in my initial reply to
Ian, I didn't put it as clearly as you did. Thank you. In the later
replies to your mail, we see that even trying to disentangle this is
difficult terminology-wise.

> There are really two separate concepts in play here:

I am wondering whether maybe one of you could work on a consensus
seeking process about these matters. I believe that resolving this using
consensus is far better than having 8 semi-random people decide upon a
policy. The consensus-based approach does not seem impossible to me at
this stage.

> What I would propose is to separate these concepts cleanly so that we can
> talk about them in a clearer way.  We should define native and non-native
> packages in terms of version numbers, and allow both native and non-native
> packages to use single-tarball source package formats.  We may want to
> steer people away from using the single-tarball source package format for
> the *normal* case of packaging upstream software, but I think it's well
> within the sorts of trade-off decisions that packagers already make and
> there are cases where a single tarball source format makes sense.

Yes, please. Though as is evidenced in the replies to your mail, I would
try to avoid "native" and "non-native" as much as possible given the
existing confusion. I suggest using something like with-revision vs
without-revision and single-tarball (from your mail) vs
patches-separated to transport the concepts. Of course "3.0 (native)" as
the content of debian/source/format cannot be avoided, but we can
describe its current and possibly desired properties in a less confusing
way.

> The name of the 3.0 (native) source package format is unfortunate in this
> context since it strongly implies that only native packages will use that
> source format.  The most formally correct thing to do would probably be to
> introduce a new 3.0 (single-tarball) source format and use 1.0 until then,
> but of course introducing a new source package format is not a fast
> process.  A more expedient thing to do may be to allow use of 3.0 (native)
> with non-native packages, since it's identical to 3.0 (single-tarball)
> except for the name, but it does leave a long-term confusing naming scheme
> in place.  Guillem may well have other and better suggestions from the
> dpkg perspective; I haven't thought all that much about a transition plan.

More and more, it seems to me that we are looking into design work as
opposed to picking an existing option. I think it is preferable to do
that with "normal" Debian processes without involving the TC, because
the latter always comes with constitutional powers attached (used or
unused). If history has taught us anything, involving the TC always
comes with escalation. We may still reach a point where it is clear that
consensus is not possible, but I don't think we're there yet.

In the spirit of consensus: Do you agree that retrying this in a
consensus-based way is still possible?

Helmut

Bug#1007717: Native source package format with non-native version

[Wookey]

On 2022-03-16 15:29 +, Ian Jackson wrote:
> In practice, the vast majority of packages are maintained in git on
> salsa.  The maintainers use those git repositories as the PFM.

> but almost everyone is already treating git as primary.

Is this definitely true? For example: I know I'm not doing this. I did
try, and I do have some git repos on salsa, but I've mostly given up
with it all and stuck with uscan and tarballs and quilt (and my trusty
'packages' directory). It's much easier for me. (The salsa repos that
exist for my packages are not canonical and often stale).

I'm sure Ian is right that there is a trend towards git from tarballs
and dscs, but I just question whether we know it is 'the vast
majority'? Are there really now very few maintainers using the
'classic tooling'? How do we know?

Wookey
-- 
Principal hats:  Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: PGP signature

Bug#1007717: Native source package format with non-native version

[Russ Allbery]

Felix Lechner  writes:
> On Tue, Mar 15, 2022 at 9:33 PM Russ Allbery  wrote:

>> We should define native and non-native packages in terms of version
>> numbers, and allow both native and non-native packages to use
>> single-tarball source package formats.

> I co-maintaintain several Debian-internal tools, and that description is
> backwards. "Native" sources are characterized by their lack of Debian
> patches.

This is exactly the semantic confusion that I believe is wrong and we
should undo.  "Native" vs. "non-native" should be a property of the
relationship between the package and a separate upstream release, which is
represented in the version.  That should be decoupled from the source
package format.

The point that Ian and Sam are raising is that a single tarball is a good
way of representing the Debian package source in several non-native cases
where we are still packaging a piece of software with an independent
upstream existence, and where having a version with a Debian revision is
still semantically correct.

> On that note, the term "native" is also not great. The words "patched"
> and "unpatched" describe the relationship between sources in the archive
> and their respective upstreams more accurately.

I think that would add even more confusion.  It's very common for
non-native packages to switch between patched and unpatched from upload to
upload, depending on whether Debian has to carry additional patches,
without any change in their version number format or in the source package
format.  This is correct and should continue to be supported.

-- 
Russ Allbery (r...@debian.org)  

Bug#1007717: Native source package format with non-native version

[Ian Jackson]

Helmut Grohne writes ("Re: Bug#1007717: Native source package format with 
non-native version"):
> Beyond the content of your request, I have a meta-question. Do you see a
> particular urgency with the processing of your request? What is - in
> your opinion - a reasonable time for resolving this? Of course, if Lucas
> et al are to proceed with their deprecation work, urgency may arise in
> your view, but let us assume that we can ask them to pause their
> efforts for a while.

As you surmise, the only urgency I feel is due to these deprecation
efforts (some of which take other forms).

> Lucas, please pause further work on deprecating the 1.0 format in order
> to give time to settle the dispute at hand. This implies not sending
> further bugs about it. On the other hand, I think closing all of the
> existing ones would not do any good at this time either.

It is a shame that these bugs were filed and a lot of maintainers
(including at least one of my sponsees) now find themselves in a
confusing position.  Perhaps it would be worth sending an agreed
holding message to these bugs, and/or blocking them by this one.

> Can I ask you to step back a bit and help us figure out what you really
> want to achieve?
> 
> Given your other messages on the matter, I suppose that you want to be
> able to use a source package format that can faithfully represent any
> valid source tree. In particular, a source tree with a Debian revision
> should be representable. You care less about being able to represent
> differences to a tarball in some cases as long as the changed tree as a
> whole is representable. This includes changes to permissions, symbolic
> links and binary files. Which source format is used is not important to
> you as long as it is able to meet these requirements.

Yes, that is by and large my SNPS (solution neutral problem statement)
for the ideal situation.

But, note that "faithfully" implies not including a copy of the Debian
delta in the form of patches within the unpacked source tree (as "3.0
(quilt)" does), since those patches must then be kept up to date.

And, I want a format that offers the bandwidth and disk space
optimisation of not re-uploading large files when only small changes
are made.  This is necessary for large packages.  The various
with-diffs/with-patches formats achieve this, but compromise on other
aspects.

> > Part II - bless continued use of 1.0-with-diff, for now at least:
> > 
> >  4. Declare that sometimes the use of 1.0-with-diff can be the best
> > tradeoff between different considerations.  In particular,
> > because 1.0 is the only format which botH:
> >  (a) Optimises bandwidth and storage by reusing the upstream 
> >  data when it hasn't changed.
> >  (b) Avoids polluting the working tree (package source code)
> >  with [patches], which cause trouble especially with
> >  git-based workflows.
> 
> I note that the 1.0-with-diff format does not meet my perception of your
> earlier requirement: Being able to represent any source tree.

You are correct.  But there is not currently any format that meets all
of my requirements.

In particular, 1.0-with-diff achieves the bandwdith/storage
optimisation, at the cost of representational capability.
Looked at this way "3.0 (quilt)" is mixed - it has some additional
representational capabilities, but reduced fidelity.

>  I also note that it is possible to choose a different conversion
> mechanism than dpkg-source -b between working tree and .dsc that
> does not incur your reported pollution. In particular, you do have
> experience in implementing such conversions as is evidenced in dgit
> and such conversions do exist.

Um.  The conversions done by dgit exist to *keep the pollution up to
date*.  And they are really quite inconvenient.  dgit must make
commits on your branch, or maintain multiple branches.  There is a
substantial amount of code there, dealing with a large variety of
different situations.

> Your request also touches the question of the preferred form for
> modification. That used to be the .dsc, but for the way you use the
> archive, it merely is an export for a preferred form for modification
> that resides elsewhere. I do not think we have consensus for this view
> either.

In practice, the vast majority of packages are maintained in git on
salsa.  The maintainers use those git repositories as the PFM.
The git repositories contain much information which is not in the
source packages (generated by gbp or dgit on maintainers' machines),
and, depending on the package, that information is sometimes critical.
IOW that ship has sailed.

My programme of properly supporting git in Debian is trying to sort
out this situation, by providing a discoverable, coherent, and
reliable gi

Bug#1007717: Native source package format with non-native version

[Felix Lechner]

Hi,

On Tue, Mar 15, 2022 at 9:33 PM Russ Allbery  wrote:
>
> We should define native and non-native
> packages in terms of version numbers, and allow both native and non-native
> packages to use single-tarball source package formats.

I co-maintaintain several Debian-internal tools, and that description
is backwards. "Native" sources are characterized by their lack of
Debian patches.

On that note, the term "native" is also not great. The words "patched"
and "unpatched" describe the relationship between sources in the
archive and their respective upstreams more accurately.

As for version strings, we need no additional restrictions. The use of
patches is declared in source format 3.0. Some folks even use Debian
revisions for unpatched sources. [1]

Most significantly, Lintian's parser is unable to deduce "nativeness"
from the version number. The native status is a required input! [2]

Please do not "define" a source's patch status via the version string.
It's what got us here in the first place. Debian version numbers are
complicated enough already. Thank you!

Kind regards,
Felix Lechner

[1] for example, https://tracker.debian.org/pkg/python3-defaults
[2] 
https://salsa.debian.org/lintian/lintian/-/blob/master/lib/Lintian/Changelog/Version.pm#L79-80

Bug#1007717: Native source package format with non-native version

[Russ Allbery]

Sam Hartman  writes:

> Specifically, I'd like to ask the TC to come up with policy on native
> packages and debian revisions using its power under 6.1.1.

As a Policy Editor, I support this request.

I've been involved in a lot of these discussions over the years, and the
tentative conclusion that I've come to is that we have unfortunately mixed
several different concepts in how we talk about source packages.  This has
not *technically* reduced the flexibility of the packaging format because
there are various workarounds, but it's *conceptually* reduced the
flexibility in a way that makes those workarounds look conceptually
incorrect.  As a result, we have constant repetitions of similar arguments
stemming from well-meaning attempts to clean up the apparent
inconsistencies (including some I have started).

There are really two separate concepts in play here:

1. Different version numbering conventions for native and non-native
   packages.  For non-native packages, we want to preserve the upstream
   version number as much as possible so that our users can correlate our
   packages to upstream releases.  We therefore have the Debian revision
   component of the version number.  But we don't want to use that
   component for native packages, since there is no corresponding upstream
   version.

   And I do believe that we need to keep the concept of native packages.
   There are configuration-only packages, metapackages, and other cases
   where even with an extremely aggressive idea of what should entail a
   separate upstream release, a separate upstream makes no sense.  And I
   can also confirm that outside of Debian native packages are heavily
   used by Debian users to package local things.

2. Source package formats.  Here, people want different things based on
   different workflows.  We may have opinions about which workflows are
   better and which are worse, but by and large Debian tries to provide
   room for innovation on workflows, and there are some important
   workflows outside of the archive (such as automated builds from CI as
   Sam has mentioned) where simplicity is far more important than careful
   tracking of distinctions between upstream and Debian changes.

   There is a clearly-expressed desire to support a package format with as
   simple of a representation as possible for some of those workflows,
   either because the complexity is being handled at another level or
   because the complexity is make-work for the desired goal that isn't
   worth doing (true for a lot of out-of-archive use cases).

   The current native format of a single tarball that you unpack and can
   then build as a Debian package with no further work is exactly what
   many of those workflows want.

We have conflated 1 and 2, but I don't think they are as closely related
as the project has conceptually presented them.  There is a tie in that
*if* there is a separate upstream release, then someone may want a package
format that allows keeping the Debian pieces and the upstream pieces
clearly separate.  But in practice there are reasons to want to use the
single-tarball source format for a non-native package.

(I admit I am unable to think of a plausible reason to use the multiple
tarball or tarball with patch format for a native package, but that may be
a failure of my imagination.  I can think of some space-conserving reasons
to want to use that packaging format if the archive would reuse the
tarballs, but the archive software won't when using native versioning, and
changing that would be extremely difficult and probably not worth the
effort.)

What I would propose is to separate these concepts cleanly so that we can
talk about them in a clearer way.  We should define native and non-native
packages in terms of version numbers, and allow both native and non-native
packages to use single-tarball source package formats.  We may want to
steer people away from using the single-tarball source package format for
the *normal* case of packaging upstream software, but I think it's well
within the sorts of trade-off decisions that packagers already make and
there are cases where a single tarball source format makes sense.

The name of the 3.0 (native) source package format is unfortunate in this
context since it strongly implies that only native packages will use that
source format.  The most formally correct thing to do would probably be to
introduce a new 3.0 (single-tarball) source format and use 1.0 until then,
but of course introducing a new source package format is not a fast
process.  A more expedient thing to do may be to allow use of 3.0 (native)
with non-native packages, since it's identical to 3.0 (single-tarball)
except for the name, but it does leave a long-term confusing naming scheme
in place.  Guillem may well have other and better suggestions from the
dpkg perspective; I haven't thought all that much about a transition plan.

Note that this only addresses the first part of Ian's request.  The second

Bug#1007717: Native source package format with non-native version

[Sam Hartman]

Dear TC,
I cannot speak for what Ian wants,
but I would also like to formally ask the TC to rule on this issue.
My hope is that what Ian and I are asking for is similar enough that the
TC can consider the issues together.

Specifically, I'd like to ask the TC to come up with policy on native
packages and debian revisions using its power under  6.1.1.
In particular I ask the TC to consider the hypothesis that:

* Debian revisions are useful when there is a separate upstream flow
  from the packaging flow.  For example when Debian versions are
  released at different times than upstream times, Debian versions have
  different content (for example a debian directory), or Debian versions
  are maintained by different people.  IN these cases a debian revision
  may be useful.

* Native source package formats (1.0 tar only and 3.0 (native)) are
  useful when the work flow
  is easier to maintain without separated Debian changes.  There are
  several cases involving git work flows where package maintenance is
  improved by not requiring this separation.

* Git work flows are a best practice.  There may be other best practices
  too.  We want to encourage git work flows to become more streamlined
  and easier; when people have found git work flows that make their job
  as maintainers easier, we want to encourage that.

I don't think the TC needs to do out-of-charter design work to come up
with policy in this regard.
I think there's been a lot of work within debian-policy, within
discussions of git packaging flows, and within the  consensus-seeking
discussions run over the years.

I'll point outt that the behavior of non-experimental package
building tools explicitly falls under 6.1.1 and does not require the TC
to override a maintainer.

I believe it is also appropriate for the TC to rule under 6.1.2.
It's clear that policy, the packaging tools, archive build tools, and
tools to support git work flows need to be in harmony for things to work
well.
It's clear there is overlap, and it is clear to me we do not have
harmony today.

I'd also like to provide some initial briefing material to the TC to
help you all come up to speend on this issue.

* Ian's mail that you were forwarded

* #bug 737634

   * My messages starting at
 
https://lists.debian.org/0144017b42b6-b65ba883-c94b-472c-89b7-7341c14ce8ab-000...@email.amazonses.com
 and following the thread.  I explain one case where you want to use
 a debian revision with native packages.  I think Ian has described
 cases that are more clearly directly related to the Debian archive,
 but I clearly outline in that bug report problems I was having and
 why proposed alternatives did not work.

* Around that time  a claim was made that debian-policy did not permit
  debian revisions with native packages.  I challenged that claim and in
  https://lists.debian.org/8761otp2r1@windlord.stanford.edu Russ
  acknowledges that the section of policy he was pointing at did not
  actually forbid debian revisions with native packages.  So at the time
  dpkg made the change it was breaking things and there was no policy
  requirement for the change.

* Ubuntu had to make a change to dpkg-source at least as used by
  launchpad for recipe builds to continue to work.  I don't know if they
  still maintain that patch, but it seems like a bad sign when a major
  downstream needs to revert a change  we've done to get work done.  It
  would perhaps be valuable to look into whether they still have that
  delta and if not how they work around things.

* https://lists.debian.org/tsla738bpwz@suchdamage.org the most
  interesting bit there is a consensus of the project that if your
  upstream uses git, best practice is for your packaging to use git.

* https://lists.debian.org/tslr252ioa3.fsf...@suchdamage.org Consensus:
  native packages are sometimes an appropriate tool to use in contrast
  to earlier consensus that we wanted to move away from them.

* Plus the recent thread on debian-devel.

* The git packaging BOF at DebConf19.

Thanks for your consideration,

--Sam


signature.asc
Description: PGP signature

Bug#1007717: Native source package format with non-native version

[Lucas Nussbaum]

Hi,

First, some context with numbers:
Source packages in testing: 32247
Source packages in testing using 3.0 (native): 690 (2.1%)
Source packages in testing using 3.0 (quilt): 30937 (95.9%)
Source packages in testing using 1.0: 620 (1.9%)

Those 620 packages probably fit in two categories:
A) those whose maintainers are inactive, or unaware that the
  package uses 1.0, and would switch to 3.0 if they knew about it
B) those whose maintainers choose to stay with 1.0 for a reason

Estimating the number of packages in both categories in difficult.
Packages obviously in (B) are those maintained by the Debian X team,
by iwj, by tfheen, or by az. Those that provide debian/source/format
are also good candidates (but there might be some false positives).
The number of each case and for the combination of both are:
... with explicit debian/source/format: 178
... maintained or co-maintained by Debian X, iwj, tfheen, az: 129
... with explicit debian/source/format OR maintained or
co-maintained by Debian X, iwj, tfheen, az: 262 (0.8%)

So, we are talking about approximately 0.8% of packages here.
There's a page at https://udd.debian.org/cgi-bin/format10.cgi that
allows exploring the list of packages (the classification criteria are
the ones used for the MBF, not the ones above).
There are also graphs at
https://trends.debian.net/#source-formats-and-patch-systems to see the
evolution over time.

On 15/03/22 at 20:08 +0100, Helmut Grohne wrote:
> On Tue, Mar 15, 2022 at 04:29:17PM +, Ian Jackson wrote:
> > (Sorry for the resend, this should have gone to the BTS the first
> > time; have fixed a slip in the wording.)
> 
> Thank you for resubmitting your issue as a bug report.
> 
> Beyond the content of your request, I have a meta-question. Do you see a
> particular urgency with the processing of your request? What is - in
> your opinion - a reasonable time for resolving this? Of course, if Lucas
> et al are to proceed with their deprecation work, urgency may arise in
> your view, but let us assume that we can ask them to pause their
> efforts for a while.
> 
> Lucas, please pause further work on deprecating the 1.0 format in order
> to give time to settle the dispute at hand. This implies not sending
> further bugs about it. On the other hand, I think closing all of the
> existing ones would not do any good at this time either.

I do not plan to upgrade the severity of the bugs, nor to file other
bugs. I am fine with maintainers/co-maintainers closing bugs (especially
with a rationale), and do not plan to reopen bugs that will be closed. I
do not plan to try to do further cleanup using a more complex heuristic
as I think that it's valuable to use those bugs to further identify
what are the blocking factors for migration to 3.0.

I can agree that the heuristic to determine which packages to file bugs
against could have been designed better. However, given that it's
impossible to read maintainers' mind, it's probably impossible to
achieve perfection here. (For example, I identified that the Debian X
team was using 1.0 with a specific workflow and excluded its packages, I
should probably have excluded Ian's packages as well, but only
discovered the objection of other maintainers because I did the MBF.)


A point that I find important is the following: as a project, I think
that we care about facilitating the review of changes we make to
upstream source. I think that the preferred method (and widely accepted
method) for that is currently to use the 3.0 (quilt) format with
DEP-3-documented patches, on top of a tarball that contains the pristine
upstream source.

The git packaging workflows that generate source packages using either
1.0 native packages, or 1.0 non-native packages without patches, make it
harder to identify and review those changes, as they require browsing
the git repository, which sometimes is not properly documented using
Vcs-*.

I do not think that the practice of making "fake" native source packages
should be encouraged by allowing 3.0 (native) packages to have Debian
revisions.

So my "ideal" TC ruling on those matters would be something along the
lines of:
- Use of 3.0 (quilt) or 3.0 (native) is recommended
- Use of 1.0 is discouraged
- After a proper work to identify affected packages and notify
  maintainers in advance, providing a debian/source/format file
  can be made mandatory to accelerate the transition to 3.0 (which the
  ability for packages to stay with 1.0 if explicit)
- Packages still using 1.0 must provide an adequate way to review Debian
  changes, such as a functional VCS repository.
- Maintainers that rely on workflows that require the use of 1.0 are
  encouraged to explore how to move to 3.0 without compromising the
  capability to review changes using the source package.

Lucas

Bug#1007717: Native source package format with non-native version

[Felix Lechner]

Hi Ian,

On Tue, Mar 15, 2022 at 11:54 AM Ian Jackson
 wrote:
>
> I do remember this coming up
> before, but I don't seem to be able to find a record of it now.

Perhaps you were thinking about this discussion in a bug against Lintian?

Ideally I would like dpkg-source to permit source format
`3.0 (native)' packages to have a non-native version number. [1]

Kind regards,
Felix Lechner

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=953554#30

Bug#1007717: Native source package format with non-native version

[Helmut Grohne]

Hi Ian,

On Tue, Mar 15, 2022 at 04:29:17PM +, Ian Jackson wrote:
> (Sorry for the resend, this should have gone to the BTS the first
> time; have fixed a slip in the wording.)

Thank you for resubmitting your issue as a bug report.

Beyond the content of your request, I have a meta-question. Do you see a
particular urgency with the processing of your request? What is - in
your opinion - a reasonable time for resolving this? Of course, if Lucas
et al are to proceed with their deprecation work, urgency may arise in
your view, but let us assume that we can ask them to pause their
efforts for a while.

Lucas, please pause further work on deprecating the 1.0 format in order
to give time to settle the dispute at hand. This implies not sending
further bugs about it. On the other hand, I think closing all of the
existing ones would not do any good at this time either.

> Please:

I note that you request a number of partially independent matters that
form disagreements with various people. The request is fairly entangled
(as is the matter at hand). I'm also wondering whether your request may
be over-specific.

Can I ask you to step back a bit and help us figure out what you really
want to achieve?

Given your other messages on the matter, I suppose that you want to be
able to use a source package format that can faithfully represent any
valid source tree. In particular, a source tree with a Debian revision
should be representable. You care less about being able to represent
differences to a tarball in some cases as long as the changed tree as a
whole is representable. This includes changes to permissions, symbolic
links and binary files. Which source format is used is not important to
you as long as it is able to meet these requirements.

Do you consider the above representation of your view accurate? If not,
please point out differences.

>  3. Consequently, declare that the recent MBF on this topic ought not
> to have been filed against packages where simply changing the
> source format does not currently work.  That would include at
> least 1.0 native packages with Debian revisions.

On a technical level, there is not much point in ruling that something
should not have happened. It did happen. The strongest thing that would
be possible here would be to rule that these bugs should be closed.

> Part II - bless continued use of 1.0-with-diff, for now at least:
> 
>  4. Declare that sometimes the use of 1.0-with-diff can be the best
> tradeoff between different considerations.  In particular,
> because 1.0 is the only format which botH:
>  (a) Optimises bandwidth and storage by reusing the upstream 
>  data when it hasn't changed.
>  (b) Avoids polluting the working tree (package source code)
>  with [patches], which cause trouble especially with
>git-based workflows.

I note that the 1.0-with-diff format does not meet my perception of your
earlier requirement: Being able to represent any source tree. I also
note that it is possible to choose a different conversion mechanism than
dpkg-source -b between working tree and .dsc that does not incur your
reported pollution. In particular, you do have experience in
implementing such conversions as is evidenced in dgit and such
conversions do exist.

Your request also touches the question of the preferred form for
modification. That used to be the .dsc, but for the way you use the
archive, it merely is an export for a preferred form for modification
that resides elsewhere. I do not think we have consensus for this view
either.

I also caution that whenever we perform a fundamental change, some
things that used to work well, stop working well. Many improvements
regress on other aspects that are deemed niche features. I think it is
fair to say that use of 1.0 packaging formats is a niche at this point.
Consistency has value, but it comes with a price of making some niche
features impossible. We are discussing a trade-off between consistency
and niche features here.

Helmut

Bug#1007717: Native source package format with non-native version

[Ian Jackson]

A friend suggested some information that I could usefully provide:

Firstly, I posted to -devel a summary of why format 3.0 is not
pareto-better than 1.0.  I have c&p it below.

Secondly, I was asked if there was a bug against src:dpkg asking for
"3.0 (native)" to allow packages with Debian revisions.  I don't think
there is - at least, I didn't find one.  I do remember this coming up
before, but I don't seem to be able to find a record of it now.

Sean Whitton writes ("Re: Bug#1007717: Native source package format with 
non-native version"):
> I think that you are asking us to use 6.1.1/6.1.2 for most of this, but
> for I.3 are you asking for us to use 6.1.4 or 6.1.5?  "Request" seems
> more like 6.1.5 but I would like to check.

I think you mean "for I.2 are you asking".  I am not asking for the TC
to overrule the dpkg maintainer under 6.1.4.

Let me requote myself with annotations:

> On Tue 15 Mar 2022 at 04:29pm GMT, Ian Jackson wrote:
> > Please:
> >
> > Part I - belss continued use of 1.0 native format, for now at least:
> >
> >  1. Declare explicitly that there is nothing wrong with a package with
> > a native format, but a non-native version number.

 6.1(1) Decide on any matter of technical policy.

> >  2. Request that the dpkg maintainer relax the restriction which
> > prevents the use of 3.0 native with Debian revision.

 6.1(5) Offer advice.

> >  3. Consequently, declare that the recent MBF on this topic ought not
> > to have been filed against packages where simply changing the
> > source format does not currently work.  That would include at
> > least 1.0 native packages with Debian revisions.

 6.1(5) Offer advice.

> > Part II - bless continued use of 1.0-with-diff, for now at least:
> >
> >  4. Declare that sometimes the use of 1.0-with-diff can be the best
> > tradeoff between different considerations.  In particular,
> > because 1.0 is the only format which botH:
> >  (a) Optimises bandwidth and storage by reusing the upstream
> >  data when it hasn't changed.
> >  (b) Avoids polluting the working tree (package source code)
> >  with [patches], which cause trouble especially with
> >  git-based workflows.

 6.1(5) Offer advice.

> >  5. Consequently, declare that the recent MBF on this topic ought not
> > to have been filed against 1.0 with diff packages, at least
> > without some further filter.

 6.1(5) Offer advice.

Thanks,
Ian.


From: Ian Jackson 
To: Sean Whitton 
Cc: Russ Allbery ,
debian-de...@lists.debian.org
Subject: Re: proposed MBF: packages still using source format 1.0
Date: Wed, 9 Mar 2022 16:38:43 +

Sean Whitton writes ("Re: proposed MBF: packages still using source format 
1.0"):
> On Wed 09 Mar 2022 at 01:08pm +01, Lucas Nussbaum wrote:
> > On 08/03/22 at 17:33 -0700, Sean Whitton wrote:
> >> Lucas, as I've had a lot to do with these git workflows and have
> >> probably done the most work documenting them, I can help with any
> >> specific follow-up questions you might have.
> >
> > Thanks!
> >
> > So the main question I think I have is:
> >
> > can we find a middleground where the git workflows don't require staying
> > with 1.0? Even if that means switching to 3.0 (quilt) using the
> > single-debian-patch approach?
> 
> dgit-maint-merge(7) works with single-debian-patch and that's what I
> use.  But it is not clear to me that there are any advantages at all to
> that over 3.0 (quilt) with single-debian-patch?

The situation here is complicated.


The tl;dr is that

 * there are several situations where 1.0-native is the best answer,
 * there are several situations where 1.0-with-diff is the best answer,

The root cause of both of these situations is that 3.0, sadly,
is not always better in every respect than 1.0.


1. Why is 1.0-without-diff not always worse than 3.0 (native) ?

1.0 native is sometimes better than 3.0 (native) because dpkg-source
refuses to build a 3.0 native package with a Debian revision in its
version number.

This prohibition exists solely because of a doctrinal objection to
native-format packages with Debian revisions.  There is no technical
reason why this restriction could not be lifted.  I sometimes upload
this way and I have never had anyone report problems[1] with it.

IMO there is nothing wrong with native format packages with Debian
revisions.  They work just fine.  For a small paockage, this is often
a good choice, because it avoids dealing with patches at all.

For anything but a small package, use of diffs is needed as a storage
and distribution optimisation.


2. Why is 1.0-with-diff not always worse than some 3.0 

Bug#1007717: Native source package format with non-native version

[Sam Hartman]

> "Ian" == Ian Jackson  writes:
Ian>  3. Consequently, declare that the recent MBF on this topic
Ian> ought not to have been filed against packages where simply
Ian> changing the source format does not currently work.  That would
Ian> include at least 1.0 native packages with Debian revisions.

I'm a little confused.
My reading of the debian-devel discussion was that Lucas had already
modified the MBF in a manner consistent with what you're asking for.
It looks like there was a bug in the criteria he used, and it sounds
like he already agrees that a few bugs were filed that should not have
been.
It sounds like he's actively working to fix that.


I think asking the TC to get involved here is a great idea on the
other issues you ask about.
I'm just not sure there is a contraversyfor the TC on the specific
case of the MBF itself.

Bug#1007717: Native source package format with non-native version

[Sean Whitton]

Hello,

On Tue 15 Mar 2022 at 04:29pm GMT, Ian Jackson wrote:

> Please:
>
> Part I - belss continued use of 1.0 native format, for now at least:
>
>  1. Declare explicitly that there is nothing wrong with a package with
> a native format, but a non-native version number.
>
>  2. Request that the dpkg maintainer relax the restriction which
> prevents the use of 3.0 native with Debian revision.
>
>  3. Consequently, declare that the recent MBF on this topic ought not
> to have been filed against packages where simply changing the
> source format does not currently work.  That would include at
> least 1.0 native packages with Debian revisions.
>
> Part II - bless continued use of 1.0-with-diff, for now at least:
>
>  4. Declare that sometimes the use of 1.0-with-diff can be the best
> tradeoff between different considerations.  In particular,
> because 1.0 is the only format which botH:
>  (a) Optimises bandwidth and storage by reusing the upstream
>  data when it hasn't changed.
>  (b) Avoids polluting the working tree (package source code)
>  with [patches], which cause trouble especially with
>git-based workflows.
>
>  5. Consequently, declare that the recent MBF on this topic ought not
> to have been filed against 1.0 with diff packages, at least
> without some further filter.

I think that you are asking us to use 6.1.1/6.1.2 for most of this, but
for I.3 are you asking for us to use 6.1.4 or 6.1.5?  "Request" seems
more like 6.1.5 but I would like to check.

-- 
Sean Whitton

Bug#1007717: Native source package format with non-native version

[Ian Jackson]

Package: tech-ctte

(Sorry for the resend, this should have gone to the BTS the first
time; have fixed a slip in the wording.)

Please:

Part I - belss continued use of 1.0 native format, for now at least:

 1. Declare explicitly that there is nothing wrong with a package with
a native format, but a non-native version number.

 2. Request that the dpkg maintainer relax the restriction which
prevents the use of 3.0 native with Debian revision.

 3. Consequently, declare that the recent MBF on this topic ought not
to have been filed against packages where simply changing the
source format does not currently work.  That would include at
least 1.0 native packages with Debian revisions.

Part II - bless continued use of 1.0-with-diff, for now at least:

 4. Declare that sometimes the use of 1.0-with-diff can be the best
tradeoff between different considerations.  In particular,
because 1.0 is the only format which botH:
 (a) Optimises bandwidth and storage by reusing the upstream 
 data when it hasn't changed.
 (b) Avoids polluting the working tree (package source code)
 with [patches], which cause trouble especially with
 git-based workflows.

 5. Consequently, declare that the recent MBF on this topic ought not
to have been filed against 1.0 with diff packages, at least
without some further filter.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

Native source package format with non-native version

[Ian Jackson]

Package: tech-ctte

Please:

Part I - belss continued use of 1.0 native format, for now at least:

 1. Declare explicitly that there is nothing wrong with a package with
a native format, but a non-native version number.

 2. Request that the dpkg maintainer relax the restriction which
prevents the use of 3.0 native with Debian revision.

 3. Consequently, declare that the recent MBF on this topic ought not
to have been filed against packages where simply changing the
source format does not currently work.  That would include at
least 1.0 native packages with Debian revisions.

Part II - bless continued use of 1.0-with-diff, for now at least:

 4. Declare that sometimes the use of 1.0-with-diff can be the best
tradeoff between different considerations.  In particular,
because 1.0 is the only format which botH:
 (a) Optimises bandwidth and storage by reusing the upstream 
 data when it hasn't changed.
 (b) Avoids polluting the working tree (package source code)
 with packages, which cause trouble especially with
 git-based workflows.

 5. Consequently, declare that the recent MBF on this topic ought not
to have been filed against 1.0 with diff packages, at least
without some further filter.

-- 
Ian JacksonThese opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.