Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Russ Allbery]

Bill Allombert  writes:

> The magic of dh comes by making assumption on the upstream build system.
> When these assumptions are correct then it is much less verbose than
> debhelper. When they are not correct the maintainer needs to override
> all incorrect guesses, in addition to writing the correct code, at which
> point the benefit of dh is greatly reduced.

This statement is quite contrary to my experience with dh.  The magic of
dh is to automate the normal sequence of debhelper commands, which is far
more than just integrating with the upstream build system.  When upstream
has an odd build system, I usually only have to override three dh
sequences (dh_auto_configure, dh_auto_build, and dh_auto_install); the
vast majority of dh and its accompanying simplification is still intact,
and the resulting overrides are still the same as or less verbose than
writing the equivalent code with debhelper.

The magic of dh is not having to remember what order you have to put
dh_strip and dh_fixperms in, or to remmeber to include dh_lintian when I
add an override, or to allow us to add dh_strip_nondeterminism globally
for most packages in Debian without requiring a mass bug filing.

> Updating dh to a new compat level might include extra assumption that
> will then need to be overrided.

The same is true of using any packaging helper that uses debhelper, and is
still true for new versions of Policy even if you hand-code everything.

-- 
Russ Allbery (r...@debian.org)   

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Bill Allombert]

On Sat, May 25, 2019 at 01:26:47PM -0400, Sam Hartman wrote:
> 
> Hi. Almost two weeks ago [1] I  started a discussion on whether we
> wanted to increase the strength of our recommendation of the dh
> sequencer from debhelper.
> This message is a consensus call summarizing my reading of the
> discussion.
> 
>   [1]
>   https://lists.debian.org/msgid-search/tsla7fqjzyv@suchdamage.org

The magic of dh comes by making assumption on the upstream build system.
When these assumptions are correct then it is much less verbose than
debhelper. When they are not correct the maintainer needs to override
all incorrect guesses, in addition to writing the correct code, at which
point the benefit of dh is greatly reduced.  

Updating dh to a new compat level might include extra assumption that
will then need to be overrided.

The maintainer is the best placed to know when the trade off should be.

Cheers,
-- 
Bill. 

Imagine a large red swirl here.

Re: Do we want to Require or Recommend DH

[Andreas Tille]

On Tue, Jun 04, 2019 at 03:06:13PM +, Thorsten Glaser wrote:
> Ian Jackson dixit:
> >But our one un-shirkable responsibility is that of creating an
> >environment where *others* can contribute.

@Ian:  I really like that quote that could define a modernised Debian.
 
> Oh, sorry, but, I disagree. Others can contribute in other packages,
> I can do mine just fine. (Of course, the occasional contribution is
> welcome, but I’m not going to bend my ways for it.)

IMHO this specifies Debian 15 years ago.  I fear this attitude will
decrease the relevance of Debian in future if we do not have the power
to change. 

Kind regards

   Andreas.

-- 
http://fam-tille.de

Re: Do we want to Require or Recommend DH

[Alf Gaida]

I think such a GR would be a collosal waste of time.  This issue is
not important enough.  In particular, because the consensus is *not*

GR's can be man made a collosal waste of time.


Well, a GR can be quick and it would help to know where people stand
instead of having a few vocal people decide for everyone. I think we
should impose the use of "dh" for bullseye (with an exception for teams
with more then 50 packages), but I honestly don't know how much this
opinion is shared.
I followed the discussion closely but i don't get some points. I assume 
that "dh" is here to stay - so in that case new packages should be done 
with "dh", older packages should be converted. There might be some 
packages which are not worth the additional work. Just leave a note why 
and everyone is happy. So a GR would be a appr. tool to solve this 
endless discussion fast.

last "technical" GR was for systemd in 2014. We are in a project where
it is very hard to be heard because you can only participate in endless
debates.
I for myself have no time for endless debates - i have things to do in 
Debian and upstream. And it's just boring to read the same arguments pro 
and esp. contra again and again. I was quiet until now because the 
debate don't change anything for me firsthand. I use dh for all my 
packages already and don't think i will change it in the future. The 
very most packages i'm interested in are dh - so no problem for me to 
read and maybe fix them if needed. Will i touch complex things written 
in cdbs? Hell no. Will i touch other complex things not in dh - hell, 
no, not worth the time. One might think of as a bit stubborn or short 
sighted, but to be true: It's lack of motivation - learning a bunch of 
things i'm not interested in to change things i'm not that interested 
in? Sounds nuts.


A solution could be: Deprecate some thing like cdbs an others if they 
are really deprecated, be verbose about why and don't let packages that 
using these things go into the repository. Can be done stepwise.


My 2¢

Alf

Re: Do we want to Require or Recommend DH

[Vincent Bernat]

 ❦  4 juin 2019 15:47 +01, Ian Jackson :

> If not, how do you think the question you pose should be answered ?
> Since it is a question of tradeoffs, with no definite right or wrong
> answer, perhaps we should hold a GR ?  What do you think the result of
> such a GR would be ?
>
> I think such a GR would be a collosal waste of time.  This issue is
> not important enough.  In particular, because the consensus is *not*
> that you will *have to* change your packages.  What this discussion
> has mostly concluded is that we should issue a *recommendation*.
> *Not* a mandate.

Well, a GR can be quick and it would help to know where people stand
instead of having a few vocal people decide for everyone. I think we
should impose the use of "dh" for bullseye (with an exception for teams
with more then 50 packages), but I honestly don't know how much this
opinion is shared.

It seems there is a pattern to dissuade people to hold a GR. The last GR
I remember is about changing "Chairman" to "Chair" in our constitution.
I don't remember it was a waste of time and it was pretty quick. And the
last "technical" GR was for systemd in 2014. We are in a project where
it is very hard to be heard because you can only participate in endless
debates.
-- 
Let him choose out of my files, his projects to accomplish.
-- Shakespeare, "Coriolanus"


signature.asc
Description: PGP signature

Re: Do we want to Require or Recommend DH

[Thorsten Glaser]

Ian Jackson dixit:

>There is QA work on the many packages with no specific maintainer;

Sure, in that case I’ll have to take it over or deal with it.

>there are cross-archive campaigns such as reproducible builds,
>architecture support, init system diversity, i18n/l10n, and so on.

These are done by specialists. I didn’t say *nobody* would need
to learn the others, just that not *everyone* needs, especially
not at first.

>There is RC bugfixing etc. to try to make a release.  There is

This is either a good learning chance, or just tackle an RC bug
in another package.

>security support, stable release management, and backports.

Again, specialists.

>Well, Sam posed a consultation here on debian-devel.  My assessment of
>the rough consensus of the discussion here is very similar to Sam's.

My point was to raise concerns of mine.

>Do you agree with Sam's assessment of the apparent consensus on this
>list ?

I’ve not read the entire discussion here. I stumbled upon this
by reading the DPL news and thus posted because I feared that
the point important to me might be underrepresented.

>Since it is a question of tradeoffs, with no definite right or wrong
>answer, perhaps we should hold a GR ?  What do you think the result of
>such a GR would be ?

Hmm, did not consider it, but a GR could fight being forced to
use dh7 if needed. Thanks for the idea.

>You may be gently encouraged to change your packages.  In practice if
>you refuse, it is not likely that anyone will want to fight you over
>it.  You will probably be able to leave the bug open "wontfix", if

Perhaps. Perhaps not. Perhaps, if that’s acceptable, it’ll be enough.

>there is a bug at all, indefinitely.

If. If it isn’t, leaving it wontfix or closing is guaranteed to be
acceptable. If it is, some clarification that it is acceptable is
needed or we’re entering vague territories (such as, where people
who don’t like a maintainer file RM requests against their packages
because they don’t follow this-and-that latest fad).

>So, nothing will be forced onto you and you do not need to fight this.

If optimistic, yes.

>But I would like you to consider this: the primary responsibility of
>the maintainer of a Debian package is a *management* responsibility.

Oh sure. But I just want to make the world better by packaging some
software for Debian, some of which I happen to be upstream of, some
of which I happen to have become upstream because of packaging it.
I occasionally join in teams, but mostly just wish to quietly do my
thing, undisturbed.

>But our one un-shirkable responsibility is that of creating an
>environment where *others* can contribute.

Oh, sorry, but, I disagree. Others can contribute in other packages,
I can do mine just fine. (Of course, the occasional contribution is
welcome, but I’m not going to bend my ways for it.)

Just like when I contribute somewhere, I’m, sometimes extremely rudely,
asked to take my problem (or even patch) upstream myself or go sod off.

bye,
//mirabilos
-- 
 exceptions: a truly awful implementation of quite a nice idea.
 just about the worst way you could do something like that, afaic.
 it's like anti-design.   that too… may I quote you on that?
 sure, tho i doubt anyone will listen ;)

Re: Do we want to Require or Recommend DH

[Andrey Rahmatullin]

On Tue, Jun 04, 2019 at 02:27:03PM +, Thorsten Glaser wrote:
> No. A maintainer normally deals with their own packages, or with
> .dsc and debdiff, for NMU. (This is also an answer to the reply
> from wrar. Oh, jonas also said so, reloading the list index page.)
A maintainer normally deals with their own packages, except those people
who actually prepare that NMU debdiff, yeah. Not sure what did you mean by
this.

> Yes, some must learn those ways, but I don’t mind; 
Does this mean "some must learn many different things to be able to make
NMUs, but I don’t mind" or am I mistaken?

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Do we want to Require or Recommend DH

[Ian Jackson]

Thorsten Glaser writes ("Re: Do we want to Require or Recommend DH"):
> No. A maintainer normally deals with their own packages, or with
> .dsc and debdiff, for NMU. (This is also an answer to the reply
> from wrar. Oh, jonas also said so, reloading the list index page.)

"Maintainer" is precisely the hat we wear when working on our own
packages.  But working with Debian is much more than just working on
our own packages.

There is QA work on the many packages with no specific maintainer;
there are cross-archive campaigns such as reproducible builds,
architecture support, init system diversity, i18n/l10n, and so on.
There is RC bugfixing etc. to try to make a release.  There is
security support, stable release management, and backports.

And of course as users and downstreams we wish to exercise our
software freedom: ie to modify the programs that run on our computer.
That freedom being the point of Debian.

For all these tasks, we often need to interact with or modify the
package's build machinery (to varying extent).  That means learning
the build machinery of every package we work on - at least well enough
to accomplish the task at hand.

> Is 'the Debian of today' the *Debian* of today, or just a couple of
> very involved people?

Well, Sam posed a consultation here on debian-devel.  My assessment of
the rough consensus of the discussion here is very similar to Sam's.
Do you agree with Sam's assessment of the apparent consensus on this
list ?

If not, how do you think the question you pose should be answered ?
Since it is a question of tradeoffs, with no definite right or wrong
answer, perhaps we should hold a GR ?  What do you think the result of
such a GR would be ?

I think such a GR would be a collosal waste of time.  This issue is
not important enough.  In particular, because the consensus is *not*
that you will *have to* change your packages.  What this discussion
has mostly concluded is that we should issue a *recommendation*.
*Not* a mandate.

You may be gently encouraged to change your packages.  In practice if
you refuse, it is not likely that anyone will want to fight you over
it.  You will probably be able to leave the bug open "wontfix", if
there is a bug at all, indefinitely.

> I doubt those very involved people, with hundreds of packages in their
> DDPO already (don't laugh, I saw that), could shoulder the burden, were
> those others to leave disgruntled by things being forced onto them.

So, nothing will be forced onto you and you do not need to fight this.


But I would like you to consider this: the primary responsibility of
the maintainer of a Debian package is a *management* responsibility.
Our job as maintainer is not to do all the work.  If we like to do the
work ourselves, great.  If we don't and the work goes undone, well,
c'est la vie; maybe someone else will have the energy.

But our one un-shirkable responsibility is that of creating an
environment where *others* can contribute.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: Do we want to Require or Recommend DH

[Thorsten Glaser]

Sam Hartman dixit:

>He doesn't actually make that argument.

Hmm. Right, he doesn’t spell it out, but I got the impression.
Perhaps my reading was wrong.

>There are several reasons for not using dh we've already identified.

Sure… but…
>The fun factor is important.
… that.

>My reading of the community consensus is that the points you bring up
>have been consider by the community.
>You did not bring up new issues.

This is diametral opposite to:

>my reading is that the community believes that the fun factor of more
>uniformity when dealing with a lot of packages justifies the restriction
>of maintainer preference when there's not a sufficient justification for
>a tool other than dh.

No. A maintainer normally deals with their own packages, or with
.dsc and debdiff, for NMU. (This is also an answer to the reply
from wrar. Oh, jonas also said so, reloading the list index page.)

Yes, some must learn those ways, but I don’t mind; that doesn’t
mean I’m more comfortable in dh7. Usually I’m not except for
extremely simple packages, or, partially, really new packages.

>You're absolutely right that there is a tradeoff here.
>And I think that Debian of 10 or 15 years ago would have evaluated the
>tradeoff between the needs of a single maintainer and the needs of
>people doing work across a lot of packages differently.

Is “the Debian of today” the *Debian* of today, or just a couple of
very involved people?

Do you consider all those people who just take care of their own
package, or couple of packages, in what little spare time they have?

I doubt those very involved people, with hundreds of packages in their
DDPO already (don’t laugh, I saw that), could shoulder the burden, were
those others to leave disgruntled by things being forced onto them.

bye,
//mirabilos
-- 
„Cool, /usr/share/doc/mksh/examples/uhr.gz ist ja ein Grund,
mksh auf jedem System zu installieren.“
-- XTaran auf der OpenRheinRuhr, ganz begeistert
(EN: “[…]uhr.gz is a reason to install mksh on every system.”)

Re: Do we want to Require or Recommend DH

[Andrey Rahmatullin]

On Tue, Jun 04, 2019 at 04:10:38PM +0200, Jonas Smedegaard wrote:
> > > I’d also throw in that monocultures are not good, and that people in 
> > > general are happier when they aren’t forced into anything.
> > Yet people in general are also happier when they don't need to learn 
> > all ways to do something.
> 
> Who says people need to learn _all_ ways?
> 
> Must we all learn the ways of Java and DKMS and Haskell and and...?
> 
> Nice if we can _generally_ handle _most_ packages.
To be able to handle most packages we must mandate that most packages use
an uniform workflow. That's different from "monocultures are not good"
etc.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Do we want to Require or Recommend DH

[Sam Hartman]

> "Jonas" == Jonas Smedegaard  writes:

Jonas> Quoting Andrey Rahmatullin (2019-06-04 15:58:33)
>> On Tue, Jun 04, 2019 at 01:37:46PM +, Thorsten Glaser wrote:
>> > I’d also throw in that monocultures are not good, and that
>> people in > general are happier when they aren’t forced into
>> anything.  Yet people in general are also happier when they don't
>> need to learn all ways to do something.

Jonas> Who says people need to learn _all_ ways?

Jonas> Must we all learn the ways of Java and DKMS and Haskell and
Jonas> and...?

no, but some of us must or at least must be prepared to.

Re: Do we want to Require or Recommend DH

[Sam Hartman]

> "Thorsten" == Thorsten Glaser  writes:

Thorsten> I would very much like to argue that not using dh is not a
Thorsten> bug, but Joey Hess, with his credentials ☺, did that
Thorsten> already (and much better than I could):

Thorsten> http://joeyh.name/blog/entry/80_percent/

He doesn't actually make that argument.

>Not being part of Debian anymore, I'm in the position of needing to
>point out something important about it anyway. So this post is less
>about pointing in a specific direction as giving a different angle to
>think about things.

He argues that dh may have evolved to be around a 96% solution at this
point.

That's entirely consistent with the idea that not using dh could be a
bug absent an exceptional situation.  (Appologies for not looking up the
wording from Ian's message; I mean what Ian calls unusual here.)



Thorsten> tl;dr: dh started as 80% solution, it’s maybe an 96%
Thorsten> solution now, but it’s not intended as, and won’t be, a
Thorsten> 100% solution.

Nor have we claimed that it is.
There are several reasons for not using dh we've already identified.

Thorsten> I’d also throw in that monocultures are not good, and that
Thorsten> people in general are happier when they aren’t forced into
Thorsten> anything.

Agreed.
Working on new tooling clearly needs to be one of the reasons for not
using dh to allow for development of new things.

Thorsten> Just yesterday I had a bug that wouldn’t have
Thorsten> happened with a non-dh7 rules file (incidentally, ordering
Thorsten> matters, so I had to add a call to mkdir -p
Thorsten> debian/binarypackagename/some/directory into an over‐
Thorsten> ride). And finally, rules with too many overrides are
Thorsten> actually worse readable than classic debhelper style.

Thorsten> I also have packages where the automatic build system
Thorsten> detection of dh is wrong. Understandably wrong, but wrong
Thorsten> nevertheless.

Thorsten> Oh, and… to learn, automagisms are not so good, because
Thorsten> you don’t see what’s going on, and can’t change it on an
Thorsten> intuitive or more fine-granular level (though with
Thorsten> DH_VERBOSE=1 mandated by default by recent Policy changes,
Thorsten> this may have improved a little).

Thorsten> So… to each their own. I’d make a case for non-debhelper
Thorsten> to be allowed, but I know that’s not majority-capable… but
Thorsten> if people wish to use debhelper, dh7, or even *shudder*
Thorsten> dbs or cdbs, fine.  Remember people make this often in
Thorsten> their spare time and aren’t getting paid for it so please
Thorsten> keep the fun factor.

The fun factor is important.

My reading of the community consensus is that the points you bring up
have been consider by the community.
You did not bring up new issues.

my reading is that the community believes that the fun factor of more
uniformity when dealing with a lot of packages justifies the restriction
of maintainer preference when there's not a sufficient justification for
a tool other than dh.

You're absolutely right that there is a tradeoff here.
And I think that Debian of 10 or 15 years ago would have evaluated the
tradeoff between the needs of a single maintainer and the needs of
people doing work across a lot of packages differently.

--Sam

Re: Do we want to Require or Recommend DH

[Jonas Smedegaard]

Quoting Andrey Rahmatullin (2019-06-04 15:58:33)
> On Tue, Jun 04, 2019 at 01:37:46PM +, Thorsten Glaser wrote:
> > I’d also throw in that monocultures are not good, and that people in 
> > general are happier when they aren’t forced into anything.
> Yet people in general are also happier when they don't need to learn 
> all ways to do something.

Who says people need to learn _all_ ways?

Must we all learn the ways of Java and DKMS and Haskell and and...?

Nice if we can _generally_ handle _most_ packages.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: Do we want to Require or Recommend DH

[Andrey Rahmatullin]

On Tue, Jun 04, 2019 at 01:37:46PM +, Thorsten Glaser wrote:
> I’d also throw in that monocultures are not good, and that people
> in general are happier when they aren’t forced into anything.
Yet people in general are also happier when they don't need to learn all
ways to do something.

> Just yesterday I had a bug that wouldn’t have happened with a non-dh7
> rules file (incidentally, ordering matters, so I had to add a call to
> mkdir -p debian/binarypackagename/some/directory into an over‐ ride). 
dh_installdirs runs pretty early in the install target, what needed that
directory before dh_installdirs?

> I also have packages where the automatic build system detection
> of dh is wrong. Understandably wrong, but wrong nevertheless.
It's a feature of dh_auto_* and not of dh.

> Oh, and… to learn, automagisms are not so good, because you don’t
> see what’s going on,
You see what helpers are executed. You don't always see what do they do but
that's unrelated to dh(1).

> (though with DH_VERBOSE=1 mandated by default by recent Policy changes,
> this may have improved a little).
If you mean "The package build should be as verbose as reasonably
possible" then it doesn't really mandate DH_VERBOSE=1.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Do we want to Require or Recommend DH

[Thorsten Glaser]

I would very much like to argue that not using dh is not a bug,
but Joey Hess, with his credentials ☺, did that already (and much
better than I could):

http://joeyh.name/blog/entry/80_percent/

tl;dr: dh started as 80% solution, it’s maybe an 96% solution now,
but it’s not intended as, and won’t be, a 100% solution.

I’d also throw in that monocultures are not good, and that people
in general are happier when they aren’t forced into anything. Just
yesterday I had a bug that wouldn’t have happened with a non-dh7
rules file (incidentally, ordering matters, so I had to add a call
to mkdir -p debian/binarypackagename/some/directory into an over‐
ride). And finally, rules with too many overrides are actually
worse readable than classic debhelper style.

I also have packages where the automatic build system detection
of dh is wrong. Understandably wrong, but wrong nevertheless.

Oh, and… to learn, automagisms are not so good, because you don’t
see what’s going on, and can’t change it on an intuitive or more
fine-granular level (though with DH_VERBOSE=1 mandated by default
by recent Policy changes, this may have improved a little).

So… to each their own. I’d make a case for non-debhelper to be
allowed, but I know that’s not majority-capable… but if people
wish to use debhelper, dh7, or even *shudder* dbs or cdbs, fine.
Remember people make this often in their spare time and aren’t
getting paid for it so please keep the fun factor.

Thanks,
//mirabilos
-- 
This space for rent.

https://paypal.me/mirabilos to support my work.

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Adrian Bunk]

On Sun, May 26, 2019 at 11:14:33PM +0200, Philip Hands wrote:
> Adrian Bunk  writes:
> ...
> > Often the most difficult part of packaging are the unique rules the 
> > Debian ftp team requires for debian/copyright that are not required in 
> > distributions with actual lawyers. That's a completely separate topic.
> 
> That seems needlessly snide, and glosses over the fact that we are bound
> by constraints that other distributions are not, such as giving a damn
> about the license conditions we inflict on our downstreams, which I
> suspect the lawyers you allude to are not being retained to contemplate.

That seems needlessly snide, I am pretty sure the lawyers at companies 
like Red Hat or Oracle are retained to contemplate avoiding unnecessary
legal risks.

In legally risky areas like MP3 companies like Red Hat also have a 
history of being far more cautious than what we put on our mirrors.

It would be even more reason to get a statement from an actual lawyer,
instead of basing everything on "the FTP team's view" of non-lawyers.

Status quo is that the ftp team sets rules that cause plenty of work and 
frustration among contributors - and that might not even be required.

It would make life easier for many people if there would be:
1. A proper legal opinion on what is actually required for debian/copyright.
2. Some research what is actually useful for our users.
3. A discussion what can be done without too much burden for maintainers.

There are so many things about the current debian/copyright
handling that don't make much sense, e.g. the problem that
DEP-5 debian/copyright does show the actual copyright of a
binary package in all cases makes it much less useful for users.

> Cheers, Phil.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Andreas Tille]

On Mon, May 27, 2019 at 08:42:26AM +0200, Jonas Smedegaard wrote:
> 
> Does that paper you talk about point to _causes_ Debian packaging being 
> more scary? Is it a) complexities related to hardening, cross-building, 
> bootstrapping etc. or b) lack of a single¹ unified build framework, or 
> c) that Debian is "different" in package naming and code availability 
> than other system package managers or or misc. language-specific package 
> managers, or d) something else than what you snipped from the beginning 
> of this subthread?

No, this is basically a paper about BioConda and is just mentioning
Debian (Med) as something like the only honest "competitor".  As Alex
mentiones in the other response there are other reasons why Conda
packaging is prefered over Debian packages.  So it the paper is not
about the details you are asking for - but it also meniones that Debian
is to complex for upstreams to consider it.  (I personally do not fully
agree with this since there are lots of programs in this scope that are
pretty easy to package and if they are not its mostly a problem of the
software itself that it is hard to package - but that's a different
discussion.)

Kind regards

   Andreas.

-- 
http://fam-tille.de

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Sam Hartman]

> "Ian" == Ian Jackson  writes:

Ian> Firstly, I want to say that I think this is an awesome way to
Ian> conduct this discussion/decisionmaking/whatever.  Thank you.

Thanks.
I'm really hoping it does end up working well and that we can train many
people to do it.

Ian> Sam Hartman writes ("Consensus Call: Do We Want to Require or
Ian> Recommend DH; comments by 2019-06-16"):
>> Recommendation ==
>> 
>> There are some exceptions where we think using dh is the wrong
>> choice; see below.  We have a strong consensus that other than in
>> exceptional circumstances, new packages should use dh.

Ian> I would use the word "unusual" rather than the word
Ian> "exceptional".  Legalistically they have nearly the same
Ian> meaning, but they convey a difference of emphasis: a difference
Ian> in how strong a reason is good enough for not using dh; or, in
Ian> what proportion of exceptions we are expecting.


Ian> I haven't systematically reread the thread as you have, but my
Ian> impression is that we have a "strong consensus" as you put it
Ian> that dh should be used unless there is "some reasonable reason"
Ian> not to.

I actually like the wording of "some reasonable reason".  I'll see if I
can work that in rather than exceptional or unusual.  I agree with the
change in emphasis you're trying to make, but at least on this side of
the pond, I'm not actually sure unusual connotes more common than
exceptional.  I suspect the real place where this matters is how it gets
written up in policy.

Ian> I don't think we have "strong consensus" that these exceptions
Ian> are going to be rare.  "Exceptional" conveys both that there is
Ian> some reason, but also an expectation of rarity.

The Haskell exception for example will not be rare at all in that part
of the archive.

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Ian Jackson]

Firstly, I want to say that I think this is an awesome way to conduct
this discussion/decisionmaking/whatever.  Thank you.

Sam Hartman writes ("Consensus Call:  Do We Want to Require or Recommend DH; 
comments by 2019-06-16"):
> Recommendation
> ==
> 
> There are some exceptions where we think using dh is the wrong choice;
> see below.  We have a strong consensus that other than in exceptional
> circumstances, new packages should use dh.

I would use the word "unusual" rather than the word "exceptional".
Legalistically they have nearly the same meaning, but they convey a
difference of emphasis: a difference in how strong a reason is good
enough for not using dh; or, in what proportion of exceptions we are
expecting.

I haven't systematically reread the thread as you have, but my
impression is that we have a "strong consensus" as you put it that dh
should be used unless there is "some reasonable reason" not to.

I don't think we have "strong consensus" that these exceptions are
going to be rare.  "Exceptional" conveys both that there is some
reason, but also an expectation of rarity.

So I would write "Unusual Circumstances" etc.

> Exceptional Circumstances
> =
...
> I think there is rough consensus (although rougher than some other
> things) that individual maintainer preference is not in and of itself a
> justification for not using dh.

Having said that, I agree with you here.  We *do* have rough consensus
that pure maintainer preference is not a good reason not to use dh.

> we're not coming after people with pitchforks if they don't use dh.  It
> might simply mean their packages have a bug.  It certainly doesn't mean
> they are obligated to fix that bug, although best practice in our
> community is to work with submitters of patches to review those patches.

I agree, but this will need some delicate handling.

> Is Not Using DH a Bug?
> ==
...
> It doesn't mean you should file that bug and it doesn't mean that you
> should go fix that bug.  We definitely didn't get the kind of support
> we'd be needing for a mass bug filing or anything like that.  It
> wouldn't serve a point.  This isn't atypical.  There are a lot of things
> lintian flags that are technically bugs, but we wouldn't want to mass
> file all lintian tags (even if we could filter out false positives) as
> bugs.
> 
> This paragraph is very much my interpretation.  I'd personally say that
> if you're going to file a bug that a particular package doesn't use dh,
> have a good reason and document it in that bug.  Your reason might be "I
> want to contribute; I'm willing to dedicate time and updating the
> packaging would make it much more appealing to work on."  Often your
> reason will be that there's some other problem, migrating to dh will fix
> that problem, and between the time you're willing to spend and the time
> you hope the maintainer will spend it's worth doing a good job of that
> migration.
> 
> My interpretation of our standard practices is that maintainers have
> wide discretion in which bugs they work on.  That said, if someone
> submits a patch, it's good if you review it.  It's fine to ask them to
> do the necessary testing work and it's fine to hold them to the same
> high standards that you hold yourself to.  If they are less experienced
> with the package it might make sense for them to do tests that make up
> for that experience gap.  None of this  changes any of that or asks
> maintainers to treat bugs about dh differently than other bugs.

My personal position is that I agree with your conclusions here.

I don't feel confident to say what the consensus was.  I have often
experienced serious difficulty even communicating clearly with people
about these matters, let alone coming to agreement on practical steps
in edge cases.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: Do we want to Require or Recommend DH

[Jonas Smedegaard]

Quoting Guillem Jover (2019-05-28 12:46:34)
>   [dh] has proper documentation compared to cdbs which does not, which 
>   I think is the main reason I always found cdbs unappealing as you 
>   need to read its source code making its entire implementation its 
>   interface.

Another common technical¹ reason for disliking cdbs besides its lack of 
documentation is its exposing the make language where dh hides it.


>   This made me realize the same applies to dpkg's make fragments 
>   (which tend to be used with dh too), so I'll be adding man pages for 
>   those during 1.20.x. :)

Nice to know that my total² lack of documenting cdbs has at least been 
helpful in improving _other_ documentation ;-)


 - Jonas


¹ A common _non-technical_ reason is that cdbs is less popular than dh 
and therefore a reason Debian has less contributors than if they were 
given less choice.

² All existing documentation predates my taking lead of cdbs in 2009.

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: Do we want to Require or Recommend DH

[Guillem Jover]

On Mon, 2019-05-13 at 08:33:44 -0400, Sam Hartman wrote:
> As promised, I'd like to start a discussion on whether we want to
> recommend using the dh command from debhelper as our preferred build
> system.

So, here's my take on this. I do use debhelper everywhere, I also use
dh mostly at work, and personally perhaps when it requires no overrides
or very very few of them, or as a replacement for equivs. I do not have
any major problem with debhelper nor dh, I find *both* styles more
pleasant than a hand-crafted debian/rules, but from here it seems like
some in this thread are oblivious to problems with its usage.

Design
--

* debhelper design's principle has been to be a thin layer, most of the
  heavy lifting used to get delegated to other tools and/or lower-layers.
* Both cdbs and dh are based on similar principles, which is to cover as
  much as possible automatically, and override/diverge on the specifics.
  The problem I see though is that they are implemented inside-out, and
  that is one big reason both end up being so much more complex than they
  need to be. This is obviously not a problem of their making!
* Both debhelper and cdbs have a concept of a versioned API. Which is
  nice because it makes it possible to introduce staged changes that
  could otherwise break the world. debhelper seems to be more strict
  about adhering to its API though, and it has proper documentation
  compared to cdbs which does not, which I think is the main reason I
  always found cdbs unappealing as you need to read its source code
  making its entire implementation its interface. This made me realize
  the same applies to dpkg's make fragments (which tend to be used with
  dh too), so I'll be adding man pages for those during 1.20.x. :)
* dh uses an implementation hack to be able to introspect on make.
  This is something that has always slightly put me off. I'm counting
  the days until I can remove a similar but way less clever hack in
  dpkg-buildpackage that tries to introspect on the build targets!

Technical
-

* Not so simple:
  The proclaimed simplicity is only apparently so. Using dh might result
  in shorter debian/rules files, it might (perhaps) result in debian/rules
  listing only the exceptions; all these are one side of simplicity. At
  the same time it makes things more complex because it's yet another
  layer, an abstraction which hides (at the source level) implementation
  details, which you still need to know when having to modify the
  packaging; you need to know how each of dh, debhelper, the various tools
  debhelper uses, make, shell, the upstream build system, the source format
  and patching, the dpkg toolchains, etc. all work, and how they interact.
* Complex is still complex:
  For simple cases, it can indeed be very simple and obvious, for complex
  cases with tons of overrides and make variables, many times it does not
  seem better than an unrolled debhelper debian/rules file.
* Not so uniform:
  Uniformity even with required usage of dh (is not and) would not be
  as uniform as people might want to believe. This is still dependent
  on other helpers used (and which, say dh-exec), on make/shell style,
  on whether you implement the logic in make or shell or an external
  package-specific program (in whatever language) that you call from
  debian/rules, etc.
* No free global change:
  Makes pushing global changes somewhat easier? Yes, but also taking into
  account that any such change in most cases will still require a compat
  level bump, the maintainers making sure it works, and that any override
  has the potential to disable those changes. The main advantage is that
  it might remove the load on maintainers having to recall to do that
  specific change, they just need to make sure it works. It might also
  play in the other direction, with maintainers missing that they need to
  check for these changes "because dh takes care of everything". But this
  just enables a "passive deferred transition", and people that want to
  see a transition done *now* still might need to do tons of work.

Social
--

* It has already naturally gained majority mindshare, because in most
  cases and for most people, it's obviously better than the alternatives,
  I expect this will keep going as debhelper/dh improves, why the need
  for a forced push then for the people who are not yet convinced? This
  always looks suspect to me. For cases like this, I always find it's
  better to convince people with good arguments or better implementation
  and/or documentation, not with force, but dunno.
* A hard requirement means innovation might be thwarted, even if
  innovation is excempt, it would require people to justify beforehand
  (either publicly or to themselves) whether the new solution is going
  to end up being really better than the status quo even before it's
  started.
* Optimizing for NMUs seems wrong, we should be optimizing for long
  term maintainership. Something that can only 

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Russ Allbery]

Jonas Meurer  writes:

> Depending on the software you packages, doing the initial packaging
> already requires a lot of knowledge about library handling, doc build
> systems, makefiles, the filesystem hierarchy standard, language-specific
> toolchains, etc.

> To properly build the package you have to learn either sbuild or
> pbuilder. Which involves understanding and creating chroots/VMs/...

> For proper version controlling, things like git-buildpackage (and/or
> dgit) and the "3.0 (quilt)" format need to be understood.

> And for testing, you need to learn about piuparts, autopkgtest, as well
> as again chroots and/or containers for local testing.

> That's a very high bar for entering the world of Debian packaging.

I completely agree.

How can we fix this?  I've written some documentation of my personal
approaches, but I think documentation, even with step-by-step
instructions, may not be the best way to cut through all that complexity.

When I look at other ecosystems, there often is less diversity than we
have in Debian (newer languages like Rust or Go, for instance, have
converged strongly on a single choice of build system), but perhaps even
more helpfully they've automated a lot of that flow or at least wrapped it
into single tools.

Debian rightfully values its diversity, and while I think pushing us to
think about whether that diversity is still serving a purpose is valuable,
I am sure we don't want to give it up entirely.  But I think there's a
missing gap here for a very opinionated combination of a tool and a
document that says "here is one coherent way to package things for Debian;
you don't have to do things this way at all, but if you don't have
opinions about all this stuff, use this."  I don't think we have that
right now.

By this, I mean way more than just the packaging.  Imagine a world where
you apt-get install decisive (name made up) and then run decisive setup
and it sets up an sbuild environment for you, and then run decisive init
on an upstream directory and it builds a packaging template using dh and
3.0 (quilt) and autogenerates your copyright file using licensecheck.
Running decisive format runs cme fix, running decisive lint runs lintian
and cme check, running decisive check runs puiparts for you, and
autopkgtest, and whatever else we can come up with.  Running decisive
upload runs dgit push for you.  And so forth.

We have some of those pieces already, of course, and writing this sort of
tool is a little fraught for all the tools upstream of them since they'll
get an influx of user bugs from people who don't really understand how the
tool works because they're not using it directly.  So this isn't without
cost.  But I don't think we've ever assembled the *entire* packaging
workflow into a tool like this that has strong opinions and a mandate to
*not* allow every possible variation and instead just focus on being a
very simple and easy-to-understand workflow for the 80% case.  (Obviously
we'd want to try to design it so that it's relatively easy to use
selectively as someone gets better at packaging and makes some alternate
choices.)

To be clear, I personally don't have time to work on this.  But I think it
would fill a valuable niche in our ecosystem, particularly if accompanied
by a supporting document that lays out *why* those tools were chosen and
provides a few pointers to alternatives for someone who starts getting
curious about other ways to do things.  I've seen this kind of approach be
successful elsewhere; the idea is to make common things easy while getting
out of your way if you need to do something uncommon and complicated.

-- 
Russ Allbery (r...@debian.org)   

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Scott Kitterman]

On May 27, 2019 11:50:38 AM UTC, Sam Hartman  wrote:
>> "Scott" == Scott Kitterman  writes:
>Scott> If we want to make not using dh except in certain situations
>Scott> a bug, it seems like something for a policy should kind of
>Scott> item.  We have an existing process for updating policy, so
>Scott> this should probably be kicked over there for further
>Scott> evaluation.
>
>So a discussion like this tends to serve as inputs to discussions in
>the
>rest of the project including policy.  However, I'd hope that delegates
>like the policy editors or the TC would be guided by project consensus.
>
>And If this consensus call stands, my next action as DPL will be to
>talk
>to various people who are impacted about next steps.
>
>The project is not well served by having each team rehash all the
>global
>discussions.
>Absolutely, whoever takes this on on the policy front (TC or policy
>editors) should figure out how to say this in policy.  Absolutely if
>they find factors we failed to consider globally they should come back
>to us.  If they read the discussion and believe I made the wrong
>consensus call for that discussion, they should come back and say so.
>
>But no I think we as a community right here get to decide what's a bug
>if we want to, and I think it's reasonable for us to hold our delegates
>accountable for implementing that decision or having a good reason to
>do
>otherwise.
>
>--Sam

That's fine.  I've no objection to the determination that there is rough 
consensus among the DD's that participate on debian-devel that packages should 
use dh.

My concern is that this is not sufficient to resolve the question of if a 
package buggy that does not do so.  I'd like it also to be clear that because 
lintian flags something like this isn't enough to make a package buggy.

I think the policy process is the next step to implement this aspect of the 
discussion.

Scott K

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Scott Leggett]

On 2019-05-27.12:27, Jonas Meurer wrote:
> Unfortunately I don't have *links* either, but when introducing people
> into the world of Debian packaging recently, I always got the impression
> that they were heavily overwhelmed by the complexity of the Debian
> ecosystem.

As a recently promoted DM, this was definitely my experience. It's taken
me years to become somewhat competent at building packages.

> Depending on the software you packages, doing the initial packaging
> already requires a lot of knowledge about library handling, doc build
> systems, makefiles, the filesystem hierarchy standard, language-specific
> toolchains, etc.
> 
> To properly build the package you have to learn either sbuild or
> pbuilder. Which involves understanding and creating chroots/VMs/...

You also have to even know that sbuild and pbuilder are options, how
those relate to all the other tools like debuild, pdebuild,
dpkg-buildpackage etc..

> For proper version controlling, things like git-buildpackage (and/or
> dgit) and the "3.0 (quilt)" format need to be understood.

This is also not at all obvious for a newcomer.

> And for testing, you need to learn about piuparts, autopkgtest, as well
> as again chroots and/or containers for local testing.

For a long time I didn't use piuparts or autopkgtest due to barely being
aware that they existed. And don't forget the dozens of useful
devscripts like wrap-and-sort that it seems not all packagers know
about.

> That's a very high bar for entering the world of Debian packaging.
>
> My opinion is that more uniformity in packaging practices will bring a
> bit more simplicity as well. Therefore I applaud Sam's initiative to
> require DH whereever it's sensible.

Absolutely. This is an incremental change to simplify one corner of the
sprawling Debian packaging ecosystem. I hope it's just the first of
many recommendations for a "happy path" to Debian packaging nirvana.

> I think that Debian would gain a lot if the vast majority of packages
> were packaged using DH and development would happen in Git on Salsa
> using a common Git format. I agree that there should be exceptions.

Agreed.

-- 
Regards,
Scott Leggett.


signature.asc
Description: PGP signature

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Jonas Meurer]

Hi Sam,

Sam Hartman:
>> "Jonas" == Jonas Meurer  writes:
> Jonas> My opinion is that more uniformity in packaging practices
> Jonas> will bring a bit more simplicity as well. Therefore I applaud
> Jonas> Sam's initiative to require DH whereever it's sensible.
> 
> Hi.
> I'm acting as a facilitator here not as a proponent.
> My job as DPL in this discussion is not to guide us to requiring dh.
> My job is to understand what we as a project want and then to ask the
> appropriate teams in our project to implement this.
> 
> It would be inappropriate for me as DPL to actually be trying to get us
> to require dh until we as a community decide (as I believe we've done)
> that in a lot of circumstances, that's what we want.

Sorry for not being clear here. I didn't mean that you're the one who
*decides* on this question, but that you're the one who took the
initiative to drive forward a project-wide consensus to require DH
whereever sensible.

Cheers
 jonas



signature.asc
Description: OpenPGP digital signature

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Sam Hartman]

> "Jonas" == Jonas Meurer  writes:


Jonas> My opinion is that more uniformity in packaging practices
Jonas> will bring a bit more simplicity as well. Therefore I applaud
Jonas> Sam's initiative to require DH whereever it's sensible.

Hi.
I'm acting as a facilitator here not as a proponent.
My job as DPL in this discussion is not to guide us to requiring dh.
My job is to understand what we as a project want and then to ask the
appropriate teams in our project to implement this.

It would be inappropriate for me as DPL to actually be trying to get us
to require dh until we as a community decide (as I believe we've done)
that in a lot of circumstances, that's what we want.


--Sam

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Alex Mestiashvili]

On 5/27/19 6:29 AM, Andreas Tille wrote:
> On Sun, May 26, 2019 at 07:28:55PM +0200, Vincent Bernat wrote:
>>> We "uphold this reputation" by maintaining many packages, which is
>>> good.
>>
>> Do we? I am now using nix to get packages for stuff not in Debian. Our
>> package count is artificially inflated by *-perl packages, golang-*
>> packages which may not be present in some other distributions. But for
>> some ecosystems, we are severely behind. We may argue we are better on
>> some metrics, but this has nothing to do with the fact we have so many
>> ways to build a package.
> 
> Some Debian Med people are concerned about the droping usage of Debian
> Med packages since people prefer BioConda[1] over it.  There is even a
> scientific paper (I've only seen a printed version not online yet) who
> compares ways to package biology software.  We are way better than other
> distributions - but we are lagging begind BioConda a lot.  We have some
> upstreams who are doing Debian packaging by the help of the Debian Med
> team but that's just a minor fraction.  Lots of BioConda packages are
> maintained by Upstream since they consider it easy.
> 
> In short:  Our "reputation" is scaring people away to favour other
> techniques. 
> 
> Kind regards
> 
>Andreas.
> 
> [1] https://bioconda.github.io/
> 

Debian packaging is also pretty easy if one doesn't care about FHS,
Mutli-Arch, licensing, reproducibility, autopkgtests, hardening and so
on. I believe the list is quite long.

One of the killer features of conda is that conda software can be
managed without root. In some cases it is very important. And of course
that conda is available on osx and windows.

Best,
Alex

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Sam Hartman]

> "Scott" == Scott Kitterman  writes:
Scott> If we want to make not using dh except in certain situations
Scott> a bug, it seems like something for a policy should kind of
Scott> item.  We have an existing process for updating policy, so
Scott> this should probably be kicked over there for further
Scott> evaluation.

So a discussion like this tends to serve as inputs to discussions in the
rest of the project including policy.  However, I'd hope that delegates
like the policy editors or the TC would be guided by project consensus.

And If this consensus call stands, my next action as DPL will be to talk
to various people who are impacted about next steps.

The project is not well served by having each team rehash all the global
discussions.
Absolutely, whoever takes this on on the policy front (TC or policy
editors) should figure out how to say this in policy.  Absolutely if
they find factors we failed to consider globally they should come back
to us.  If they read the discussion and believe I made the wrong
consensus call for that discussion, they should come back and say so.

But no I think we as a community right here get to decide what's a bug
if we want to, and I think it's reasonable for us to hold our delegates
accountable for implementing that decision or having a good reason to do
otherwise.

--Sam

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Jonas Meurer]

Adrian Bunk:
> On Sun, May 26, 2019 at 11:34:39AM +0200, Vincent Bernat wrote:
>> ...
>> We have a reputation of having difficult
>> packaging practices. We uphold this reputation as long as we have so
>> many ways to do the same thing.
> 
>   [citation needed]
> 
> I do honestly not know what statements/comparisons from people outside
> Debian are the basis for this claim, and whether making dh more mandatory
> is even related to them.
> 
> [...]
> 
> Often the most difficult part of packaging are the unique rules the 
> Debian ftp team requires for debian/copyright that are not required in 
> distributions with actual lawyers. That's a completely separate topic.
> 
> It is perfectly possible that there is something else I am not aware
> of, and you are assuming everyone in this discussion is.
> 
> It would therefore be really useful if you could send some links to 
> statements from people outside Debian *why* they consider Debian to
> have difficult packaging practices.

Unfortunately I don't have *links* either, but when introducing people
into the world of Debian packaging recently, I always got the impression
that they were heavily overwhelmed by the complexity of the Debian
ecosystem.

Depending on the software you packages, doing the initial packaging
already requires a lot of knowledge about library handling, doc build
systems, makefiles, the filesystem hierarchy standard, language-specific
toolchains, etc.

To properly build the package you have to learn either sbuild or
pbuilder. Which involves understanding and creating chroots/VMs/...

For proper version controlling, things like git-buildpackage (and/or
dgit) and the "3.0 (quilt)" format need to be understood.

And for testing, you need to learn about piuparts, autopkgtest, as well
as again chroots and/or containers for local testing.

That's a very high bar for entering the world of Debian packaging.

My opinion is that more uniformity in packaging practices will bring a
bit more simplicity as well. Therefore I applaud Sam's initiative to
require DH whereever it's sensible.

I think that Debian would gain a lot if the vast majority of packages
were packaged using DH and development would happen in Git on Salsa
using a common Git format. I agree that there should be exceptions.

Cheers
 jonas



signature.asc
Description: OpenPGP digital signature

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Jonas Smedegaard]

Quoting Andreas Tille (2019-05-27 06:29:05)
> On Sun, May 26, 2019 at 07:28:55PM +0200, Vincent Bernat wrote:
> > > We "uphold this reputation" by maintaining many packages, which is 
> > > good.
> > 
> > Do we? I am now using nix to get packages for stuff not in Debian. 
> > Our package count is artificially inflated by *-perl packages, 
> > golang-* packages which may not be present in some other 
> > distributions. But for some ecosystems, we are severely behind. We 
> > may argue we are better on some metrics, but this has nothing to do 
> > with the fact we have so many ways to build a package.
> 
> Some Debian Med people are concerned about the droping usage of Debian 
> Med packages since people prefer BioConda[1] over it.  There is even a 
> scientific paper (I've only seen a printed version not online yet) who 
> compares ways to package biology software.  We are way better than 
> other distributions - but we are lagging begind BioConda a lot.  We 
> have some upstreams who are doing Debian packaging by the help of the 
> Debian Med team but that's just a minor fraction.  Lots of BioConda 
> packages are maintained by Upstream since they consider it easy.
> 
> In short: Our "reputation" is scaring people away to favour other 
> techniques.

We agree that some get scared away from Debian.  Question is why.

Like rpm, Debian arguably has a single unified _core_ structure: 
debian/rules must be a make file.  hat Sam, me, and Vincent disagree on 
in this subthread is is lack of a unified _framework_ like short-form dh 
sequencer being mandatory is what scares people off.

Does that paper you talk about point to _causes_ Debian packaging being 
more scary? Is it a) complexities related to hardening, cross-building, 
bootstrapping etc. or b) lack of a single¹ unified build framework, or 
c) that Debian is "different" in package naming and code availability 
than other system package managers or or misc. language-specific package 
managers, or d) something else than what you snipped from the beginning 
of this subthread?


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Scott Kitterman]

On May 25, 2019 5:26:47 PM UTC, Sam Hartman  wrote:
>
>Hi. Almost two weeks ago [1] I  started a discussion on whether we
>wanted to increase the strength of our recommendation of the dh
>sequencer from debhelper.
>This message is a consensus call summarizing my reading of the
>discussion.
...
>
>Please send any comments by 2019-06-16.
>Please distinguish cases where you believe I've misread the discussion
>from new contributions intended to change the community's view.
>
>...
>Is Not Using DH a Bug?
>==
>
>It's certainly not an RC bug.  There was some talk of it eventually
>being an RC bug if a new package doesn't use DH (and doesn't fall under
>an exception).  I don't think there's consensus for that today.
>
>It's obviously not a bug if one of the exceptions applies, but once the
>lintian tag exists, overriding that tag sounds to me like best practice
>to document the exception.  (Presumably for things like Haskell we'd
>want Lintian to be smart enough to figure that out on its own)
>
>To some extent I'm extrapolating from implications of the rest of the
>consensus call for the rest of this section.  There was some discussion
>of whether not using dh should be a normal bug, but the comments about
>that were inconsistent with the rest of the discussion.  But if the
>consensus call above that existing packages (absent exceptional
>circumstances) should use dh stands, that's approximately the
>definition
>of a normal bug.  So my reading is that absent exceptional
>circumstances, not using dh is a normal severity bug.
>
>It doesn't mean you should file that bug and it doesn't mean that you
>should go fix that bug.  We definitely didn't get the kind of support
>we'd be needing for a mass bug filing or anything like that.  It
>wouldn't serve a point.  This isn't atypical.  There are a lot of
>things
>lintian flags that are technically bugs, but we wouldn't want to mass
>file all lintian tags (even if we could filter out false positives) as
>bugs.
>
>This paragraph is very much my interpretation.  I'd personally say that
>if you're going to file a bug that a particular package doesn't use dh,
>have a good reason and document it in that bug.  Your reason might be
>"I
>want to contribute; I'm willing to dedicate time and updating the
>packaging would make it much more appealing to work on."  Often your
>reason will be that there's some other problem, migrating to dh will
>fix
>that problem, and between the time you're willing to spend and the time
>you hope the maintainer will spend it's worth doing a good job of that
>migration.
>
>My interpretation of our standard practices is that maintainers have
>wide discretion in which bugs they work on.  That said, if someone
>submits a patch, it's good if you review it.  It's fine to ask them to
>do the necessary testing work and it's fine to hold them to the same
>high standards that you hold yourself to.  If they are less experienced
>with the package it might make sense for them to do tests that make up
>for that experience gap.  None of this  changes any of that or asks
>maintainers to treat bugs about dh differently than other bugs.

On what basis is it a bug of any kind?

A lintian check does not a buggy package make.

Assuming a package that is otherwise working correctly, you have a policy 
compliant, fully functional package.  That seems to me like the very definition 
of not a bug.

If we want to make not using dh except in certain situations a bug, it seems 
like something for a policy should kind of item.  We have an existing process 
for updating policy, so this should probably be kicked over there for further 
evaluation.

Scott K

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Andreas Tille]

On Sun, May 26, 2019 at 07:28:55PM +0200, Vincent Bernat wrote:
> > We "uphold this reputation" by maintaining many packages, which is
> > good.
> 
> Do we? I am now using nix to get packages for stuff not in Debian. Our
> package count is artificially inflated by *-perl packages, golang-*
> packages which may not be present in some other distributions. But for
> some ecosystems, we are severely behind. We may argue we are better on
> some metrics, but this has nothing to do with the fact we have so many
> ways to build a package.

Some Debian Med people are concerned about the droping usage of Debian
Med packages since people prefer BioConda[1] over it.  There is even a
scientific paper (I've only seen a printed version not online yet) who
compares ways to package biology software.  We are way better than other
distributions - but we are lagging begind BioConda a lot.  We have some
upstreams who are doing Debian packaging by the help of the Debian Med
team but that's just a minor fraction.  Lots of BioConda packages are
maintained by Upstream since they consider it easy.

In short:  Our "reputation" is scaring people away to favour other
techniques. 

Kind regards

   Andreas.

[1] https://bioconda.github.io/

-- 
http://fam-tille.de

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Philip Hands]

Adrian Bunk  writes:

...
> Often the most difficult part of packaging are the unique rules the 
> Debian ftp team requires for debian/copyright that are not required in 
> distributions with actual lawyers. That's a completely separate topic.

That seems needlessly snide, and glosses over the fact that we are bound
by constraints that other distributions are not, such as giving a damn
about the license conditions we inflict on our downstreams, which I
suspect the lawyers you allude to are not being retained to contemplate.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Adrian Bunk]

On Sun, May 26, 2019 at 11:34:39AM +0200, Vincent Bernat wrote:
>...
> We have a reputation of having difficult
> packaging practices. We uphold this reputation as long as we have so
> many ways to do the same thing.

  [citation needed]

I do honestly not know what statements/comparisons from people outside
Debian are the basis for this claim, and whether making dh more mandatory
is even related to them.

It is a problem for people making their first contributions to Debian to 
get them into unstable. The problem here is lack of sponsors willing to 
do proper reviews and then uploads. Usually the package in question is 
already using dh.

Often the most difficult part of packaging are the unique rules the 
Debian ftp team requires for debian/copyright that are not required in 
distributions with actual lawyers. That's a completely separate topic.

It is perfectly possible that there is something else I am not aware
of, and you are assuming everyone in this discussion is.

It would therefore be really useful if you could send some links to 
statements from people outside Debian *why* they consider Debian to
have difficult packaging practices.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Vincent Bernat]

 ❦ 26 mai 2019 12:04 +02, Jonas Smedegaard :

>> > * People who make changes across the archive such as enabling 
>> >   hardening, cross-building, bootstrapping, etc benefit 
>> >   significantly from more uniformity in packaging practices.  The 
>> >   time they spend working on packages that use dh is significantly 
>> >   less.  That is, people working on making Debian more reproducible 
>> >   benefit from when we adopt more uniform practices.
>> 
>> It has also been said that non-unformity makes also the life of 
>> everybody more difficult when they look at a random package. This 
>> includes non-DD/DM people. We have a reputation of having difficult 
>> packaging practices. We uphold this reputation as long as we have so 
>> many ways to do the same thing.
>
> We "uphold this reputation" by maintaining many packages, which is
> good.

Do we? I am now using nix to get packages for stuff not in Debian. Our
package count is artificially inflated by *-perl packages, golang-*
packages which may not be present in some other distributions. But for
some ecosystems, we are severely behind. We may argue we are better on
some metrics, but this has nothing to do with the fact we have so many
ways to build a package.
-- 
English literature's performing flea.
-- Sean O'Casey on P. G. Wodehouse


signature.asc
Description: PGP signature

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Jonas Smedegaard]

Quoting Vincent Bernat (2019-05-26 11:34:39)
>  ❦ 25 mai 2019 13:26 -04, Sam Hartman :
> 
> > * People who make changes across the archive such as enabling 
> >   hardening, cross-building, bootstrapping, etc benefit 
> >   significantly from more uniformity in packaging practices.  The 
> >   time they spend working on packages that use dh is significantly 
> >   less.  That is, people working on making Debian more reproducible 
> >   benefit from when we adopt more uniform practices.
> 
> It has also been said that non-unformity makes also the life of 
> everybody more difficult when they look at a random package. This 
> includes non-DD/DM people. We have a reputation of having difficult 
> packaging practices. We uphold this reputation as long as we have so 
> many ways to do the same thing.

We "uphold this reputation" by maintaining many packages, which is good.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Vincent Bernat]

 ❦ 25 mai 2019 13:26 -04, Sam Hartman :

> * People who make changes across the archive such as enabling hardening,
>   cross-building, bootstrapping, etc benefit significantly from more
>   uniformity in packaging practices.  The time they spend working on
>   packages that use dh is significantly less.  That is, people working
>   on making Debian more reproducible benefit from when we adopt more
>   uniform practices.

It has also been said that non-unformity makes also the life of
everybody more difficult when they look at a random package. This
includes non-DD/DM people. We have a reputation of having difficult
packaging practices. We uphold this reputation as long as we have so
many ways to do the same thing.
-- 
Say what you mean, simply and directly.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Consensus Call: Do We Want to Require or Recommend DH; comments by 2019-06-16

[Sam Hartman]

Hi. Almost two weeks ago [1] I  started a discussion on whether we
wanted to increase the strength of our recommendation of the dh
sequencer from debhelper.
This message is a consensus call summarizing my reading of the
discussion.

  [1]
  https://lists.debian.org/msgid-search/tsla7fqjzyv@suchdamage.org

My approach for calling consensus is heavily influenced by RFC 7282 [2],
which talks about some approaches for evaluating rough consensus within
the IETF.  That document is not normative policy there and certainly has
no force here, but the authors have spent a lot of time thinking about
how to understand what an opinionated group of technologists mean when
writing to a mailing list.  In judging consensus, it's much more about
whether issues have been considered and whether the community believes
the issues have been adequately resolved than numbers of people on any
side of a given issue.

  [2] https://tools.ietf.org/html/rfc7282

Please send any comments by 2019-06-16.
Please distinguish cases where you believe I've misread the discussion
from new contributions intended to change the community's view.

At the core of this discussion are a number of key issues that were
reflected in comments throughout the discussion.

* People who make changes across the archive such as enabling hardening,
  cross-building, bootstrapping, etc benefit significantly from more
  uniformity in packaging practices.  The time they spend working on
  packages that use dh is significantly less.  That is, people working
  on making Debian more reproducible benefit from when we adopt more
  uniform practices.

* People who spend time fixing the archive, fixing RC bugs, etc don't
  like to see churn in areas that are already working.  If it's working
  well, don't break it.  Developers doing NMUs do not always perform
  adequate QA of their changes.

* Traditionally we've valued respecting the maintainer's preference
  above most other factors.  In the limit, short of orphaning packages
  or removing maintainers, we don't actually have tools to get
  maintainers to do things they don't want to.  Per our constitution no
  one in Debian is obligated to do anything besides not stand in the way
  of others.  And yet this discussion is about pushing things towards
  uniformity and in a small way away from maintainer preference.

Recommendation
==

There are some exceptions where we think using dh is the wrong choice;
see below.  We have a strong consensus that other than in exceptional
circumstances, new packages should use dh.

There's a weaker consensus that existing packages should use dh, but
migrating working packages to use dh is often not the best use of our
resources and can be harmful when it introduces bugs.  A number of
people spoke against choosing dh for existing packages.  I looked at the
reasons expressed.  A number of these focused on concerns about
introducing bugs.  When I read through all the messages I think it was
fair to interpret a lot of these concerns as asking us to be careful
about how we spend our resources.  It is generally harmful to convert a
package to dh without adequate testing.  It often makes sense to spend
time on something other than converting working packages to dh unless
doing so fixes something else or makes something easier.

But if a maintainer asks if in an ideal world should their package use
dh, the rough consensus of this discussion is that other than
exceptional circumstances, "yes".

Exceptional Circumstances
=

This is not a complete list of reasons not to use dh.  That list will
likely evolve over time, but here are reasons identified in the
discussion.

* If you're actively working on something new, innovation is still
  something we care about.  We're not trying to close down development
  of the next great thing.

* Cdbs has a few features that dh doesn't have.  The Haskell tools in
  cdbs ar critical to our current Haskell infrastructure.  Cdbs flavors
  don't seem to have a dh analogue.  For examples compare
  https://salsa.debian.org/multimedia-team/snd/blob/master/debian/rules
  to
  https://salsa.debian.org/multimedia-team/soundscaperenderer/blob/master/debia
  n/rules .  If cdbs is doing something for you, then it probably makes
  sense to stay with it for now.  But it sounds like there's discussion
  of retiring cdbs longterm.  So if you're using cdbs for things dh is
  perfectly good at, the cdbs exception is much weaker and possibly may
  not apply.

* Consistency with teams.  If your team is using something else, then it
  might be worth having a discussion within the team about whether
  that's still the best choice, but consistency within teams is
  important.

* We need to go back and check if there are any bootstrapping concerns
  that should affect build system decisions for specific packages.

There was strong consensus that we should have a lintian tag that can be
overridden to indicate why a package is not using debhelper.

Re: NMUs: Do we want to Require or Recommend DH

[Sean Whitton]

Hello,

On Fri 24 May 2019 at 04:01PM +02, Lucas Nussbaum wrote:

> Hi,
>
> On 14/05/19 at 14:30 -0400, Sam Hartman wrote:
>> I think there's a fairly clear consensus emerging that it's worth having
>> things to check when making a build system conversion.  Looking at
>> debdiff, ditherscope and reproducibility of the build all appear to be
>> important things to consider in such a case.
>>
>> So, I think there is an emerging consensus against the idea of people
>> NMUing a package simply to convert it to dh.
>>
>> First, I'd like to explicitly call for any last comments from people who 
>> would
>> like to see us permit NMUs simply to move packages toward dh.  Are there
>> any cases in which such an NMU should be permitted?
>
> Our NMU policy (Sec 5.11.1 of developers-reference[1]) tries hard to

Note that nothing in dev-ref is binding on developers, so I think it's a
bit misleading to use the term 'policy'.  All of dev-ref is guidelines.

Otherwise, I think your summary of what dev-ref says about NMUs is
correct.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: NMUs: Do we want to Require or Recommend DH

[Lucas Nussbaum]

Hi,

On 14/05/19 at 14:30 -0400, Sam Hartman wrote:
> I think there's a fairly clear consensus emerging that it's worth having
> things to check when making a build system conversion.  Looking at
> debdiff, ditherscope and reproducibility of the build all appear to be
> important things to consider in such a case.
> 
> So, I think there is an emerging consensus against the idea of people
> NMUing a package simply to convert it to dh.
> 
> First, I'd like to explicitly call for any last comments from people who would
> like to see us permit NMUs simply to move packages toward dh.  Are there
> any cases in which such an NMU should be permitted?

Our NMU policy (Sec 5.11.1 of developers-reference[1]) tries hard to
give some standards of when and how it's acceptable to do an NMU. It is
complex, but in the end, I think that it boils down to:
  NMUs are always permitted, but discouraged in some (many?) cases, and
  extensive use of the DELAYED queue is recommended.

It also explicitely discourages NMUs for packaging style changes:
> Fixing cosmetic issues or changing the packaging style (e.g. switching
> from cdbs to dh) in NMUs is discouraged.

Do you want to change this and explicitely forbid NMUs for converting to
dh? I think that the current policy is quite balanced (but I'm biaised
since I contributed to its adoption a long time ago :) ). I also think
that we should trust the judgement of DDs, and that completely
forbidding some changes via NMUs would be a regression compared to the
current policy.

- Lucas

[1] https://www.debian.org/doc/manuals/developers-reference/ch05.en.html#nmu

Re: Do we want to Require or Recommend DH

[Steve Langasek]

On Tue, May 21, 2019 at 05:46:11AM -0400, Reinhard Tartler wrote:
> On Tue, May 21, 2019, 03:41 Vincent Bernat  wrote:.

> > Is there an example of a package where dh cannot be used? Making 96% of
> > packages simpler and 4% of packages moderately more complex seems to be
> > a good argument to uniformize our packaging practices towards dh.
> > --
> > Use the fundamental control flow constructs.
> > - The Elements of Programming Style (Kernighan & Plauger)

> I looked yesterday at the boxbackup source package and contemplated
> converting it to dh from debhelper. I decided to not, because I'm having a
> hard time seeing a significant simplification potential. Maybe I'm just not
> seeing it?

> Note that I orphaned the package quite some time ago, so feel welcome to
> simplify it as much as possible on Salsa.

Well, I took a stab at this, but wasted far more time dealing with crlf
nonsense on the vcxproj.user files in the git tree than on the actual
conversion to dh(1).

So here's a patch which shows that even in its most direct form, converting
this package to dh results in a shorter debian/rules which trims a fair
amount of boilerplate.  Further improvements are definitely possible.

Simplifying debian/rules to only need to declare the exceptions, and not the
boilerplate, is always a win.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developer   https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
commit a5e8d7c7c2b16ade5197c497491cb73e82784c19
Author: Steve Langasek 
Date:   Tue May 21 10:38:49 2019 -0700

Convert to dh(1).

diff --git a/debian/changelog b/debian/changelog
index 4ff829f4..97e4dca9 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+boxbackup (0.13~~git20180819.g2f5b556-2) UNRELEASED; urgency=medium
+
+  * Convert to dh(1).
+
+ -- Steve Langasek   Tue, 21 May 2019 10:38:32 -0700
+
 boxbackup (0.13~~git20180819.g2f5b556-1) unstable; urgency=medium
 
   * New upstream pre-release
diff --git a/debian/clean.sh b/debian/clean.sh
index 90a30513..1482640c 100644
--- a/debian/clean.sh
+++ b/debian/clean.sh
@@ -1,8 +1,6 @@
 #!/bin/sh
 
-rm -rf debug/
 rm -rf local/
 rm -rf parcels/
-rm -rf release/
 rm -rf docs/htmlguide/
 rm -rf docs/man/
diff --git a/debian/rules b/debian/rules
index b67b9519..a44d062e 100755
--- a/debian/rules
+++ b/debian/rules
@@ -5,73 +5,56 @@
 
 include /usr/share/dpkg/pkg-info.mk
 
-# These are used for cross-compiling and for saving the configure script
-# from having to guess our platform (since we know it already)
-DEB_HOST_GNU_TYPE   ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE)
-DEB_BUILD_GNU_TYPE  ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)
-
 TMP:=$(CURDIR)/debian/tmp
 
-ifneq (,$(findstring debug,$(DEB_BUILD_OPTIONS)))
-	CFLAGS += -g
-endif
-ifeq (,$(findstring nostrip,$(DEB_BUILD_OPTIONS)))
-	INSTALL_PROGRAM += -s
-endif
+%:
+	dh $@
 
-configure: configure-stamp
-configure-stamp:
-	dh_testdir
+override_dh_autoreconf:
 	echo "$(DEB_VERSION_UPSTREAM_REVISION)" > VERSION.txt
 	echo "boxbackup" >> VERSION.txt
-	sh -x ./bootstrap
-	./configure $(DEB_EXTRA_CONFIG_FLAGS) LDFLAGS="-Wl,--as-needed"
-	touch configure-stamp
+	dh_autoreconf sh -- -x ./bootstrap
+
+configure: override_dh_autoreconf
+	:
+
+override_dh_auto_configure: configure
+	dh_auto_configure -- LDFLAGS="-Wl,--as-needed"
 
-build-stamp: configure-stamp
-	dh_testdir
-	$(MAKE) V=1
+override_dh_auto_build:
+	dh_auto_build -- V=1
+
+override_dh_auto_test:
 # the testsuite is only really maintained on i386 and amd64
 ifneq (,$(filter $(DEB_HOST_ARCH),i386 amd64))
 ifeq (,$(findstring nocheck,$(DEB_BUILD_OPTIONS)))
 	./runtest.pl ALL
 endif
 endif
-	touch build-stamp
 
 docs/docbook/instguide.pdf:
 	$(MAKE) -C docs instguide
 	cd docs/docbook && docbook2pdf instguide.xml
 
-docs/docbook/adminguide.pdf: configure-stamp
+docs/docbook/adminguide.pdf: override_dh_auto_configure
 	$(MAKE) -C docs adminguide
 	cd docs/docbook && docbook2pdf adminguide.xml
 
 docs: docs/docbook/instguide.pdf docs/docbook/adminguide.pdf
 	$(MAKE) -C docs manpages
 
-build-arch: build-stamp
-build-indep: docs
-
-build: build-arch build-indep
+build-arch: docs
 
-clean:
-	dh_testdir
-	dh_testroot
-	dh_clean build-stamp configure-stamp
+override_dh_auto_clean:
 	echo "USE_SVN_VERSION" > VERSION.txt
 	echo "boxbackup" >> VERSION.txt
-	[ ! -f Makefile ] || make clean
-	sh debian/clean.sh
-	dh_clean config.log config.status 
+	dh_auto_clean
 
-install: DH_OPTIONS=
-install: build
-	dh_testdir
-	dh_testroot
-	dh_prep
-	dh_installdirs
+override_dh_clean:
+	sh debian/clean.sh
+	dh_clean
 
+override_dh_install:
 	mkdir -p $(TMP)/etc/logcheck/ignore.d.workstation
 	mkdir -p $(TMP)/etc/logcheck/ignore.d.server
 	install -m 644 debian/boxbackup-server.logcheck.ignore 

Re: Do we want to Require or Recommend DH

[Sam Hartman]

Reinhard challenged me offlist to look at whether  boxbackup would
actually be more maintainable with dh than with its current use of
debhelper.

Here are things I noticed that I wouldn't have to think about with dh.
The package may be correct, but if I were trying to maintain the package
I'd need to evaluate all of these.  These are more or less just as
complicated with dh, but the entire point is that I only need to
remember one complicated thing rather than lots.

* The package uses findstring rather than filter when searching
  DEB_BUILD_OPTIONS.  I know that filter is preferred I think because
  findstring will do substring matches.  I need to remember why that's
  bad for DEB_BUILD_OPTIONS and consider whether it's a bug.

* What's with the debug DEB_BUILD_OPTION.  I'd need to make sure that by
  default -g is included per policy.

* It doesn't look like noopt is explicitly handled. Does dpkg-info.mk do
  that for me?  I can't remember; better go look.  It's probably going
  to be a combination of dpkg-buildflags, the makefile fragment,
  configure's handling of environment variables, etc.  Yeah, it's just
  as complicated with dh, but I have high confidence dh gets this right.

* It looks like the version and configure handling are a little prickly
  in this package; I get the impression that the configure script may
  not be entirely well behaved.  Or perhaps it's just maintainer style.
  With dh that would stand out one way or the other and be obvious.

*There's an init script and a systemd unit.  That'll break when this
 goes to compat 11 or 12.  (Well really that situation's going to be
 broken with compat 11 anyway), but getting the advantages of the
 systemd handling in compat 12 would be better with dh.

* As Simon pointed out I'd need to go check the ordering of all the
  debhelper calls.  I'm fairly sure this is correct, but it would be
  nice to not need to worry about that.

So, yes, this package would be much more appealing to third parties who
need to look at it after the dh conversion.
I fully acknowledge that conversion is real work.  Each of the bullet
points I enumerate above and several others would need to get checked.

We've talked about how when doing such conversions we should check and
make sure we get identical binaries out.  Here I'm not sure that's
right.  I'm not sure the existing buildflag, debug, optimization and
stripping behavior is right.  So, the conversion might be harder to the
extent I'm needing to check correctness of the existing packaging.

Am I jumping up and down to go do that work on this package?  Absolutely
not.
do I hope whoever adopts it does that work?  Yes.

Re: Do we want to Require or Recommend DH

[Ian Jackson]

Reinhard Tartler writes ("Re: Do we want to Require or Recommend DH"):
> I looked yesterday at the boxbackup source package and contemplated
> converting it to dh from debhelper. I decided to not, because I'm
> having a hard time seeing a significant simplification
> potential. Maybe I'm just not seeing it?

I think the dh-based rules file would be about 2/3 the length.  I look
particularly at the binary-arch target and I think "why are we listing
all of this explicitly, and specifying an order".  And what is that
DEB_HOST_GNU_TYPE for ?  Seems like hand-coded cross-compilation
support.  I bet dh would Just DTRT.  etc.

> Note that I orphaned the package quite some time ago, so feel welcome to
> simplify it as much as possible on Salsa.

Heh.  Well, like Sam, I am responding because I failed my saving throw
against XKCD-386 [1], not to try to badger you into doing work on a
package you don't much care about any more.  I don't feel converting
this package, even as an example to others, is a thing that I want to
spend my time on...

Regards,
Ian.

[1] https://xkcd.com/386/

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: Do we want to Require or Recommend DH

[Sam Hartman]

> "Reinhard" == Reinhard Tartler  writes:

Reinhard>I looked yesterday at the boxbackup source package and
Reinhard> contemplated converting it to dh from debhelper. I decided
Reinhard> to not, because I'm having a hard time seeing a
Reinhard> significant simplification potential. Maybe I'm just not
Reinhard> seeing it?

This has been said earlier in the discussion, but to summarize.
Even if you don't simplify the package, uniformity has significant
value.
First, it makes it easier for others to analyze correctness.
Second, it makes it easier to adopt future changes.

I'm not asking you to go mess with a package you orphaned, simply
summarizing an answer that it sounds like you missed.

--Sam

Re: Do we want to Require or Recommend DH

[Reinhard Tartler]

On Tue, May 21, 2019, 03:41 Vincent Bernat  wrote:.

>
> Is there an example of a package where dh cannot be used? Making 96% of
> packages simpler and 4% of packages moderately more complex seems to be
> a good argument to uniformize our packaging practices towards dh.
> --
> Use the fundamental control flow constructs.
> - The Elements of Programming Style (Kernighan & Plauger)
>

I looked yesterday at the boxbackup source package and contemplated
converting it to dh from debhelper. I decided to not, because I'm having a
hard time seeing a significant simplification potential. Maybe I'm just not
seeing it?

Note that I orphaned the package quite some time ago, so feel welcome to
simplify it as much as possible on Salsa.

Best
-rt

>

Re: Do we want to Require or Recommend DH

[Adrian Bunk]

On Tue, May 21, 2019 at 09:40:38AM +0200, Vincent Bernat wrote:
>  ❦ 19 mai 2019 23:53 -04, Sam Hartman :
> 
> > >> As promised, I'd like to start a discussion on whether we want to
> > >> recommend using the dh command from debhelper as our preferred
> > >> build system.
> >
> > Sean> For those who haven't seen it, the original author of dh, Joey
> > Sean> Hess, has just published a blog post relevant to the
> > Sean> discussion we're having:
> >
> > I've read the blog post.
> >
> > I think Joey's points are very consistent with our discussion here.
> 
> Is there an example of a package where dh cannot be used? Making 96% of
> packages simpler and 4% of packages moderately more complex seems to be
> a good argument to uniformize our packaging practices towards dh.

This would be less uniform that it sounds.

Technically a package that overrides *all* dh targets with something 
completely different is using dh.

And the result of someone spending several man months on converting
the Debian Haskell scripts to dh might be relatively close to that.

As an example, look at
https://tracker.debian.org/media/packages/h/haskell-sandi/rules-0.4.2-2

You can see by the path that /usr/share/cdbs/1/class/hlibrary.mk
uses CDBS, but if it would use dh instead this wouldn't make
a huge difference here.

As developer you are not usually working directly with low-level
CDBS or dh functionality, what you are using is the higher level
haskell-devscripts functionality built on top of that.

If you ever have to use low-level CDBS or dh functionality directly, 
then things can get really messy since you might break what
haskell-devscripts is doing.

DEB_ENABLE_TESTS is an API provided by haskell-devscripts.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Do we want to Require or Recommend DH

[Jonas Smedegaard]

Quoting Vincent Bernat (2019-05-21 09:40:38)
>  ❦ 19 mai 2019 23:53 -04, Sam Hartman :
> 
> > >> As promised, I'd like to start a discussion on whether we 
> > >> want to recommend using the dh command from debhelper as our 
> > >> preferred build system.
> >
> > Sean> For those who haven't seen it, the original author of dh, Joey
> > Sean> Hess, has just published a blog post relevant to the
> > Sean> discussion we're having:
> >
> > I've read the blog post.
> >
> > I think Joey's points are very consistent with our discussion here.
> 
> Is there an example of a package where dh cannot be used? Making 96% 
> of packages simpler and 4% of packages moderately more complex seems 
> to be a good argument to uniformize our packaging practices towards 
> dh.

I guess the answer to your question is "no" but please beware that this 
thread is about what is _convenient_ (for the package maintainer and/or 
for the majority of Debian), not what is _possible_,

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: Do we want to Require or Recommend DH

[Vincent Bernat]

 ❦ 19 mai 2019 23:53 -04, Sam Hartman :

> >> As promised, I'd like to start a discussion on whether we want to
> >> recommend using the dh command from debhelper as our preferred
> >> build system.
>
> Sean> For those who haven't seen it, the original author of dh, Joey
> Sean> Hess, has just published a blog post relevant to the
> Sean> discussion we're having:
>
> I've read the blog post.
>
> I think Joey's points are very consistent with our discussion here.

Is there an example of a package where dh cannot be used? Making 96% of
packages simpler and 4% of packages moderately more complex seems to be
a good argument to uniformize our packaging practices towards dh.
-- 
Use the fundamental control flow constructs.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: NMUs: Do we want to Require or Recommend DH

[Jonathan Dowland]

On Wed, May 15, 2019 at 03:22:16PM +0200, Andreas Tille wrote:

On Wed, May 15, 2019 at 02:09:14PM +0200, Benjamin Drung wrote:

Am Mittwoch, den 15.05.2019, 11:31 +0200 schrieb Enrico Zini:



Or introduce a lintian check for not using dh. Then the maintainer
could override lintian and document the reason for it in the override.


... but I think the documented lintian override is better since
it is more connected to code than free text.


I agree; also, the new lintian check/warning showing up in places like
Tracker may help adoption of the documentation (as an override) to
happen across the distribution more quickly.


--
Jonathan Dowland

Re: NMUs: Do we want to Require or Recommend DH

[Adrian Bunk]

On Wed, May 15, 2019 at 11:31:46AM +0200, Enrico Zini wrote:
>...
>  - if a package has had an inactive and unresponsive maintainer for a
>long time, it would indeed be a case for salvaging.
> 
>I could however imagine someone having enough energy to dust off old
>packages in the archive, while not having enough energy to pick up
>maintenance of lots of old dusty packages.

Does the person have the energy to properly test that the changes don't
cause any regressions?

Uploading invasive changes in packages you don't use is usually
a bad idea, and if it ain't broken don't fix it.

>I would consider the idea of salvaging+orphaning, like following the
>salvaging procedure but setting the maintainer to qa instead.
>...

Such salvaging+orphaning instead of following the proper MIA process
would be abusive behaviour.

Even more if it is just for a packaging change that would be expected
to result in the same binary packages.

In the worst case you get a pissed off maintainer leaving the project,
and an orphaned package with some RC regression caused by the change.
Without any actual benefits of the whole change.

> With a consensus of having dh as the default build system, and the
> understanding that some packages have good reasons not to use dh, I'd
> like a way to tell when a package is not using dh for a reason, from
> when a package is not using dh because the maintainer has not gotten
> around to it yet.
>
> I'd propose to recommend dh as the default build system, and document in
> README.source if there are reasons to use something else.
>
> At that point, one could look at README.source to see if changing build
> system would be an possible strategy for fixing bugs in an NMU.
>...

There's a much easier (and faster!) solution that has already been used 
in the past:

For migrating away from dpatch someone made the changes to the packages,
and then sent the debdiffs to the BTS.

If the maintainer liked the change, it was applied.

If the maintainer was MIA, the patch was applied whenever the package 
was orphaned later.

For a standard conversion to 3-line dh not much is lost if some of the 
patches end up being rejected.

For non-trivial conversions it also sets the right incentive that the 
person doing the change contacts the maintainer first, NMU plus revert 
of the NMU would be a solution inferior to asking the maintainer before 
doing plenty of work.

> Enrico

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Do we want to Require or Recommend DH

[Sam Hartman]

> "Sean" == Sean Whitton  writes:

Sean> Hello,
Sean> On Mon 13 May 2019 at 08:33AM -04, Sam Hartman wrote:

>> As promised, I'd like to start a discussion on whether we want to
>> recommend using the dh command from debhelper as our preferred
>> build system.

Sean> For those who haven't seen it, the original author of dh, Joey
Sean> Hess, has just published a blog post relevant to the
Sean> discussion we're having:

I've read the blog post.

I think Joey's points are very consistent with our discussion here.

I hope it's not too much of a spoiler for my upcoming summary, but my
view is that we identified several areas in the discussion where dh is
not the right solution.
We'll probably identify more over time.
There will be good reasons for not using dh in a particular situation.

--Sam

Re: Do we want to Require or Recommend DH

[Sean Whitton]

Hello,

On Mon 13 May 2019 at 08:33AM -04, Sam Hartman wrote:

> As promised, I'd like to start a discussion on whether we want to
> recommend using the dh command from debhelper as our preferred build
> system.

For those who haven't seen it, the original author of dh, Joey Hess, has
just published a blog post relevant to the discussion we're having:

I added `dh` to debhelper a decade ago, and now Debian is
considering making use of `dh` mandatory. Not being part of Debian
anymore, I'm in the position of needing to point out something
important about it anyway. So this post is less about pointing in a
specific direction as giving a different angle to think about
things.

http://joeyh.name/blog/entry/80_percent/

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: Do we want to Require or Recommend DH

[Sam Hartman]

Hi.
It looks like the discussion is simmering down.

My plan is to read over it all and come up with a consensus call
indicating areas where I believe we have consensus and what I think that
consensus is.

After I post that people will have an opportunity  to comment on whether
I've judged consensus correctly.

Then we'll go from there.

It's going to take me a few days to write up such a consensus call.  I'm
working on other things too, and reading a mailing list discussion like
this to judge consensus requires care.

--Sam

Re: Do we want to Require or Recommend DH

[Andreas Tille]

Hi Tollef,

On Wed, May 15, 2019 at 09:54:50PM +0200, Tollef Fog Heen wrote:
> ]] Andreas Tille 
> 
> > Can you give an example for a package that has a non-dh rules file
> > "working for years" that gives as a result a package with no lintian
> > warnings without changing this d/rules file?
> 
> If you're talking about the binary package, fortunes-bofh-excuses.  It
> has some lintian warnings on the source package (primarily because it's
> not been uploaded for more than ten years) of which only two out of six
> warnings would be handled automatically by dh.

>From a developer point of view I'm talking about *any* lintian warnings
so for sure also about source package warnings.
 
> (Yes, also not an important package in the grand scheme of things, and
> I'm not disagreeing that using dh is a good thing overall.  I should
> probably upload it just to get some of the dust off it.)

:-)

Kind regards

Andreas. 

-- 
http://fam-tille.de

Re: Do we want to Require or Recommend DH

[Sean Whitton]

Hello,

On Tue 14 May 2019 at 12:30PM +01, Ian Jackson wrote:

> Sean Whitton writes ("Re: Do we want to Require or Recommend DH"):
>> I agree with Scott's emphasis on the distinction between new and
>> existing packages.  Perhaps application of the distinction could be
>> extended: perhaps there are other things that we could require of new
>> packages, while creating no expectation that these requirements be met
>> of older packages.
>>
>> In general, if a policy requirement or convention should apply to new
>> packages, then it should apply to existing packages, too.  But
>> specifically where applying the requirement to an existing package is
>> hugely more work than applying it to a new package, perhaps the
>> requirement ought to be limited to new packages alone.
>
> There is a big distinction between recommendations which directly
> affect the functioning of the binary packages, and recommendations
> which make future development easier.

Good point.

> For the latter - and use of dh is an example - it makes a lot of sense
> to make the recomendations stronger for new packages.
>
> I also think it would be very valuable for policy to recommend things
> like this as the usual case, without mandating them.  If for any
> reason the maintainer doesn't want to use dh, then they can leave a
> wontfix bug open (or something) to document the reasons.

It might be useful to note here that patches to debian-policy which
suggest using particular tools, without using "recommends", "should" or
"must" (etc.), do not need to go through the Policy Changes Process, and
can just be applied immediately.

For example, in Policy we currently have:

The easiest way to call "dpkg-shlibdeps" correctly is to use a
package helper framework such as debhelper. If you are using
debhelper, the "dh_shlibdeps" program will do this work for you.

and

Note that under some circumstances it may be useful to install a
shared library unstripped, for example when building a separate
package to support debugging.  The debhelper *dh_strip`* tool can
create such packages automatically.

This sort of text, which is informative rather than normative, does not
need to go through the consensus-building process.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: Do we want to Require or Recommend DH

[Tollef Fog Heen]

]] Andreas Tille 

> Can you give an example for a package that has a non-dh rules file
> "working for years" that gives as a result a package with no lintian
> warnings without changing this d/rules file?

If you're talking about the binary package, fortunes-bofh-excuses.  It
has some lintian warnings on the source package (primarily because it's
not been uploaded for more than ten years) of which only two out of six
warnings would be handled automatically by dh.

(Yes, also not an important package in the grand scheme of things, and
I'm not disagreeing that using dh is a good thing overall.  I should
probably upload it just to get some of the dust off it.)

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Re: QA expectations (Was: Do we want to Require or Recommend DH)

[Helmut Grohne]

On Wed, May 15, 2019 at 09:28:59AM +0100, Simon McVittie wrote:
> Prior art: Ubuntu already does this, gating the transition with a version
> of Debian's testing migration scripts that has been configured for a 0 day
> delay for all urgencies.

Yes. Colin Watson even had a talk about this in Vaumarcus.
http://penta.debconf.org/dc13_schedule/events/1028.en.html Copying the
strategy to Debian failed to gain traction thus far. I wonder whether we
could do this opt-in (or maybe as some external service that forwards
tested packages to the archive) anyhow. I have little clue about what
would be required to implement it in the archive.

> Ubuntu is more able to do this than Debian, because Ubuntu's slowest,
> least reliable and least-well-supported architectures are faster, more
> reliable and better-supported than Debian's. I'm not sure that we really
> want to be waiting for important fixes (especially in large packages
> like compilers, web browsers and the kernel) to build successfully on
> mips(el), or requiring that their build-time tests have few enough race
> conditions to be successful even on slower architectures, before they
> can reach the part of the archive that developers use in practice?

Given my experience with Multi-Arch: same skews in unstable, it does not
appear to me that packages get stuck for that long. If they do, that's
due to a FTBFS (which is exactly the thing we want to prevent from
entering unstable). buildd speed is a concern for other reasons, so if
an architecture fails to keep up for a longer time, it should likely be
demoted to ports.

Helmut

Re: Do we want to Require or Recommend DH

[Jonathan Dowland]

On Mon, May 13, 2019 at 05:58:47PM +0200, Thomas Goirand wrote:

Why would one want to switch that one to something else? The package,
basically, consists of a shell script and a man page only. The
minimalism of this package doesn't require an over-engineered dh
sequencer, does it?


I maintain one of the simplest possible packages (in non-free),
doom-wad-shareware, that is even simpler: it consists of three files total:

   /usr/share/doc/doom-wad-shareware/changelog.Debian.gz
   /usr/share/doc/doom-wad-shareware/copyright
   /usr/share/games/doom/doom1.wad

For the source package, I thought "why do I need debhelper for such a simple
package". And so I did things by hand instead¹, and I still screwed something
up².

This is clearly a stupid case of premature optimisation, yak shaving, etc.; I
suspect many other instances of "why bother for such a simple package" in the
archive have elements of these too.

(An unrelated, but amusing mess-up in this trivial package: for the first 11
years, there was a version mismatch between what was actually in the .deb and
what the version claimed)

1. https://sources.debian.org/src/doom-wad-shareware/1.9.fixed-2/debian/rules/
2. 
https://tracker.debian.org/news/449441/accepted-doom-wad-shareware-19fixed-2-source-all/

--

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Jonathan Dowland
⢿⡄⠘⠷⠚⠋⠀ https://jmtd.net
⠈⠳⣄ Please do not CC me, I am subscribed to the list.

Re: Do we want to Require or Recommend DH

[Jonathan Dowland]

On Tue, May 14, 2019 at 02:27:30PM +0800, Paul Wise wrote:

I think conversion to dh should only be done when doing hostile
hijacking of packages, salvaging packages, adopting packages,
orphaning packages or team/maintainer uploads and only if the person
doing the conversion builds the package twice (with and without dh),
inspects the resulting changes to the binary packages with diffoscope
and is confident that each change is appropriate.


Here you state your opinion but you haven't provided any rationale for
it.

--

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Jonathan Dowland
⢿⡄⠘⠷⠚⠋⠀ https://jmtd.net
⠈⠳⣄ Please do not CC me, I am subscribed to the list.

Re: NMUs: Do we want to Require or Recommend DH

[Andreas Tille]

Hi,

On Wed, May 15, 2019 at 02:09:14PM +0200, Benjamin Drung wrote:
> Am Mittwoch, den 15.05.2019, 11:31 +0200 schrieb Enrico Zini:
> > I'd propose to recommend dh as the default build system, and document
> > in README.source if there are reasons to use something else.
> > 
> > At that point, one could look at README.source to see if changing
> > build system would be an possible strategy for fixing bugs in an NMU.

The README.source idea is good ...
 
> Or introduce a lintian check for not using dh. Then the maintainer
> could override lintian and document the reason for it in the override.

... but I think the documented lintian override is better since
it is more connected to code than free text.

Kind regards

Andreas.

-- 
http://fam-tille.de

Re: NMUs: Do we want to Require or Recommend DH

[Ian Jackson]

Enrico Zini writes ("Re: NMUs: Do we want to Require or Recommend DH"):
> I'd propose to recommend dh as the default build system, and document in
> README.source if there are reasons to use something else.
> 
> At that point, one could look at README.source to see if changing build
> system would be an possible strategy for fixing bugs in an NMU.

This is a great idea.  Obviously such a policy/process change would
have to come with a significant lead time so that everyone would have
a chance to do an upload with an appropriate README.source.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: NMUs: Do we want to Require or Recommend DH

[Benjamin Drung]

Am Mittwoch, den 15.05.2019, 11:31 +0200 schrieb Enrico Zini:
> I'd propose to recommend dh as the default build system, and document
> in README.source if there are reasons to use something else.
> 
> At that point, one could look at README.source to see if changing
> build system would be an possible strategy for fixing bugs in an NMU.

Or introduce a lintian check for not using dh. Then the maintainer
could override lintian and document the reason for it in the override.

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

1&1 IONOS Cloud GmbH | Greifswalder Str. 207 | 10405 Berlin | Germany
E-mail: benjamin.dr...@cloud.ionos.com | Web: www.ionos.de

Head Office: Berlin, Germany
District Court Berlin Charlottenburg, Registration number: HRB 125506 B
Executive Management: Christoph Steffens, Matthias Steinberg, Achim
Weiss

Member of United Internet

dh_testroot usage is still always required (was Re: Do we want to Require or Recommend DH)

[Guillem Jover]

On Wed, 2019-05-15 at 09:18:26 +0100, Simon McVittie wrote:
> It uses dh_testroot, so it probably can't have Rules-Requires-Root: no,
> and needs to be built as (fake)root indefinitely - even though a package
> this simple can almost certainly be built correctly without fakeroot.

You've mention this twice in the thread, and this is incorrect. Actually,
removing dh_testroot would be just harmful. Whether a package is to be
built with R³ or not depends in most part on the builder environment,
not the packaging. If you use a new enough dpkg-buildpackage, then it
will enable it if the packaging allows it, otherwise it should require
(fake)root; or if calling debian/rules by hand, even when the packaging
has specified it can be built with no R³.

(See dpkg itself for an example of this.)

Thanks,
Guillem

Re: NMUs: Do we want to Require or Recommend DH

[Jonas Smedegaard]

Quoting Enrico Zini (2019-05-15 11:31:46)
> On Tue, May 14, 2019 at 02:30:52PM -0400, Sam Hartman wrote:
> 
> > How do we feel about people making build system conversions when 
> > those conversion make it easier to fix some other bug that they are 
> > fixing as part of an NMU?
> > That is, imagine that a package is mishandling the combination of 
> > systemd units and an init script.  As someone preparing an NMU, is 
> > it reasonable to move to dh compat 12 from some other build system 
> > if I believe doing so will make it easier for me to fix the bug and 
> > verify the fix?
> 
> I see various scenarios:
> 
>  - if a package is generally actively maintained, except the 
>maintainer is currently unresponsive for some reason and there is a 
>RC bug to fix, I could understand frowning upon a build system 
>conversion in an NMU.
> 
>  - if a package has bugs that can all be fixed trivially with a build 
>system conversion, I would see no reason not to do such a 
>conversion, even in an NMU.
> 
>  - a change of build system in a complex package would be more 
>controversial than in a trivial package.
> 
>  - if a package has had an inactive and unresponsive maintainer for a 
>long time, it would indeed be a case for salvaging.
> 
>I could however imagine someone having enough energy to dust off 
>old packages in the archive, while not having enough energy to pick 
>up maintenance of lots of old dusty packages.
> 
>I would consider the idea of salvaging+orphaning, like following 
>the salvaging procedure but setting the maintainer to qa instead.
> 
>  - I'd say that orphaned packages can be uncontroversially be 
>converted to dh.

Very well said.  I agree ith all of it.


> With a consensus of having dh as the default build system, and the 
> understanding that some packages have good reasons not to use dh, I'd 
> like a way to tell when a package is not using dh for a reason, from 
> when a package is not using dh because the maintainer has not gotten 
> around to it yet.
> 
> I'd propose to recommend dh as the default build system, and document 
> in README.source if there are reasons to use something else.
> 
> At that point, one could look at README.source to see if changing 
> build system would be an possible strategy for fixing bugs in an NMU.

Great suggestion!


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: NMUs: Do we want to Require or Recommend DH

[Enrico Zini]

On Tue, May 14, 2019 at 02:30:52PM -0400, Sam Hartman wrote:

> How do we feel about people making build system conversions when those
> conversion make it easier to fix some other bug that they are fixing as
> part of an NMU?
> That is, imagine that a package is mishandling the combination of
> systemd units and an init script.  As someone preparing an NMU, is it
> reasonable to move to dh compat 12 from some other build system if I
> believe doing so will make it easier for me to fix the bug and verify
> the fix?

I see various scenarios:

 - if a package is generally actively maintained, except the maintainer
   is currently unresponsive for some reason and there is a RC bug to
   fix, I could understand frowning upon a build system conversion in an
   NMU.

 - if a package has bugs that can all be fixed trivially with a build
   system conversion, I would see no reason not to do such a conversion,
   even in an NMU.

 - a change of build system in a complex package would be more
   controversial than in a trivial package.

 - if a package has had an inactive and unresponsive maintainer for a
   long time, it would indeed be a case for salvaging.

   I could however imagine someone having enough energy to dust off old
   packages in the archive, while not having enough energy to pick up
   maintenance of lots of old dusty packages.

   I would consider the idea of salvaging+orphaning, like following the
   salvaging procedure but setting the maintainer to qa instead.

 - I'd say that orphaned packages can be uncontroversially be converted
   to dh.


With a consensus of having dh as the default build system, and the
understanding that some packages have good reasons not to use dh, I'd
like a way to tell when a package is not using dh for a reason, from
when a package is not using dh because the maintainer has not gotten
around to it yet.

I'd propose to recommend dh as the default build system, and document in
README.source if there are reasons to use something else.

At that point, one could look at README.source to see if changing build
system would be an possible strategy for fixing bugs in an NMU.


Enrico

-- 
GPG key: 4096R/634F4BD1E7AD5568 2009-05-08 Enrico Zini 


signature.asc
Description: PGP signature

Re: QA expectations (Was: Do we want to Require or Recommend DH)

[Simon McVittie]

On Tue, 14 May 2019 at 18:19:47 +0200, Johannes Schauer wrote:
> So maybe instead of creating unstable-proposed, stuff should move from
> buildd-unstable to unstable only after it successfully passed all kinds of
> automatable QA tests?

Prior art: Ubuntu already does this, gating the transition with a version
of Debian's testing migration scripts that has been configured for a 0 day
delay for all urgencies.

The down side of this is that families of packages occasionally get stuck
in their incoming-equivalent because of versioned dependencies, and a
transition is needed to get them into their testing-equivalent. Perhaps
some Ubuntu developers could comment on the extent to which this is a
practical problem?

> It could also have other nice properties that currently only testing has,
> like no Multi-Arch:same version skews because stuff could only move to 
> unstable
> after it has been built on all arches.

Ubuntu is more able to do this than Debian, because Ubuntu's slowest,
least reliable and least-well-supported architectures are faster, more
reliable and better-supported than Debian's. I'm not sure that we really
want to be waiting for important fixes (especially in large packages
like compilers, web browsers and the kernel) to build successfully on
mips(el), or requiring that their build-time tests have few enough race
conditions to be successful even on slower architectures, before they
can reach the part of the archive that developers use in practice?

smcv

Re: Do we want to Require or Recommend DH

[Simon McVittie]

On Mon, 13 May 2019 at 17:58:47 +0200, Thomas Goirand wrote:
> Now, I have another example, which is quite the opposite one of what you
> gave as example:
> 
> https://salsa.debian.org/openstack-team/debian/openstack-debian-images/blob/debian/stein/debian/rules
> 
> Why would one want to switch that one to something else? The package,
> basically, consists of a shell script and a man page only.

I would personally *especially* use dh for packages this simple.

It uses dh_testroot, so it probably can't have Rules-Requires-Root: no,
and needs to be built as (fake)root indefinitely - even though a package
this simple can almost certainly be built correctly without fakeroot.

At some point in the past someone, probably you, has had to think about:

- which dh_foo helpers need to be included in the list and which can be
  excluded
- which order they go in

which has probably been done correctly, but maybe not - I can't tell. The
fact that they might have been done incorrectly isn't so bad: if they're
wrong it's just a bug, and we can fix bugs. The fact that it would take
thought to work out whether they're correct is more problematic.

At some point in the past someone, probably you, has had to make
this package Policy-compliant when the -arch and -indep targets became
mandatory in order to make Build-Depends-Indep practically useful (perhaps
this package is new enough that you did that as part of its initial
packaging, but either way, someone had to think about it). Adding the
-arch and -indep targets went as slowly as it did because many packages
needed (trivial) per-package changes to enact it.

Part of the value of a dh rules file is that (as Ian Jackson said
elsewhere in this thread) the boilerplate part is trivial, and each
non-boilerplate part (override target) indicates something that is unusual
about this package. A minimal dh rules file immediately tells a reader
"this is a completely ordinary build process" and that's a valuable
thing to convey to a reader.

smcv

Re: NMUs: Do we want to Require or Recommend DH

[Jonas Smedegaard]

Quoting Scott Kitterman (2019-05-15 04:47:48)
> 
> 
> On May 15, 2019 1:13:52 AM UTC, Paul Wise  wrote:
> >On Wed, May 15, 2019 at 2:31 AM Sam Hartman wrote:
> >
> >> How do we feel about people making build system conversions when 
> >> those conversion make it easier to fix some other bug that they are 
> >> fixing as part of an NMU?
> >
> >If the maintainer is MIA enough to not express an opinion when asked 
> >if adding a dh conversion to the NMU is fine, probably the package 
> >should be orphaned/salvaged instead of NMUed, which would bring the 
> >dh conversion into scope.
> 
> I'd think the timeline for that would be longer than the week or two 
> it takes to do an NMU.

Yes the timeline of an NMU being short is the very issue as I see it: 
Switching build system may be easy to do but less easy to maintain for 
the maintainer - regardless of popularity in general.

No, major package refactoring like change of build system is 
unacceptable in NMUs, because it strongly affects long-term maintenance.

Real underlying question seems instead to be this:

Do we tolerate maintainers using a "wrong" packaging style - i.e. an 
unpopular style fewer find easy to do NMUs for?

Yes, let's recommend what is popular.  But not this.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: NMUs: Do we want to Require or Recommend DH

[Scott Kitterman]

On May 15, 2019 1:13:52 AM UTC, Paul Wise  wrote:
>On Wed, May 15, 2019 at 2:31 AM Sam Hartman wrote:
>
>> How do we feel about people making build system conversions when
>those
>> conversion make it easier to fix some other bug that they are fixing
>as
>> part of an NMU?
>
>If the maintainer is MIA enough to not express an opinion when asked
>if adding a dh conversion to the NMU is fine, probably the package
>should be orphaned/salvaged instead of NMUed, which would bring the dh
>conversion into scope.

I'd think the timeline for that would be longer than the week or two it takes 
to do an NMU.

Scott K

Re: NMUs: Do we want to Require or Recommend DH

[Paul Wise]

On Wed, May 15, 2019 at 2:31 AM Sam Hartman wrote:

> How do we feel about people making build system conversions when those
> conversion make it easier to fix some other bug that they are fixing as
> part of an NMU?

If the maintainer is MIA enough to not express an opinion when asked
if adding a dh conversion to the NMU is fine, probably the package
should be orphaned/salvaged instead of NMUed, which would bring the dh
conversion into scope.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: Do we want to Require or Recommend DH

[Moritz Mühlenhoff]

Simon McVittie  schrieb:
> Packages using dh also make it a lot more straightforward to do
> archive-wide changes - similar to the benefit of using debhelper, but
> for changes that affect the "shape" of the build system rather than the
> details of individual steps. As a concrete example,

Or e.g. the changes which were necessary to inject the hardening
build flags; I submitted a few hundred to the BTS when we got started
with it, which was very simple for anything using dh (where is was
enabled by default when using compat level 9) and needed more effort
for anything based on traditional debhelper and or even not using
debhelper at all.

Cheers,
Moritz

Re: Do we want to Require or Recommend DH

[Bernd Zeimetz]

On 5/13/19 11:31 PM, Thomas Goirand wrote:
>> - it's also simpler to understand.
> 
> There, I don't agree. To fully understand how the dh sequencer works,
> one must first understand the 6 mandatory debian/rules targets, and how
> they are called.

You have to understand that in any case. Doesn't matter if its dh, cdbs
or old dh_* stuff.

> dh will hide everything. It isn't simpler, it *LOOKS*
> simpler, but it's a way more complex.

No idea where you think that dh hides something - make and dh both print
what they are doing.



-- 
 Bernd ZeimetzDebian GNU/Linux Developer
 http://bzed.dehttp://www.debian.org
 GPG Fingerprint: ECA1 E3F2 8E11 2432 D485  DD95 EB36 171A 6FF9 435F

Re: NMUs: Do we want to Require or Recommend DH

[Adrian Bunk]

On Tue, May 14, 2019 at 02:30:52PM -0400, Sam Hartman wrote:
>...
> How do we feel about people making build system conversions when those
> conversion make it easier to fix some other bug that they are fixing as
> part of an NMU?

What happens if the maintainer dislikes the change?

The maintainer clearly has the right to revert such an NMU,
and the opinion about the NMU expressed in the changelog or
elsewhere might not be friendly.

There might be cases where it is appropriate, but I wouldn't issue
a general recommendation to do build system conversions in NMUs.

> That is, imagine that a package is mishandling the combination of
> systemd units and an init script.  As someone preparing an NMU, is it
> reasonable to move to dh compat 12 from some other build system if I
> believe doing so will make it easier for me to fix the bug and verify 
> the fix?

Doing a build system conversion properly requires first understanding 
what the old build system did, and later verifying that the changes 
didn't break anything.

It might bring long-term benefits, but if anyone claims that for an NMU 
this is easier than a minimal fix I strongly suspect it is only easy due 
to lack of diligence.

> --Sam

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: QA expectations (Was: Do we want to Require or Recommend DH)

[Adrian Bunk]

On Tue, May 14, 2019 at 05:19:23PM +0200, Helmut Grohne wrote:
> Let me briefly hijack the discussion for a side note. ;)
> 
> On Tue, May 14, 2019 at 11:30:11AM +0200, Andreas Tille wrote:
> > NMUers should do debdiff - no matter what change was done.  And yes, it
> > happened also to me in the past once or twice that I uploaded an empty
> > package or package missing some files.  I do not remember whether it was
> > connected to a change to dh or not ... but mistakes happen.  The fact
> > that we might end up with broken NMUs is IMHO orthogonal to any cdbs to
> > dh change.
> 
> After a failure, we tend to bash uploaders:
>  * Why didn't they look at the debdiff?
>  * Why didn't they run piuparts?
>  * Why didn't they run lintian?

Not running lintian before uploading to unstable shouldn't happen.

>  * Why didn't they run autopkgtest?
>  * Why didn't they do an arch-only/indep-only build?
>  * Why didn't they test their package?
>  * ...
> 
> The things you have to remember before doing an upload are insane.
>...

"Check that your changes are working" is pretty basic and sane.

If the only change was fixing a missing dependency, check that the built 
binary package actually has the correct dependency added.

If you rewrote the whole build system, check that the built packages
still have the same contents and still work.

If you are trying to fix a build failure on an architecture,
make a testbuild on a porterbox instead of 10 uploads to unstable.

>...
> People shouldn't have to remember all the QA. QA should just work and 
> QA should tell people about the (unusual) failures.
>...

QA is good for finding some kinds of common problems.

But it is not a replacement for people being careful when making changes.

> Helmut

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: NMUs: Do we want to Require or Recommend DH

[Andreas Tille]

Hi Sam,

On Tue, May 14, 2019 at 02:30:52PM -0400, Sam Hartman wrote:
> So, I think there is an emerging consensus against the idea of people
> NMUing a package simply to convert it to dh.
> 
> First, I'd like to explicitly call for any last comments from people who would
> like to see us permit NMUs simply to move packages toward dh.

I admit despite I'm in big favour of having the majority of packages
converted to dh I would not feel my time well spent to browse the
archive for packages that might be "smelling" like candidates.

> Are there
> any cases in which such an NMU should be permitted?

If a package has a RC bug or some important bug that annoys me for a
certain reason and fixing it would be easier by just doing a dh
conversion is a pretty good candidate for me.  Or if I need to touch
such a package and the conversion is obviously very simple I would like
to do so.  I will do so in case I'm a member of the team that maintains
the package without question - otherwise I'd give the maintainer a
warning and ask for permission (but will usually write something like
"If I do not hear from you in X days I assume you agree with this.")
 
> Finally, I'd like to focus discussion on an area where emerging
> consensus is much less clear.
> 
> How do we feel about people making build system conversions when those
> conversion make it easier to fix some other bug that they are fixing as
> part of an NMU?

That's one of the cases I mentioned above.

> That is, imagine that a package is mishandling the combination of
> systemd units and an init script.  As someone preparing an NMU, is it
> reasonable to move to dh compat 12 from some other build system if I
> believe doing so will make it easier for me to fix the bug and verify
> the fix?

Good example for a valid dh conversion.

Kind regards

  Andreas.

-- 
http://fam-tille.de

Re: Do we want to Require or Recommend DH

[gregor herrmann]

On Tue, 14 May 2019 11:11:46 +0300, Adrian Bunk wrote:

> On Mon, May 13, 2019 at 11:08:21PM +0200, gregor herrmann wrote:
> > On Mon, 13 May 2019 22:22:32 +0300, Adrian Bunk wrote:
> > > In my experience, keeping existing packages at exotic build systems or 
> > > ancient dh compat levels causes fewer problems than people trying to 
> > > change that just for the sake of it.
> > In my experience ancient debian/rules runes are also a cause for
> > repeated RC bugs and the need for NMUs.
> > Real life example:
> > https://tracker.debian.org/media/packages/libm/libmp3-info-perl/changelog-1.24-1.2
> How well are you testing such conversions?
> Based on work I've seen from you I'd guess your NMU would be better than 
> average. Unfortunately this is not generally true.

I hope I test them well enough :)
(And thanks for the kind feedback.)
 
> There is no perfect solution here and I also get your point,
> what should be taken into consideration is that there is a
> tradeoff between the benefits and the regressions of breaking
> changes like dh compat bumps or even conversions to dh.

Agreed; additional changes are additional chances for mistakes.

Still, I wanted to make the points that
- further adoption of dh(1) would make my life easier by creating
  fewer bugs of certain categories, and
- the possibility to switch a package to dh(1) (in cases where I know
  what that means, as in the example above with a typical perl
  module; not in complex cases like Marco's examples, and not just for
  the sake of it) would make bug fixing in NMUs easier for me and
  even prevent future bugs.
 

Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at -- Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
   `-   NP: Rigmor Gustafsson: The Moon Is A Harsh Mistress


signature.asc
Description: Digital Signature

Re: NMUs: Do we want to Require or Recommend DH

[Sam Hartman]

I think there's a fairly clear consensus emerging that it's worth having
things to check when making a build system conversion.  Looking at
debdiff, ditherscope and reproducibility of the build all appear to be
important things to consider in such a case.

So, I think there is an emerging consensus against the idea of people
NMUing a package simply to convert it to dh.

First, I'd like to explicitly call for any last comments from people who would
like to see us permit NMUs simply to move packages toward dh.  Are there
any cases in which such an NMU should be permitted?

Finally, I'd like to focus discussion on an area where emerging
consensus is much less clear.

How do we feel about people making build system conversions when those
conversion make it easier to fix some other bug that they are fixing as
part of an NMU?
That is, imagine that a package is mishandling the combination of
systemd units and an init script.  As someone preparing an NMU, is it
reasonable to move to dh compat 12 from some other build system if I
believe doing so will make it easier for me to fix the bug and verify
the fix?

--Sam

Re: QA expectations (Was: Do we want to Require or Recommend DH)

[Johannes Schauer]

Quoting Holger Levsen (2019-05-14 17:38:15)
> > Now one can turn this argument upside down. One can say: unstable is the QA
> > area. Britney prevents testing migration on autopkgtest/piuparts/ missing
> > binaries. In that case, we should simply stop filing such things in the BTS
> > and stop doing manual QA on unstable. It should be ok to break unstable.
> > But this is not going to work with transitions. Thus I still think we're
> > doing it wrong and unstable isn't the place to do the QA we expect from
> > everyone.
> 
> have uploads go to unstable-proposed and then, after basic automatic QA
> checks, go to unstable? (and then testing as usual today...)

Doesn't a repository where all binary packages go before they are pushed into
unstable already exist?

deb http://incoming.debian.org/debian-buildd/ buildd-unstable main

So maybe instead of creating unstable-proposed, stuff should move from
buildd-unstable to unstable only after it successfully passed all kinds of
automatable QA tests?

Such an intermediate repository (be it unstable-proposed or buildd-unstable)
could highly improve the quality of unstable and make sure that whatever lands
in unstable will only have those bugs that are usually discovered by humans
only. It could also have other nice properties that currently only testing has,
like no Multi-Arch:same version skews because stuff could only move to unstable
after it has been built on all arches.

Thanks!

cheers, josch


signature.asc
Description: signature

Re: QA expectations (Was: Do we want to Require or Recommend DH)

[Holger Levsen]

On Tue, May 14, 2019 at 05:19:23PM +0200, Helmut Grohne wrote:
> The things you have to remember before doing an upload are insane.
> Having humans remember all this crap is not a reasonable expectation. I
> think our upload process is a bit like classical debhelper: You remember
> to do all the things. We've seen the argument that the dh sequencer
> sheds light on the unusual aspects of a package. I argue that this
> should apply to QA as well. People shouldn't have to remember all the
> QA. QA should just work and QA should tell people about the (unusual)
> failures.

agreed.

> Now one can turn this argument upside down. One can say: unstable is the
> QA area. Britney prevents testing migration on autopkgtest/piuparts/
> missing binaries. In that case, we should simply stop filing such things
> in the BTS and stop doing manual QA on unstable. It should be ok to
> break unstable. But this is not going to work with transitions. Thus I
> still think we're doing it wrong and unstable isn't the place to do the
> QA we expect from everyone.

have uploads go to unstable-proposed and then, after basic automatic QA
checks, go to unstable? (and then testing as usual today...)


-- 
tschau,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

QA expectations (Was: Do we want to Require or Recommend DH)

[Helmut Grohne]

Let me briefly hijack the discussion for a side note. ;)

On Tue, May 14, 2019 at 11:30:11AM +0200, Andreas Tille wrote:
> NMUers should do debdiff - no matter what change was done.  And yes, it
> happened also to me in the past once or twice that I uploaded an empty
> package or package missing some files.  I do not remember whether it was
> connected to a change to dh or not ... but mistakes happen.  The fact
> that we might end up with broken NMUs is IMHO orthogonal to any cdbs to
> dh change.

After a failure, we tend to bash uploaders:
 * Why didn't they look at the debdiff?
 * Why didn't they run piuparts?
 * Why didn't they run lintian?
 * Why didn't they run autopkgtest?
 * Why didn't they do an arch-only/indep-only build?
 * Why didn't they test their package?
 * ...

The things you have to remember before doing an upload are insane.
Having humans remember all this crap is not a reasonable expectation. I
think our upload process is a bit like classical debhelper: You remember
to do all the things. We've seen the argument that the dh sequencer
sheds light on the unusual aspects of a package. I argue that this
should apply to QA as well. People shouldn't have to remember all the
QA. QA should just work and QA should tell people about the (unusual)
failures.

Now one can turn this argument upside down. One can say: unstable is the
QA area. Britney prevents testing migration on autopkgtest/piuparts/
missing binaries. In that case, we should simply stop filing such things
in the BTS and stop doing manual QA on unstable. It should be ok to
break unstable. But this is not going to work with transitions. Thus I
still think we're doing it wrong and unstable isn't the place to do the
QA we expect from everyone.

Now I wonder how to move forward with this (after the dh discussion).

Helmut

Re: Do we want to Require or Recommend DH

[Andreas Tille]

On Tue, May 14, 2019 at 03:11:23PM +0100, Ian Jackson wrote:
> 
> But my point is if you have a handwritten rules file it ends up so
> full of "obvious" boilerplate that it is difficult to see the trees
> for the wood, and there isn't anywhere obvious to put this kind of
> commentary.  I think both dh and cdbs make this easier.

That's another nice summary (and admittedly also true for cdbs).

Kind regards

 Andreas. 

-- 
http://fam-tille.de

Re: Do we want to Require or Recommend DH

[Ian Jackson]

Holger Levsen writes ("Re: Do we want to Require or Recommend DH"):
> On Tue, May 14, 2019 at 12:30:02PM +0100, Ian Jackson wrote:
> > This provides an excellent
> > opportunity to leave a comment next to each weird thing explaining why
> > it's there.
> >   https://browse.dgit.debian.org/xen.git/tree/debian/rules#n111
> 
> wow, that's a *really* nicely documented rules file, kudos!

Thanks for the compliment.  This is there for my benefit as much as
anyone else's.  Having done all the digging to figure all these things
out, I definitely want so save my future self from doing it all again!

Also documenting things means both that everyone else is much less
likely to delete it or break it, and that if it goes wrong or becomes
obsolete, we will be able to fix it or drop it as applicable.

But my point is if you have a handwritten rules file it ends up so
full of "obvious" boilerplate that it is difficult to see the trees
for the wood, and there isn't anywhere obvious to put this kind of
commentary.  I think both dh and cdbs make this easier.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: Do we want to Require or Recommend DH

[Ben Hutchings]

On Tue, 2019-05-14 at 12:54 +0200, Andreas Tille wrote:
> On Tue, May 14, 2019 at 10:38:06AM +0100, Ben Hutchings wrote:
> > On Tue, 2019-05-14 at 11:07 +0200, Andreas Tille wrote:
> > > Can you give an example for a package that has a non-dh rules file
> > > "working for years" that gives as a result a package with no lintian
> > > warnings without changing this d/rules file?
> > 
> > linux is one.
> > 
> > I did a lot of work to address lintian warnings last year, and most of
> > that did not involve debian/rules*.
>  
> Please call me over-picky but without checking the history is your
> "most" not a sign that there is at least one change to d/rules which was
> needed to fix some lintian issue.

I can only see one change that dh would have handled for us.

> I do not want you to change linux
> d/rules file and I think it is pretty safe from beeing NMUed just to
> switch to dh.

Indeed.  There is definitely scope for simplification, but it takes a
lot of knowledge and time to do that.

Given what others have said, I doubt that dh in its current state could
handle the multiple configurations, which is a shame.

> My point was that I doubt that any "working for years"
> d/rules has no lintian issues.

If "working for years" also implies unchanged for years, yes I agree.

Ben.

-- 
Ben Hutchings
I haven't lost my mind; it's backed up on tape somewhere.




signature.asc
Description: This is a digitally signed message part

Re: Do we want to Require or Recommend DH

[Thibaut Paumard]

Le 14/05/2019 à 11:07, Andreas Tille a écrit :
> Can you give an example for a package that has a non-dh rules file
> "working for years" that gives as a result a package with no lintian
> warnings without changing this d/rules file?

Turns out I can't... I was thinking of some packages that I didn't have
to touch for years, but I checked and I actually converted them to dh
that many years ago...

Regards, Thibaut.




signature.asc
Description: OpenPGP digital signature

Re: d-shlibs (Was: Do we want to Require or Recommend DH)

[Jonas Smedegaard]

Quoting Adrian Bunk (2019-05-14 13:47:02)
> On Tue, May 14, 2019 at 12:50:54PM +0200, Andreas Tille wrote:
> > On Tue, May 14, 2019 at 01:12:17PM +0300, Adrian Bunk wrote:
> > > On Tue, May 14, 2019 at 11:30:11AM +0200, Andreas Tille wrote:
> > > > On Mon, May 13, 2019 at 10:22:32PM +0300, Adrian Bunk wrote:
> > > > > > >things simple for team mates.  I consider it a valid 
> > > > > > >request to every single maintainer to respect that other 
> > > > > > >people have good reasons to change her/his package.
> > > > > >...
> > > > > 
> > > > > Based on this rationale, Andreas should stop using d-shlibs.
> > > > > 
> > > > > Weird tools on top of dh are not that different from using a 
> > > > > weird buildsystem when debugging other peoples packages, and 
> > > > > d-shlibs is something I've seen involved in bugs more than 
> > > > > once.
> > > > 
> > > > Its the first time that I hear criticism about d-shlibs usage
> > > 
> > > It is fine in the current "maintainer can do anything" world.
> > 
> > Hmmm, I don't get it: I'm using dh and in addition I'm using a tool 
> > that enforces library packaging policy.
> >...
> 
> A tool nearly noone else uses or knows.
> 
> Spending time on trying to understand what such a tool does or why it 
> is needed in a specific package is not really different from spending 
> time trying to understand how a buildsystem works.
> 
> If it is generally useful it should be done by debhelper 
> automatically, otherwise it should not be used at all if the goal is 
> to make it easier for other people to make changes to your packages.

I like the smell of flexibility - e.g. debhelper offering a baseline on 
top of which can be sprinkled more exotic tweaks as needed.

I dislike the smell of monoculture - e.g. banning or merging into 
debhelper itself debhelper addon packages.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Re: Do we want to Require or Recommend DH

[Holger Levsen]

On Tue, May 14, 2019 at 12:30:02PM +0100, Ian Jackson wrote:
> One thing that I found really good about dh is that you only have to
> write code about things that are unusual.  

indeed.

> This provides an excellent
> opportunity to leave a comment next to each weird thing explaining why
> it's there.
>   https://browse.dgit.debian.org/xen.git/tree/debian/rules#n111

wow, that's a *really* nicely documented rules file, kudos!


-- 
tschau,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: d-shlibs (Was: Do we want to Require or Recommend DH)

[Adrian Bunk]

On Tue, May 14, 2019 at 12:50:54PM +0200, Andreas Tille wrote:
> On Tue, May 14, 2019 at 01:12:17PM +0300, Adrian Bunk wrote:
> > On Tue, May 14, 2019 at 11:30:11AM +0200, Andreas Tille wrote:
> > > On Mon, May 13, 2019 at 10:22:32PM +0300, Adrian Bunk wrote:
> > > > > >things simple for team mates.  I consider it a valid request to every
> > > > > >single maintainer to respect that other people have good reasons to
> > > > > >change her/his package.
> > > > >...
> > > > 
> > > > Based on this rationale, Andreas should stop using d-shlibs.
> > > > 
> > > > Weird tools on top of dh are not that different from using a weird 
> > > > buildsystem when debugging other peoples packages, and d-shlibs is 
> > > > something I've seen involved in bugs more than once.
> > > 
> > > Its the first time that I hear criticism about d-shlibs usage
> > 
> > It is fine in the current "maintainer can do anything" world.
> 
> Hmmm, I don't get it:  I'm using dh and in addition I'm using a tool
> that enforces library packaging policy.
>...

A tool nearly noone else uses or knows.

Spending time on trying to understand what such a tool does or why it is 
needed in a specific package is not really different from spending time 
trying to understand how a buildsystem works.

If it is generally useful it should be done by debhelper automatically,
otherwise it should not be used at all if the goal is to make it easier
for other people to make changes to your packages.

> Kind regards
> 
>   Andreas. 

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Do we want to Require or Recommend DH

[Ian Jackson]

Sean Whitton writes ("Re: Do we want to Require or Recommend DH"):
> I agree with Scott's emphasis on the distinction between new and
> existing packages.  Perhaps application of the distinction could be
> extended: perhaps there are other things that we could require of new
> packages, while creating no expectation that these requirements be met
> of older packages.
> 
> In general, if a policy requirement or convention should apply to new
> packages, then it should apply to existing packages, too.  But
> specifically where applying the requirement to an existing package is
> hugely more work than applying it to a new package, perhaps the
> requirement ought to be limited to new packages alone.

There is a big distinction between recommendations which directly
affect the functioning of the binary packages, and recommendations
which make future development easier.

For the latter - and use of dh is an example - it makes a lot of sense
to make the recomendations stronger for new packages.

I also think it would be very valuable for policy to recommend things
like this as the usual case, without mandating them.  If for any
reason the maintainer doesn't want to use dh, then they can leave a
wontfix bug open (or something) to document the reasons.

My own anecodote:

Last year I converted src:xen to dh from the previous baroque system
(which I think was based on a clone-and-hack of the machinery in
src:linux.)  The result is massively better.

But it was serious work to repeatedly debdiff the results, review each
individual weird thing and decide whether to keep it, try to reproduce
its effects with a dh override or whatever, etc.  This needed a high
level of familiarity with both the underlying software and Debian's
packaging practices.  Even so I caused a handful of regressions.

One thing that I found really good about dh is that you only have to
write code about things that are unusual.  This provides an excellent
opportunity to leave a comment next to each weird thing explaining why
it's there.
  https://browse.dgit.debian.org/xen.git/tree/debian/rules#n111

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: Do we want to Require or Recommend DH

[Andreas Tille]

On Tue, May 14, 2019 at 10:38:06AM +0100, Ben Hutchings wrote:
> On Tue, 2019-05-14 at 11:07 +0200, Andreas Tille wrote:
> > 
> > Can you give an example for a package that has a non-dh rules file
> > "working for years" that gives as a result a package with no lintian
> > warnings without changing this d/rules file?
> 
> linux is one.
> 
> I did a lot of work to address lintian warnings last year, and most of
> that did not involve debian/rules*.
 
Please call me over-picky but without checking the history is your
"most" not a sign that there is at least one change to d/rules which was
needed to fix some lintian issue.  I do not want you to change linux
d/rules file and I think it is pretty safe from beeing NMUed just to
switch to dh.  My point was that I doubt that any "working for years"
d/rules has no lintian issues.

Kind regards

 Andreas.

-- 
http://fam-tille.de

d-shlibs (Was: Do we want to Require or Recommend DH)

[Andreas Tille]

On Tue, May 14, 2019 at 01:12:17PM +0300, Adrian Bunk wrote:
> On Tue, May 14, 2019 at 11:30:11AM +0200, Andreas Tille wrote:
> > On Mon, May 13, 2019 at 10:22:32PM +0300, Adrian Bunk wrote:
> > > > >things simple for team mates.  I consider it a valid request to every
> > > > >single maintainer to respect that other people have good reasons to
> > > > >change her/his package.
> > > >...
> > > 
> > > Based on this rationale, Andreas should stop using d-shlibs.
> > > 
> > > Weird tools on top of dh are not that different from using a weird 
> > > buildsystem when debugging other peoples packages, and d-shlibs is 
> > > something I've seen involved in bugs more than once.
> > 
> > Its the first time that I hear criticism about d-shlibs usage
> 
> It is fine in the current "maintainer can do anything" world.

Hmmm, I don't get it:  I'm using dh and in addition I'm using a tool
that enforces library packaging policy.
 
> > and I'm
> > fine with discussing this but I'd prefer not to spoil the current
> > thread.
> >...
> 
> It is actually part of it, due to:
> 
> >...
> > As far as I understood the point of the discussion is that we want to
> > get the whole archive more uniform to reduce the potential causes for
> > bugs *in* *the* *future*.
> >...
> 
> If this is the point, then weird tools on top of dh are part of the 
> problem just as weird buildsystems are.

d-shlibs is not really on top of dh.  Its invoked with override_*
and thus clearly separate from dh.
 
[Haskell-example snipped since I think this was discussed somewhere
 else.]

Kind regards

  Andreas. 

-- 
http://fam-tille.de

Re: Do we want to Require or Recommend DH

[Adrian Bunk]

On Tue, May 14, 2019 at 10:27:45AM +0200, Johannes Schauer wrote:
> Quoting Adrian Bunk (2019-05-14 10:11:46)
> > 
> > How well are you testing such conversions?
> > Based on work I've seen from you I'd guess your NMU would be better than 
> > average. Unfortunately this is not generally true.
> > 
> > Based on what enters the archive, "debdiff between old and new package" 
> > already seems to be something that many people are not doing - then
> > cleaning up after mass-NMUs would be much work.
> > 
> > I have even seen maintainers blindly replacing a complex old
> > debian/rules with the dh 3-liner, and all the bugfixes and
> > workarounds in the old one were bugs again.
> > 
> > To show the quantitative side of my argument:
> > 
> > The default change to parallel building in dh compat 10 alone has caused 
> > a three digit number of RC bugs, popping up at a pace of 1-2 RC bugs
> > per week for several years.
> > 
> > The problem is that anything that works for only 99% of all packages
> > results in such a high number of new bugs.
> > 
> > Parallel build bugs slipping through an upload can happen,
> > but maintainers not going through the upgrading checklist
> > and running debdiff between old and new packages as well
> > as testing them when doing dh compat bumps is harder to
> > excuse - and in practice this does happen.
> > 
> > There is no perfect solution here
> 
> What makes reproducible-builds not the perfect fit for this?
> 
> Whenever I converted a package to dh or bumped debhelper compat level, I 
> always
> checked whether the produced binaries were bit-by-bit identical to the ones
> before.

Don't assume everyone follows the same high standards as you do.

> Are there many errors that I would be missing by relying on reproducibility
> results?

The parallel build issues I mentioned might be missed,
but this is more exceptional.

dh compat 12 defaulting to dh_dwz might make the -dbgsym packages 
different, and other intentional differences might exist.

> Thanks!
> 
> cheers, josch

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Do we want to Require or Recommend DH

[Adrian Bunk]

On Tue, May 14, 2019 at 11:30:11AM +0200, Andreas Tille wrote:
> On Mon, May 13, 2019 at 10:22:32PM +0300, Adrian Bunk wrote:
> > On Mon, May 13, 2019 at 08:33:44AM -0400, Sam Hartman wrote:
> > >...
> > > Andreas Tille's explanation (quoted below) is typical of what I've heard
> > > in this area.
> > >
> > > >To come back
> > > >to the question:  I'm positively convinced that we should strive to
> > > >unify our packaging as much as possible and in terms of d/rules file dh
> > > >is obviously the default we should pick.  I'd go that far that lintian
> > > >should issue some warning at "pedantic" level if there is no comment:
> > > >"I'm not using dh because ... ".  In other words:  Use dh in debian/rules
> > > >should make it into policy.
> > >
> > > >The rationale is that dh makes it extremely easy to understand other
> > > >d/rules files.  Specifically in freeze like now it is easy to touch
> > > >other peoples packages (just done this several times in the last weeks
> > > >and luckily close to all used dh).  Its the point of teamwork (and I
> > > >consider all people touching official Debian packages as a team) to make
> > > >things simple for team mates.  I consider it a valid request to every
> > > >single maintainer to respect that other people have good reasons to
> > > >change her/his package.
> > >...
> > 
> > Based on this rationale, Andreas should stop using d-shlibs.
> > 
> > Weird tools on top of dh are not that different from using a weird 
> > buildsystem when debugging other peoples packages, and d-shlibs is 
> > something I've seen involved in bugs more than once.
> 
> Its the first time that I hear criticism about d-shlibs usage

It is fine in the current "maintainer can do anything" world.

> and I'm
> fine with discussing this but I'd prefer not to spoil the current
> thread.
>...

It is actually part of it, due to:

>...
> As far as I understood the point of the discussion is that we want to
> get the whole archive more uniform to reduce the potential causes for
> bugs *in* *the* *future*.
>...

If this is the point, then weird tools on top of dh are part of the 
problem just as weird buildsystems are.

>...
> > When you debug for the first time a non-trivial problem in a complex
> > package like src:gcc-8 or in a package in an ecosystem like Haskell you
> > can easily spend hours just for figuring out what is actually going on
> > or how to pass additional flags. Whether or not the build machinery is
> > using dh is not a big difference when you have to understand what is
> > actually going on.
>
> I for myself prefer to debug code based on the same tool set.
>...

This is pretty much impossible.

dh is nice when it supports whatever you want to do,
but for everything else it is just another layer you
have to understand when debugging and fixing problems.

Imagine you want to make ghc pass an additional flag to gcc.

The layering is as follows:
Debian haskell scripts
upstream build system
ghc
gcc

You end up at questions:
How can I tell ghc to pass a parameter to gcc?
How can I tell the upstream build system to pass a parameter to ghc?
How can I tell the Debian haskell scripts to pass a parameter to the 
upstream build system?


debian/rules is already a 3-line
#!/usr/bin/make -f

include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/class/hlibrary.mk


It wouldn't help you if that would be changed to a 3-line
#!/usr/bin/make -f

%:
dh $@ --with haskell


In either case there is a huge Haskell-specific build machinery you 
have to understand for making some kinds of debugging or changes.


> Kind regards
> 
>   Andreas.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Do we want to Require or Recommend DH

[Ben Hutchings]

On Tue, 2019-05-14 at 11:07 +0200, Andreas Tille wrote:
> On Mon, May 13, 2019 at 04:22:49PM +0200, Thibaut Paumard wrote:
> > > Why Not Make this Change
> > > 
> > 
> > I would use dh for any new package and converting trivial packages is...
> > trivial. However converting a package with a more convoluted rules files
> > will take humanpower. While it may be justified to convert a mildly
> > complex rules file on a package that has some activity, I don't think I
> > would invest those resources to convert a package that's been working
> > for years without anyone touching it's rules files.
> 
> Can you give an example for a package that has a non-dh rules file
> "working for years" that gives as a result a package with no lintian
> warnings without changing this d/rules file?

linux is one.

I did a lot of work to address lintian warnings last year, and most of
that did not involve debian/rules*.

Ben.

-- 
Ben Hutchings
I haven't lost my mind; it's backed up on tape somewhere.




signature.asc
Description: This is a digitally signed message part

Re: Do we want to Require or Recommend DH

[Andreas Tille]

On Mon, May 13, 2019 at 10:22:32PM +0300, Adrian Bunk wrote:
> On Mon, May 13, 2019 at 08:33:44AM -0400, Sam Hartman wrote:
> >...
> > Andreas Tille's explanation (quoted below) is typical of what I've heard
> > in this area.
> >
> > >To come back
> > >to the question:  I'm positively convinced that we should strive to
> > >unify our packaging as much as possible and in terms of d/rules file dh
> > >is obviously the default we should pick.  I'd go that far that lintian
> > >should issue some warning at "pedantic" level if there is no comment:
> > >"I'm not using dh because ... ".  In other words:  Use dh in debian/rules
> > >should make it into policy.
> >
> > >The rationale is that dh makes it extremely easy to understand other
> > >d/rules files.  Specifically in freeze like now it is easy to touch
> > >other peoples packages (just done this several times in the last weeks
> > >and luckily close to all used dh).  Its the point of teamwork (and I
> > >consider all people touching official Debian packages as a team) to make
> > >things simple for team mates.  I consider it a valid request to every
> > >single maintainer to respect that other people have good reasons to
> > >change her/his package.
> >...
> 
> Based on this rationale, Andreas should stop using d-shlibs.
> 
> Weird tools on top of dh are not that different from using a weird 
> buildsystem when debugging other peoples packages, and d-shlibs is 
> something I've seen involved in bugs more than once.

Its the first time that I hear criticism about d-shlibs usage and I'm
fine with discussing this but I'd prefer not to spoil the current
thread.

 
> When you debug for the first time a non-trivial problem in a complex 
> package like src:gcc-8 or in a package in an ecosystem like Haskell you 
> can easily spend hours just for figuring out what is actually going on 
> or how to pass additional flags. Whether or not the build machinery is 
> using dh is not a big difference when you have to understand what is 
> actually going on.

I for myself prefer to debug code based on the same tool set.
 
> >...
> > And at some level I think we're asking whether it is appropriate to NMU
> > a package to convert it to dh.
> >...
> 
> Common causes of broken packages in the archive include:
> - dh compat level bumps
> - conversion of debhelper using packages to the short dh form
> - conversion from other build systems to dh

I agree that these are potential sources of bugs.  However, I do not see
much evidence that if an NMUer / team member does any edit on some
d/rules file would be different to the cases above.

> It is already a real problem when maintainers are bumping dh compat 
> levels without checking very thoroughly before uploading that this 
> didn't cause any regression - a dh compat level bump is a breaking 
> change and should not be done without due diligence.

+1
(But I do not see any argument against using dh here)
 
> And we do get broken packages packages uploaded where the NMU
> of a build system change (e.g. cdbs to dh) resulted in an empty
> package - whoever uploaded such a package clearly didn't even
> do basic checks.

NMUers should do debdiff - no matter what change was done.  And yes, it
happened also to me in the past once or twice that I uploaded an empty
package or package missing some files.  I do not remember whether it was
connected to a change to dh or not ... but mistakes happen.  The fact
that we might end up with broken NMUs is IMHO orthogonal to any cdbs to
dh change.

> In my experience, keeping existing packages at exotic build systems or 
> ancient dh compat levels causes fewer problems than people trying to 
> change that just for the sake of it.

As far as I understood the point of the discussion is that we want to
get the whole archive more uniform to reduce the potential causes for
bugs *in* *the* *future*.  This might come at the expense of some bugs
when doing so and IMHO the beginning of a release cycle (= after Buster
release) is a sensible point in time to do so.
 
BTW, Adrian, thanks for all your patience and bug fixing at an extremely
high rate.  It is really welcome and appreciated and I think its a good
chance to mention this explicitly here.
 
Kind regards

  Andreas.

-- 
http://fam-tille.de

Re: Do we want to Require or Recommend DH

[Andrey Rahmatullin]

On Tue, May 14, 2019 at 09:10:04AM +, Holger Levsen wrote:
> On Tue, May 14, 2019 at 02:07:11PM +0500, Andrey Rahmatullin wrote:
> > One can go further and say that people uploading broken packages are the
> > actual problem. After all, we have several classes of bugs caused by
> > people uploading .debs built in a broken env.
> > Not sure if we can fix this and how.
> 
> forbid binary uploads for everyone. allow binary uploads for some people
> for some packages sometimes.
People will find a way. Adrian was talking about uploading broken source
packages, I just provided a different example of the same problem.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Do we want to Require or Recommend DH

[Holger Levsen]

On Tue, May 14, 2019 at 02:07:11PM +0500, Andrey Rahmatullin wrote:
> One can go further and say that people uploading broken packages are the
> actual problem. After all, we have several classes of bugs caused by
> people uploading .debs built in a broken env.
> Not sure if we can fix this and how.

forbid binary uploads for everyone. allow binary uploads for some people
for some packages sometimes.


-- 
tschau,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C

Dance like no one's watching. Encrypt like everyone is.


signature.asc
Description: PGP signature

Re: Do we want to Require or Recommend DH

[Andrey Rahmatullin]

On Tue, May 14, 2019 at 11:11:46AM +0300, Adrian Bunk wrote:
> How well are you testing such conversions?
> Based on work I've seen from you I'd guess your NMU would be better than 
> average. Unfortunately this is not generally true.
> 
> Based on what enters the archive, "debdiff between old and new package" 
> already seems to be something that many people are not doing - then
> cleaning up after mass-NMUs would be much work.
> 
> I have even seen maintainers blindly replacing a complex old
> debian/rules with the dh 3-liner, and all the bugfixes and
> workarounds in the old one were bugs again.
> 
> To show the quantitative side of my argument:
> 
> The default change to parallel building in dh compat 10 alone has caused 
> a three digit number of RC bugs, popping up at a pace of 1-2 RC bugs
> per week for several years.
> 
> The problem is that anything that works for only 99% of all packages
> results in such a high number of new bugs.
> 
> Parallel build bugs slipping through an upload can happen,
> but maintainers not going through the upgrading checklist
> and running debdiff between old and new packages as well
> as testing them when doing dh compat bumps is harder to
> excuse - and in practice this does happen.
One can go further and say that people uploading broken packages are the
actual problem. After all, we have several classes of bugs caused by
people uploading .debs built in a broken env.
Not sure if we can fix this and how.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: Do we want to Require or Recommend DH

[Andreas Tille]

On Mon, May 13, 2019 at 04:22:49PM +0200, Thibaut Paumard wrote:
> > Why Not Make this Change
> > 
> 
> I would use dh for any new package and converting trivial packages is...
> trivial. However converting a package with a more convoluted rules files
> will take humanpower. While it may be justified to convert a mildly
> complex rules file on a package that has some activity, I don't think I
> would invest those resources to convert a package that's been working
> for years without anyone touching it's rules files.

Can you give an example for a package that has a non-dh rules file
"working for years" that gives as a result a package with no lintian
warnings without changing this d/rules file?

Kind regards

   Andreas.

-- 
http://fam-tille.de

Re: Do we want to Require or Recommend DH

[Johannes Schauer]

Quoting Adrian Bunk (2019-05-14 10:11:46)
> On Mon, May 13, 2019 at 11:08:21PM +0200, gregor herrmann wrote:
> > On Mon, 13 May 2019 22:22:32 +0300, Adrian Bunk wrote:
> > 
> > > In my experience, keeping existing packages at exotic build systems or 
> > > ancient dh compat levels causes fewer problems than people trying to 
> > > change that just for the sake of it.
> > 
> > In my experience ancient debian/rules runes are also a cause for
> > repeated RC bugs and the need for NMUs.
> > 
> > Real life example:
> > https://tracker.debian.org/media/packages/libm/libmp3-info-perl/changelog-1.24-1.2
> > 
> > Both RC bugs wouldn't have happened with dh(1); and the second NMU
> > wouldn't have been necessary if I had switched to dh(1) in the first
> > one.
> 
> How well are you testing such conversions?
> Based on work I've seen from you I'd guess your NMU would be better than 
> average. Unfortunately this is not generally true.
> 
> Based on what enters the archive, "debdiff between old and new package" 
> already seems to be something that many people are not doing - then
> cleaning up after mass-NMUs would be much work.
> 
> I have even seen maintainers blindly replacing a complex old
> debian/rules with the dh 3-liner, and all the bugfixes and
> workarounds in the old one were bugs again.
> 
> To show the quantitative side of my argument:
> 
> The default change to parallel building in dh compat 10 alone has caused 
> a three digit number of RC bugs, popping up at a pace of 1-2 RC bugs
> per week for several years.
> 
> The problem is that anything that works for only 99% of all packages
> results in such a high number of new bugs.
> 
> Parallel build bugs slipping through an upload can happen,
> but maintainers not going through the upgrading checklist
> and running debdiff between old and new packages as well
> as testing them when doing dh compat bumps is harder to
> excuse - and in practice this does happen.
> 
> There is no perfect solution here

What makes reproducible-builds not the perfect fit for this?

Whenever I converted a package to dh or bumped debhelper compat level, I always
checked whether the produced binaries were bit-by-bit identical to the ones
before.

Are there many errors that I would be missing by relying on reproducibility
results?

Thanks!

cheers, josch


signature.asc
Description: signature

Re: Do we want to Require or Recommend DH

[Adrian Bunk]

On Mon, May 13, 2019 at 11:08:21PM +0200, gregor herrmann wrote:
> On Mon, 13 May 2019 22:22:32 +0300, Adrian Bunk wrote:
> 
> > In my experience, keeping existing packages at exotic build systems or 
> > ancient dh compat levels causes fewer problems than people trying to 
> > change that just for the sake of it.
> 
> In my experience ancient debian/rules runes are also a cause for
> repeated RC bugs and the need for NMUs.
> 
> Real life example:
> https://tracker.debian.org/media/packages/libm/libmp3-info-perl/changelog-1.24-1.2
> 
> Both RC bugs wouldn't have happened with dh(1); and the second NMU
> wouldn't have been necessary if I had switched to dh(1) in the first
> one.

How well are you testing such conversions?
Based on work I've seen from you I'd guess your NMU would be better than 
average. Unfortunately this is not generally true.

Based on what enters the archive, "debdiff between old and new package" 
already seems to be something that many people are not doing - then
cleaning up after mass-NMUs would be much work.

I have even seen maintainers blindly replacing a complex old
debian/rules with the dh 3-liner, and all the bugfixes and
workarounds in the old one were bugs again.

To show the quantitative side of my argument:

The default change to parallel building in dh compat 10 alone has caused 
a three digit number of RC bugs, popping up at a pace of 1-2 RC bugs
per week for several years.

The problem is that anything that works for only 99% of all packages
results in such a high number of new bugs.

Parallel build bugs slipping through an upload can happen,
but maintainers not going through the upgrading checklist
and running debdiff between old and new packages as well
as testing them when doing dh compat bumps is harder to
excuse - and in practice this does happen.

There is no perfect solution here and I also get your point,
what should be taken into consideration is that there is a
tradeoff between the benefits and the regressions of breaking
changes like dh compat bumps or even conversions to dh.

In my experience people are already pushing too much towards 
"use dh with latest compat" in existing packages, and it would be
better to slow that down rather than accelerate it even further.

> Cheers,
> gregor

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed