Re: Bug#489771: New Build-Options field and build-arch option, please review

[Bill Allombert]

On Thu, Sep 18, 2008 at 05:36:46PM +0200, Raphael Hertzog wrote:
> On Thu, 18 Sep 2008, Bill Allombert wrote:
> > > I have to say i verry rarely do not use debuild. And 99% of the
> > > exceptions are calling debian/rules clean.
> > 
> > Precisely, debuild does not use dpkg-buildpackage, but call debian/rules
> > directly.
> 
> This has been fixed already. It calls dpkg-buildpackage now (except if you use
> its hook features).

So it still call debian/rules directly in some case.

> (And I don't see why one way would be more Debianish than the other)

Neither I do, but then I do not attempt to kill one way in favour of the other.

I would object to a proposal policy making dpkg-buildpackage mandatory
to build packages.

Cheers,
-- 
Bill. <[EMAIL PROTECTED]>

Imagine a large red swirl here. 


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Bug#489771: New Build-Options field and build-arch option, please review

[Russ Allbery]

Goswin von Brederlow <[EMAIL PROTECTED]> writes:

> Plus a note in policy clarifying that debian/rules is only an
> interface for dpkg-buildpackage but not users.

Right.  If you want to make this a rule, then we should discuss it, reach
a consensus, document and publicize the change, and so forth.

Right now, I feel like this change is being made as part of dpkg
development, without general recognition of what's happening and without
the corresponding changes to Policy so that DDs know what to expect.  (I
don't think this is *intentional* on your part, more a case of a set of
decisions that all seemed like a good idea at the time but which
cumulatively have a significant impact.)

> I have to say i verry rarely do not use debuild. And 99% of the
> exceptions are calling debian/rules clean.

The hard part of standards isn't the common case.  Currently, debian/rules
is defined as the package build interface, and while most people don't
normally rely on that, we don't know what might break; one of the points
of a standard is to let people rely on it without having to tell you first
what they're doing.

Personally, I use debian/rules build all the time for testing, and would
never have thought to mention it to anyone.

-- 
Russ Allbery ([EMAIL PROTECTED])   


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Bug#489771: New Build-Options field and build-arch option, please review

[Raphael Hertzog]

On Thu, 18 Sep 2008, Bill Allombert wrote:
> > I have to say i verry rarely do not use debuild. And 99% of the
> > exceptions are calling debian/rules clean.
> 
> Precisely, debuild does not use dpkg-buildpackage, but call debian/rules
> directly.

This has been fixed already. It calls dpkg-buildpackage now (except if you use
its hook features).

(And I don't see why one way would be more Debianish than the other)

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Bug#489771: New Build-Options field and build-arch option, please review

[Manoj Srivastava]

On Thu, Sep 18 2008, Goswin von Brederlow wrote:
>>Russ Allbery <[EMAIL PROTECTED]> writes:
> Setting the environment on a distribution wide level is ugly and
> fragile. Too many users will reset the environment in their .bashrc.
>
> Instead the idea was to have a vendor (set in
> /etc/dpkg/origins/default) that will be exported into DEB_VENDOR if
> unset and also set DEB_BUILD_OPTIONS to the vendor specifics defaults.
>
> The bugreports relevant for this have 2 solutions:
>
> 1) make dpkg-buildpackage use (or tool with equivalent environment
>setting up capabilities) mandatory
>
> 2) have debian/rules call something to set DEB_VENDOR and possibly
>more
>
>E.g. 'include /usr/share/dpkg/Makefile.dpkg'
>or   'DEB_VENDOR?= (shell dpkg-vendor -qDEB_VENDOR)
>  DEB_BUILD_OPTIONS ?= (shell dpkg-vendor -qDEB_BUILD_OPTIONS)
>
> The argument against 2 is that is requires every source to be modified
> if they want to support vendors whereas 1 only needs some small
> modification to dpkg-buildpackage to support calling arbitrary targets
> in debian/rules and a change in policy making its use mandatory.

But you need to modify the rules file anyway to take advantage
 of the DEB_VENDOR  variable, no? Currently, setting it does nothing for
 any package. So if people are changing the rules file, they can also
 add in those two lines.

>> My objection is specifically to having dpkg-buildpackage set a
>> variety of environment variables *by default*, and then telling
>> package maintainers that they should rely on those environment
>> variables being set in the default case.  That breaks the
>> debian/rules interface and requires that all package builds go
>> through dpkg-buildpackage.  Having dpkg-buildpackage set environment
>> variables in the non-default case like Emdebian is not a problem,
>> since for Emdebian builds (for example) Emdebian can decide that
>> using dpkg-buildpackage or setting the environment variables manually
>> is required.  There is no existing precedent, and they can make that
>> rule from scratch.

I tend to agree.

> Then you have one interface for Debian and one interface for every
> other vendor including ubuntu (or option 2 above).


>> My concern is for the default build where there *is* an existing
>> precedent that debian/rules build should work sanely, not for support
>> for special cases like that where the existing debian/rules interface
>> already doesn't do the right thing without additional help.

>> If you are going to go down this path of having dpkg-buildpackage set
>> up an environment that package maintainers should rely on, you or
>> someone else on the dpkg team needs to make a debian-devel-announce
>> post making it clear that debian/rules build is no longer a supported
>> interface for building packages and using dpkg-buildpackage is
>> required for consistent behavior.

I see that as a serious degradation in the quality of the
 distribution. 

> Plus a note in policy clarifying that debian/rules is only an
> interface for dpkg-buildpackage but not users.

No. Debian is a member of the free software community, and being
 able to just do a configure, or a build,  or build a single binary, by
 end users, is a feature that should not be given up easily. And
 certainly not without some significant rationale; degrating features
 for most of our users to cater to other distributions does not actually
 cut it.

>> Right now, I don't think most Debian Developers have any idea what the
>> implications of these changes are.

> I have to say i verry rarely do not use debuild. And 99% of the
> exceptions are calling debian/rules clean.

I have never ever used debuild. So there is another anecdote,
 which may or may not mean anything.

manoj
-- 
"Nothing can stop him.  Not even common sense."  Mark Komarinski
Manoj Srivastava <[EMAIL PROTECTED]>   
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Bug#489771: New Build-Options field and build-arch option, please review

[Bill Allombert]

On Thu, Sep 18, 2008 at 03:03:20PM +0200, Goswin von Brederlow wrote:
> Russ Allbery <[EMAIL PROTECTED]> writes:
> 
> > Raphael Hertzog <[EMAIL PROTECTED]> writes:
> >> On Wed, 10 Sep 2008, Bill Allombert wrote:
> >
> >>> I like to say I concurr with Russ. There are some much difference
> >>> between packages that distributions wide default does not make sense.
> >>> Such change would rather lead me to hardcode values of
> >>> DEBIAN_BUILD_OPTIONS in debian/rules if they are used blidly.
> >>
> >> But more and more people want to be able to change distribution wide
> >> default: Emdebian wants to enable "nodocs" and "nocheck" by default,
> >> other want to be able to enable hardening options by default and I agree
> >> with them that official support for such a facility is desirable.
> >
> > So they should set DEB_BUILD_OPTIONS in the environment.  That's what it's
> > for.  I don't have any objections to that, or even to doing it via
> > dpkg-buildpackage.
> 
> Setting the environment on a distribution wide level is ugly and
> fragile. Too many users will reset the environment in their .bashrc.
> 
> Instead the idea was to have a vendor (set in
> /etc/dpkg/origins/default) that will be exported into DEB_VENDOR if
> unset and also set DEB_BUILD_OPTIONS to the vendor specifics defaults.
> 
> The bugreports relevant for this have 2 solutions:
> 
> 1) make dpkg-buildpackage use (or tool with equivalent environment
>setting up capabilities) mandatory
> 
> 2) have debian/rules call something to set DEB_VENDOR and possibly
>more
> 
>E.g. 'include /usr/share/dpkg/Makefile.dpkg'
>or   'DEB_VENDOR?= (shell dpkg-vendor -qDEB_VENDOR)
>  DEB_BUILD_OPTIONS ?= (shell dpkg-vendor -qDEB_BUILD_OPTIONS)
> 
> The argument against 2 is that is requires every source to be modified
> if they want to support vendors whereas 1 only needs some small
> modification to dpkg-buildpackage to support calling arbitrary targets
> in debian/rules and a change in policy making its use mandatory.

2) is the right way to proceed for _Debian_. People in a hurry can use 1,
but not us. 

2) imply that packages will not have DEB_VENDOR support unless some
check they support it.

> > Right now, I don't think most Debian Developers have any idea what the
> > implications of these changes are.
> 
> I have to say i verry rarely do not use debuild. And 99% of the
> exceptions are calling debian/rules clean.

Precisely, debuild does not use dpkg-buildpackage, but call debian/rules
directly.

Cheers,
Bill.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Bug#489771: New Build-Options field and build-arch option, please review

[Goswin von Brederlow]

Russ Allbery <[EMAIL PROTECTED]> writes:

> Raphael Hertzog <[EMAIL PROTECTED]> writes:
>> On Wed, 10 Sep 2008, Bill Allombert wrote:
>
>>> I like to say I concurr with Russ. There are some much difference
>>> between packages that distributions wide default does not make sense.
>>> Such change would rather lead me to hardcode values of
>>> DEBIAN_BUILD_OPTIONS in debian/rules if they are used blidly.
>>
>> But more and more people want to be able to change distribution wide
>> default: Emdebian wants to enable "nodocs" and "nocheck" by default,
>> other want to be able to enable hardening options by default and I agree
>> with them that official support for such a facility is desirable.
>
> So they should set DEB_BUILD_OPTIONS in the environment.  That's what it's
> for.  I don't have any objections to that, or even to doing it via
> dpkg-buildpackage.

Setting the environment on a distribution wide level is ugly and
fragile. Too many users will reset the environment in their .bashrc.

Instead the idea was to have a vendor (set in
/etc/dpkg/origins/default) that will be exported into DEB_VENDOR if
unset and also set DEB_BUILD_OPTIONS to the vendor specifics defaults.

The bugreports relevant for this have 2 solutions:

1) make dpkg-buildpackage use (or tool with equivalent environment
   setting up capabilities) mandatory

2) have debian/rules call something to set DEB_VENDOR and possibly
   more

   E.g. 'include /usr/share/dpkg/Makefile.dpkg'
   or   'DEB_VENDOR?= (shell dpkg-vendor -qDEB_VENDOR)
 DEB_BUILD_OPTIONS ?= (shell dpkg-vendor -qDEB_BUILD_OPTIONS)

The argument against 2 is that is requires every source to be modified
if they want to support vendors whereas 1 only needs some small
modification to dpkg-buildpackage to support calling arbitrary targets
in debian/rules and a change in policy making its use mandatory.

> My objection is specifically to having dpkg-buildpackage set a variety of
> environment variables *by default*, and then telling package maintainers
> that they should rely on those environment variables being set in the
> default case.  That breaks the debian/rules interface and requires that
> all package builds go through dpkg-buildpackage.  Having dpkg-buildpackage
> set environment variables in the non-default case like Emdebian is not a
> problem, since for Emdebian builds (for example) Emdebian can decide that
> using dpkg-buildpackage or setting the environment variables manually is
> required.  There is no existing precedent, and they can make that rule
> from scratch.

Then you have one interface for Debian and one interface for every
other vendor including ubuntu (or option 2 above).

> My concern is for the default build where there *is* an existing precedent
> that debian/rules build should work sanely, not for support for special
> cases like that where the existing debian/rules interface already doesn't
> do the right thing without additional help.
>
> If you are going to go down this path of having dpkg-buildpackage set up
> an environment that package maintainers should rely on, you or someone
> else on the dpkg team needs to make a debian-devel-announce post making it
> clear that debian/rules build is no longer a supported interface for
> building packages and using dpkg-buildpackage is required for consistent
> behavior.

Plus a note in policy clarifying that debian/rules is only an
interface for dpkg-buildpackage but not users.

> Right now, I don't think most Debian Developers have any idea what the
> implications of these changes are.

I have to say i verry rarely do not use debuild. And 99% of the
exceptions are calling debian/rules clean.

MfG
Goswin

PS: with dpkg-buildpackage setting env vars like it does now you
already have a verry confusing situation.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Bug#489771: New Build-Options field and build-arch option, please review

[Neil Williams]

On Thu, 2008-09-11 at 10:53 -0700, Russ Allbery wrote:
> Raphael Hertzog <[EMAIL PROTECTED]> writes:
> > On Wed, 10 Sep 2008, Bill Allombert wrote:
> 
> >> I like to say I concurr with Russ. There are some much difference
> >> between packages that distributions wide default does not make sense.
> >> Such change would rather lead me to hardcode values of
> >> DEBIAN_BUILD_OPTIONS in debian/rules if they are used blidly.
> >
> > But more and more people want to be able to change distribution wide
> > default: Emdebian wants to enable "nodocs" and "nocheck" by default,
> > other want to be able to enable hardening options by default and I agree
> > with them that official support for such a facility is desirable.
> 
> So they should set DEB_BUILD_OPTIONS in the environment.  That's what it's
> for.  I don't have any objections to that, or even to doing it via
> dpkg-buildpackage.

That is what DEB_VENDOR seeks to achieve - set it once and it sets the
same options across all builds, in the environment.

This is getting to be a long list of CC: - isn't it worth sending to
debian-devel instead? Goswin von Brederlow and Simon Richter did a lot
of work on this at Extremadura and they aren't on the current CC:.

I'm losing track of all the bugs in the CC: and why they are listed.

> My objection is specifically to having dpkg-buildpackage set a variety of
> environment variables *by default*, and then telling package maintainers
> that they should rely on those environment variables being set in the
> default case. 

If by default you mean Debian, then NO. DEB_BUILD_OPTIONS is empty for
Debian and will continue so.

>  That breaks the debian/rules interface and requires that
> all package builds go through dpkg-buildpackage.  Having dpkg-buildpackage
> set environment variables in the non-default case like Emdebian is not a
> problem, since for Emdebian builds (for example) Emdebian can decide that
> using dpkg-buildpackage or setting the environment variables manually is
> required.  There is no existing precedent, and they can make that rule
> from scratch.

Exactly.

> My concern is for the default build where there *is* an existing precedent
> that debian/rules build should work sanely, not for support for special
> cases like that where the existing debian/rules interface already doesn't
> do the right thing without additional help.
> 
> If you are going to go down this path of having dpkg-buildpackage set up
> an environment that package maintainers should rely on, you or someone
> else on the dpkg team needs to make a debian-devel-announce post making it
> clear that debian/rules build is no longer a supported interface for
> building packages and using dpkg-buildpackage is required for consistent
> behavior.
> 
> Right now, I don't think most Debian Developers have any idea what the
> implications of these changes are.

That's fine. DEB_BUILD_OPTIONS would be empty if DEB_VENDOR is not set
or is set to Debian.

-- 


Neil Williams
=
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/




signature.asc
Description: This is a digitally signed message part

Re: Bug#489771: New Build-Options field and build-arch option, please review

[Russ Allbery]

Raphael Hertzog <[EMAIL PROTECTED]> writes:
> On Wed, 10 Sep 2008, Bill Allombert wrote:

>> I like to say I concurr with Russ. There are some much difference
>> between packages that distributions wide default does not make sense.
>> Such change would rather lead me to hardcode values of
>> DEBIAN_BUILD_OPTIONS in debian/rules if they are used blidly.
>
> But more and more people want to be able to change distribution wide
> default: Emdebian wants to enable "nodocs" and "nocheck" by default,
> other want to be able to enable hardening options by default and I agree
> with them that official support for such a facility is desirable.

So they should set DEB_BUILD_OPTIONS in the environment.  That's what it's
for.  I don't have any objections to that, or even to doing it via
dpkg-buildpackage.

My objection is specifically to having dpkg-buildpackage set a variety of
environment variables *by default*, and then telling package maintainers
that they should rely on those environment variables being set in the
default case.  That breaks the debian/rules interface and requires that
all package builds go through dpkg-buildpackage.  Having dpkg-buildpackage
set environment variables in the non-default case like Emdebian is not a
problem, since for Emdebian builds (for example) Emdebian can decide that
using dpkg-buildpackage or setting the environment variables manually is
required.  There is no existing precedent, and they can make that rule
from scratch.

My concern is for the default build where there *is* an existing precedent
that debian/rules build should work sanely, not for support for special
cases like that where the existing debian/rules interface already doesn't
do the right thing without additional help.

If you are going to go down this path of having dpkg-buildpackage set up
an environment that package maintainers should rely on, you or someone
else on the dpkg team needs to make a debian-devel-announce post making it
clear that debian/rules build is no longer a supported interface for
building packages and using dpkg-buildpackage is required for consistent
behavior.

Right now, I don't think most Debian Developers have any idea what the
implications of these changes are.

-- 
Russ Allbery ([EMAIL PROTECTED])   


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Bug#489771: New Build-Options field and build-arch option, please review

[Neil Williams]

On Thu, 2008-09-11 at 08:46 +0200, Raphael Hertzog wrote:
> On Wed, 10 Sep 2008, Bill Allombert wrote:
> > > > People have noticed that and already requested that we can call 
> > > > arbitrary
> > > > targets of debian/rules with all the proper initialization done 
> > > > precisely
> > > > for test purpose during packaging work (see #477916).
> > > 
> > > I must say, I really do not like this direction.  debhelper and cdbs and
> > > similar sytsems are the places to provide this help where people want to
> > > use it, in my opinion. 

The actual support will be implemented in debhelper - all that is needed
is a consistent manner to pass the same options to debhelper across a
range of packages. Packages then add extra support if necessary, e.g. if
a package installs docs manually instead of using dh_installdocs, then
those sections of debian/rules need to be either conditionally excluded:
ifeq (,$(findstring nodocs,$(DEB_BUILD_OPTIONS)))
install foo.1 debian/foo/usr/share/man/man1/
...
endif

or redone for debhelper support via foo.install files, etc.

After Lenny, I will be filing bugs for this support - at least for the
packages used in Emdebian.

>  We have a lot of past experience with that and we
> > > have the compatibility level to handle smoothing transitions.  (And to
> > > provide a way for people to never transition, I admit, and I see where
> > > that's the problem that you're solving, but I prefer that problem to the
> > > problems introduced by the instability of having the package build
> > > infrastructure change the input to the builds without coordination with
> > > the package.)

There has to be coordination with the package - the package needs to
support the build option, either explicitly or via debhelper. All CDBS
or any other layer needs to do is pass down the option to make it
accessible to debhelper (usually via DEB_BUILD_OPTIONS).

> > I like to say I concurr with Russ. There are some much difference
> > between packages that distributions wide default does not make sense.

On the contrary, there are clear divisions where distribution-wide build
options do make sense. Raphael correctly identifies nodocs and nocheck
as the current Emdebian distribution-wide build options. nodocs itself
needs to be refined to allow for graded levels of documentation
exclusion and other build options may change the build process itself -
all under the control of the particular package. If the package does not
understand the option, nothing happens.

e.g. Emdebian needs nodocs (or something as broad) that drops
everything, from README and TODO to changelog.gz and manpages during the
build, rather than after downloading. Preferably, nodocs would also lead
to the mandatory compression of copyright to save more space. It is not
new for DEB_BUILD_OPTIONS to break Debian Policy - supporting a
distribution-wide superset of options allows the use of the set to
conform to Emdebian Policy etc.

Other uses of options could use graduations so that examples are dropped
but not the rest, or just manpages or just HTML docs etc. Dpkg Classes
will help with graduations, as long as the distro can afford to remove
the files *after* the package has been downloaded.

> But more and more people want to be able to change distribution wide
> default: Emdebian wants to enable "nodocs" and "nocheck" by default, other
> want to be able to enable hardening options by default and I agree with
> them that official support for such a facility is desirable.
> 
> See also #498355 and #498380 for such requests from Emdebian.

Note also that Ubuntu are interested in supporting distribution-wide
build options.

-- 


Neil Williams
=
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/




signature.asc
Description: This is a digitally signed message part

Re: Bug#489771: New Build-Options field and build-arch option, please review

[Raphael Hertzog]

On Wed, 10 Sep 2008, Bill Allombert wrote:
> > > People have noticed that and already requested that we can call arbitrary
> > > targets of debian/rules with all the proper initialization done precisely
> > > for test purpose during packaging work (see #477916).
> > 
> > I must say, I really do not like this direction.  debhelper and cdbs and
> > similar sytsems are the places to provide this help where people want to
> > use it, in my opinion.  We have a lot of past experience with that and we
> > have the compatibility level to handle smoothing transitions.  (And to
> > provide a way for people to never transition, I admit, and I see where
> > that's the problem that you're solving, but I prefer that problem to the
> > problems introduced by the instability of having the package build
> > infrastructure change the input to the builds without coordination with
> > the package.)
> 
> I like to say I concurr with Russ. There are some much difference
> between packages that distributions wide default does not make sense.
> Such change would rather lead me to hardcode values of
> DEBIAN_BUILD_OPTIONS in debian/rules if they are used blidly.

But more and more people want to be able to change distribution wide
default: Emdebian wants to enable "nodocs" and "nocheck" by default, other
want to be able to enable hardening options by default and I agree with
them that official support for such a facility is desirable.

See also #498355 and #498380 for such requests from Emdebian.

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]