Re: New Build-Options field and build-arch option, please review

2008-07-14 Thread Russ Allbery
Raphael Hertzog <[EMAIL PROTECTED]> writes:

> I think we're already on that path for quite some time. If your package
> uses DEB_(BUILD|HOST)_* variables, you rely on dpkg-buildpackage setting
> them for you (with dpkg-architecture).

I most certainly do not rely on dpkg-buildpackage setting anything.  I
call dpkg-architecture directly, which is also what's in our best practice
documents.

DEB_HOST_GNU_TYPE  ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE)
DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)

I would consider packages that didn't do that and just assumed that those
variables were already set to be buggy.

> The same is expected with default values of builder/linker flags now
> that dpkg-buildpackage provides reasonable defaults.

Yeah, that bothered me too.  I made a perhaps poor tactical decision to
not fight about it since it seemed that it had a lot of momentum and I
couldn't think of specific problems other than the one that we ran into.
But this is going beyond setting some defaults that are already set in
nearly all of our packages.

> So yes, I'm somehow building on this model where dpkg-buildpackage can
> simplify the work of packager by providing some distribution-wide
> reasonable defaults.
>
> People have noticed that and already requested that we can call arbitrary
> targets of debian/rules with all the proper initialization done precisely
> for test purpose during packaging work (see #477916).

I must say, I really do not like this direction.  debhelper and cdbs and
similar sytsems are the places to provide this help where people want to
use it, in my opinion.  We have a lot of past experience with that and we
have the compatibility level to handle smoothing transitions.  (And to
provide a way for people to never transition, I admit, and I see where
that's the problem that you're solving, but I prefer that problem to the
problems introduced by the instability of having the package build
infrastructure change the input to the builds without coordination with
the package.)

Note that if you're requiring a package to participate by adding something
to Build-Options in debian/control, you have the same transition problem,
so I think that it's pretty equivalent to changing debian/rules; it's only
when you want packages to be able to change with external defaults that
you get the transition advantage.

I don't want to underestimate the transition advantage -- that is pretty
significant.  I do understand the problem that you're trying to solve, and
I understand that what I'm proposing is going to make transitions a lot
harder.

> Ok makes sense. In the case of hardening, it means that we have to
> modify each and every package to enable it though.

Well, not if you can do it via debhelper, which now with version 7 is much
more likely.  Similarly with cdbs.  But in general, yes.

For hardening, I think this is a feature; the flags aren't ones that can
just be applied to every package without breaking things.

> I suppose that the people pushing this proposal would like to have the
> option to enable it globally and have broken packages opt out and/or
> disable specific hardening options.

I think we've already found that this isn't a great approach for hardening
options in particular, since they break too many packages (and those
packages are not necessarily broken; in some cases it's the compiler
that's broken, or the assumptions behind the options).

> Without taking into account the specific risks associated to any default
> activation of build hardening, I find that having a generic system where
> you can start early with an opt-in policy, have the stuff matures, and
> switch to an opt-out policy later can make sense (if that plan is
> announced early and that people know by advance how to opt-out
> explicitely).

I agree with the benefit, but I think it's better to implement that sort
of thing in the packaging tools that already do that sort of magic and
which we already have a way of dealing with (compatibility levels in
debhelper, for example), and which continue to work with debian/rules
build.

>> See above.  By moving the logic from debian/rules into
>> dpkg-buildpackage, we would be breaking a common workflow when working
>> with packages.  Running debian/rules build in an unpacked source
>> package to test would no longer be a reasonable development step since
>> you may get a completely different compile than dpkg-buildpackage would
>> give you.

> That might be so, but I'm not sure why it would be a major problem. It
> can take some time to change habits but unless you see real drawbacks, I'm
> not convinced that there are good reasons to revert in that direction.

I'm somewhat disturbed by this.  Until this discussion, I had no idea that
you were planning on deprecating debian/rules build and expecting everyone
to use dpkg-buildpackage to get a reproducible build.  I'm not even sure
how to use dpkg-buildpackage to do the equivalent of just running
deb

Re: New Build-Options field and build-arch option, please review

2008-07-14 Thread Raphael Hertzog
On Sun, 13 Jul 2008, Russ Allbery wrote:
> > Because they want that anyone can easily rebuild it with that option
> > disabled?
> 
> That is already supported using the existing DEB_BUILD_OPTIONS mechanism.
> 
> I may be confused about your mental model here, but it seems like you're
> moving rules about how the package is built from the package itself into
> dpkg-buildpackage.  If that's really what's happening, I think that is a
> truly dreadful idea and strongly object.  It should be possible to build
> the package using whatever flags and options are the default by running
> debian/rules build without involving dpkg-buildpackage at all, which
> implies that the package should not be relying on dpkg-buildpackage to
> provide compiler and linker flags.  Those defaults should be in
> debian/rules, just as they always have been for Debian packages.

I think we're already on that path for quite some time. If your package
uses DEB_(BUILD|HOST)_* variables, you rely on dpkg-buildpackage setting
them for you (with dpkg-architecture). The same is expected with default
values of builder/linker flags now that dpkg-buildpackage provides
reasonable defaults.

So yes, I'm somehow building on this model where dpkg-buildpackage can
simplify the work of packager by providing some distribution-wide
reasonable defaults.

People have noticed that and already requested that we can call arbitrary
targets of debian/rules with all the proper initialization done precisely
for test purpose during packaging work (see #477916).

> If some set of flags, such as hardening, should be possible to easily
> disable, this is exactly the same case as we have right now with
> optimization and with stripping.  The way to support that is to specify
> another DEB_BUILD_OPTIONS flag which, if set, instructs the package to
> modify its behavior accordingly.  Furthermore, that allows the package
> maintainer to provide more useful defaults specific to that package, such
> as exactly the hardening flags that *that* package supports, rather than
> some default (and possibly changing) set from dpkg-buildpackage.

Ok makes sense. In the case of hardening, it means that we have to modify
each and every package to enable it though. I suppose that the people
pushing this proposal would like to have the option to enable it globally
and have broken packages opt out and/or disable specific hardening
options.

Without taking into account the specific risks associated to any default
activation of build hardening, I find that having a generic system where
you can start early with an opt-in policy, have the stuff matures, and
switch to an opt-out policy later can make sense (if that plan is
announced early and that people know by advance how to opt-out
explicitely).

> See above.  By moving the logic from debian/rules into dpkg-buildpackage,
> we would be breaking a common workflow when working with packages.
> Running debian/rules build in an unpacked source package to test would no
> longer be a reasonable development step since you may get a completely
> different compile than dpkg-buildpackage would give you.

That might be so, but I'm not sure why it would be a major problem. It
can take some time to change habits but unless you see real drawbacks, I'm
not convinced that there are good reasons to revert in that direction.

> I think the way that optimization and stripping are handled right now
> works fairly well in practice, and I think we should be building on that
> as a model, not replacing it with some entirely different method that
> relies on additional external programs to wrap debian/rules.
> 
> The choice between always and by default can be handled using the existing
> DEB_BUILD_OPTIONS mechanism just as optimization and stripping are now.

Well, right now buildd do not use DEB_BUILD_OPTIONS at all AFAIK. So
there's no way to enable anything globally with this method in practice.

And I certainly wouldn't want to have to manually set DEB_BUILD_OPTIONS to
get a build similar to what the buildd would do.

The current practice only has options to disable something that
is enabled by default. I'm not sure you can usefully build on that
to provide a mechanism where something is disabled by default and that
can be enabled either by the maintainer or by the builder.

But maybe such a scheme is not desirable in general, we might not want
to offer any option for the builder that has not been validated by the
maintainer. I don't know. Maybe we won't have any other situation similar
to the hardening one and it's over-kill to try to generalize it.


the

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]