Re: Environment variables, debian/rules and dpkg-buildpackage
Paul Wise wrote: It would be much nicer to discard maintainer-built packages and build everything on the buildds. Then we get build logs as well as the opportunity to replicate Ubuntu's automatically created debug debs. Yes! At least discard arch:any debs, so that we don't need to wait until building arch:all debs in the autobuilders is possible. Emilio signature.asc Description: OpenPGP digital signature
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 11, 2009 at 03:17:33PM +0200, Giacomo A. Catenazzi wrote: but I would like to have more :-) Currently I prefer to build package with --quiet flag, so that I see easily problems on building package: Neither debuild nor dpkg-buildpackage has a a --quiet flag, so I guess you weren't clear enough on what you are talking about. So I would like to have a short log (e.g. what I put in stdout/stderr, with ./configure --quiet), so that people will have no excuses for not be carefukll, but also to have access to configure.log (or/and other build time log), on build failure. OK, so I am not sure what this has to do with the thread. Are you proposing a completely different way of how autoconf packages should get configured in debian/rules? What does ./configure --quiet have to do with compiler warnings or deprecated features? Does it print out relevant parts of config.log on failure? If not, what is the point of configure.log, having the same information then than one has now in general? Is it to much to ask? You can ask anything, the question is whether it gets picked up. If what you ask for required changes in most source packages, it is likely that it will not get picked up. What I do think might make sense is changing dh7 and/or CDBS to print out config.log if configure fails; that might help debug problems on the autobuilders more quickly. Michael -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Tue, May 19, 2009 at 11:32:05AM +0200, Emilio Pozuelo Monfort wrote: Paul Wise wrote: It would be much nicer to discard maintainer-built packages and build everything on the buildds. Then we get build logs as well as the opportunity to replicate Ubuntu's automatically created debug debs. Yes! At least discard arch:any debs, so that we don't need to wait until building arch:all debs in the autobuilders is possible. It's possible right now. You just set $build_arch_all=1 in the sbuild configuration for a single arch (e.g. i386 or amd64). You would need some small changes in wanna-build to track arch-all packages and sources AFAICT, but it's not a huge task. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Michael Banck wrote: On Mon, May 11, 2009 at 03:17:33PM +0200, Giacomo A. Catenazzi wrote: So I would like to have a short log (e.g. what I put in stdout/stderr, with ./configure --quiet), so that people will have no excuses for not be carefukll, but also to have access to configure.log (or/and other build time log), on build failure. OK, so I am not sure what this has to do with the thread. Are you proposing a completely different way of how autoconf packages should get configured in debian/rules? What does ./configure --quiet have to do with compiler warnings or deprecated features? Does it print out relevant parts of config.log on failure? If not, what is the point of configure.log, having the same information then than one has now in general? Is it to much to ask? You can ask anything, the question is whether it gets picked up. If what you ask for required changes in most source packages, it is likely that it will not get picked up. What I do think might make sense is changing dh7 and/or CDBS to print out config.log if configure fails; that might help debug problems on the autobuilders more quickly. Ok, I was not so clear. I was replying to: Goswin von Brederlow wrote: Debuild already creates a build log. I think it would be nice to include that file in the changes file and have DAK forward it to buildd.debian.org for archival. Thus: my request was more about our infrastructure then the building tools. In my experience, the ./configure script are usually too verbose (whose created with autoconf in particulary). Maintainer tends not to parse carefully the output of such script, e.g. ask my sponsorees ;-) , missing important parts. So, if we centrally save the logs, I would like to allow maintainer to run configure with --quiet, but also to save extra informations for late processing. It is not only build failures, but the log (on file, which is much more verbose then normal output) contains also other useful informations: e.g. the compiler and other tool versions, etc. E.g. if we saw a bug (secutity or also miscompilation) in one package in the toolchain, we can found what package need to be recompiled (and in what architecture). I think such logs are more useful to archive, than the simple stdout/stderr. BTW I just discovered dh_buildinfo, which unfortunately has no manpage. ciao cate -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Tue, May 12 2009, Steve Langasek wrote: On Sun, May 10, 2009 at 11:30:43PM -0500, Manoj Srivastava wrote: I'm really surprised to see this approach getting traction. To me, this seems like a significant, unprecedented departure from the kinds of interfaces we've mandated in Policy in the past (i.e., environment variables, executables and command-line options). While one build I am happy to see this gain traction, really. And We have not done so in the past is a very illogical criteria to judge the technical merits of a proposal, I hope no one puts much weight on it. No, it's *surprising* because it's a departure from the mechanisms used in Policy in the past. But novelty is often a characteristic of innovative solutions. It's *bad* because using 'include' semantics for config files is Worst Practice. Sez You. I find it liberating, and far more flexible. helper or another may mandate Makefile includes, there's never been anything of the sort in Policy, and I don't think it's good to add such a thing now. I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. I find it a fine and flexible idea to implement configuration files using include mechanisms, including Perl load directives. Gives the end user full control (and yes, with full control comes great responsibility, Peter Parker would say) Why is a user who wants that kind of full control running Debian instead of Slackware? I expect Debian to provide sane delineation between supported and unsupported changes to the system, not encourage rampant foot-shooting. This seems to indicate more a failure of imagination on your part. You do not get to specify what users might want to do, or not, based on what you imagine they should be allowed to do. If there's any intention at all that Policy eventually mandate use of these Makefile includes, then at a minimum I think Policy needs to *very* tightly constrain what dpkg is allowed to put in those files, to avoid future incompatibilities. I think what dpkg sets in the env variables is a fine start. And policy should not be a stick to hit the dpkg developers on the head with. I say we let an implementation blossom in the wild, see what works, and then codif that in policy. Policy is not the place to do design in. I said that tightly-constrained config files should be a condition of mandating such includes. If you're not going to specify a reasonable interface, then fine - it doesn't need to be in Policy. The initial implementation is not going to be defined in policy, no. Once a working solution exists, and has stabilized, we may enshrine that in policy. But unfortunately, if we're going to support site files, we're in no position to enforce such requirements there; so packages are still subject to breakage from admins populating their site file with random settings (or syntax errors?). Right. The admins can break package compilation, and then they get to keep all the parts. I see no harm in this. We should get away from the Cathedral mode of looking at our end users as morons who need to have their hands held, and start trusting them to know what they do. If a site want s to do things that Debian Power Up On High consider to be The Wrong Thing, I think we should give the sites enough rope. Right - I'll be sure to forward all the resulting bug reports to you, then. So it is centralized control, or I get to deal with the bugs? Heh. Sounds like Ubuntu. manoj -- Stenderup's Law: The sooner you fall behind, the more time you will have to catch up. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Tue, May 12 2009, Steve Langasek wrote: On Mon, May 11, 2009 at 10:01:14AM -0700, Russ Allbery wrote: If they're built by the program, then anyone who wants to properly build the software, even if they don't want to go all the way to the package, will need to use the program, since people will write debian/rules such that it assumes the program is in use. They'll assume default CFLAGS are set and so forth. I don't think this is the right direction to go, but I'm not going to stomp off in a huff if we go that direction or anything. :) But I do want to be sure that we're all clear on what we're saying if we do take that approach and make dpkg-buildpackage the only supported way to build packages. I think it's likely that if we go that route, with it providing the defaults, we'll find over time that some packages will either not build or will mis-build with debian/rules build and no one will notice or be particularly interested in fixing it. That's a fair point, and if preserving the behavior of debian/rules as a standalone is important to others, then I'm happy for us to find a solution that meets this requirement *as long as* it also avoids the pain of letting mandated config files arbitrarily modify the behavior of debian/rules. I am not sure I understand this. In effect, using helper packages can allow arbitrary programs to change the build behaviour of the package, but we heartily recommend using the helper packages. But that is different you say, we do not expect debhelper to change incompatibly without changing compat levels, we trust joey hess to never do that. (I do too). But this also implies that you do not trust dpkg maintainers to not put in things in the configuration files without due delibration, and that they will not actually set up a mechanism somewhat like COMPAT_VERSION (very easy to do in make snippets, BTW). I think we ought to trust the dpkg folks. The other use case is the site admin building packages for themselves, and the buildd maintainer, to set up the environment for their build daemon. In other mails, you have argued that the only thing you are worried about is build daemons, in that case, there are scads of things that a build daemon admin can do that changes how packages are built (versions of build essential packages, stuff related to compiling in /etc, ld.so being just one) -- so we do implicitly trust the build daemon maintainers to do the right thing. So, the only places we distrust people is either the maintainer for not setting their site config right, and uploading. But if you distrust the maintainer, you might as well give it up, or set up a process to require but discard .debs created by ordinary mere mortal DD's, and let only the works of those on high (the buildd folks) to go into Debian. A losing battle, I would say. The other person you are so worried about braeking things is the end user, who might muck up their site config, and thus build bad packages. That the end user building packages for their own may wreak greater havoc with vim is overlooked, neh? What is this distrust of the configuration files all about, then? I find it strange, and I think it stems from a very Cathedral like approach of not trusting the end users to even be able to manage site config files flies in the face of the bazaar; we allow for free software adherents and our users to bui9ld packages, and let them tinker and make mistakes. The supposed control over people would not be able to afect how my package builds is already a mirage. A buildd configuration is already beyond most peoples control. A hacked debhelper is also beyond your control. One of the major shortcomings of the way we build things, and that Gentoo has a serious advantage, is the so called build flags -- with this include file approach, we can come close to the build flags approach, and let specialized build daemons be set up with site config different from the norm to build specialized binary archives. Being tied to tradition does not help us here. manoj -- Remember, UNIX spelled backwards is XINU. Mt. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 11 May 2009, Russ Allbery wrote: Raphael Hertzog hert...@debian.org writes: BTW, just to make things clear. It's likely that those Makefile snippet (if we decide to go that way) become quite more elaborated as we try integrating support for things like hardening-wrapper (see #489771). Expect stuff like if debian/control has Build-Options-Supported: hardening then use that set of flags, otherwise that one. I still think Build-Options-Supported is fundamentally the wrong way to implement that. You have to modify every package to add it anyway, in which case you can just as easily support it in the package's debian/rules the way that we already support various other DEB_BUILD_OPTIONS settings. Except that with the centralized approach we can have an opt-out policy after some time (ie use hardening options by default except for packages that have set no-hardening). My interest in centralizing those is simplifying any transition in default flags that Debian would want to do. User customization is nice but it's not what motivates me to push this forward. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 11 May 2009, Manoj Srivastava wrote: If you refer to the fact, that dpkg-buildpackage cleans and rebuilds everything and that it can take a lot of time, then please stop using arguments that do not hold at all. you can call arbitrary debian/rules targets with dpkg-buildpackage (without calling the clean rule before-hand) and I pointed that to you multiple times already. It's in dpkg 1.15.x (in experimental right now, in sid this week) and it's the --target option. Nice rant about a facility that is not yet in proper distribution (ustable, testing, or stable). You have however -nc since a long time and if you have proper Make dependencies it should be pretty quick by not redoing unnecessary work. Furthermore, we're designing stuff for the future so you have no reason to dismiss that fact even if it's not yet in sid. So, until a date in the future at this point, using dpkg-buildpackageis a resource hog, and no, you have not told me multiple times. Right I have not repeated it multiple times in the discussion because I try to avoid that but it was in the initial wiki page that started this discussion: http://wiki.debian.org/Teams/Dpkg/DebianRules Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: On Mon, 11 May 2009, Russ Allbery wrote: I still think Build-Options-Supported is fundamentally the wrong way to implement that. You have to modify every package to add it anyway, in which case you can just as easily support it in the package's debian/rules the way that we already support various other DEB_BUILD_OPTIONS settings. Except that with the centralized approach we can have an opt-out policy after some time (ie use hardening options by default except for packages that have set no-hardening). That seems orthogonal. Either way, you have to get most package maintainers to modify their packages and test to be sure that you can change the default build flags. Either way, the results of that change will produce artifacts that you can look for to see how many packages are currently building with the new flags. Either way, there is a way for maintainers to opt out of default flags. I understand conceptually how Build-Options-Supported could be useful, but I've yet to see a specific application for it that doesn't seem like it would be better done some other way. For example, saying that you support parallel builds there seems inferior to just enabling parallel builds if the DEB_BUILD_OPTIONS flag is set and you support it. Opting in to specific features similarly makes more sense as DEB_BUILD_OPTIONS to me. Opting out of default flags can be done with make's filter support. build-arch/build-indep should just be *done* rather than asking packages to say whether they support it, IMO. My interest in centralizing those is simplifying any transition in default flags that Debian would want to do. User customization is nice but it's not what motivates me to push this forward. Sure, that's the point of the whole discussion: how best to do that across the archive and whether we should break debian/rules build in order to do so. I think there's general agreement on the goal. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 11 May 2009, Russ Allbery wrote: Raphael Hertzog hert...@debian.org writes: On Mon, 11 May 2009, Russ Allbery wrote: I still think Build-Options-Supported is fundamentally the wrong way to implement that. You have to modify every package to add it anyway, in which case you can just as easily support it in the package's debian/rules the way that we already support various other DEB_BUILD_OPTIONS settings. Except that with the centralized approach we can have an opt-out policy after some time (ie use hardening options by default except for packages that have set no-hardening). That seems orthogonal. Either way, you have to get most package maintainers to modify their packages and test to be sure that you can change the default build flags. Either way, the results of that change will produce artifacts that you can look for to see how many packages are currently building with the new flags. Either way, there is a way for maintainers to opt out of default flags. The fact that we can filter out some default flags doesn't make it a better approach IMO. If you just want to disable hardening for your package, it would be a pain to have to filter out a possibly evolving list of default flags. And yes, it's best when all package maitainers test their package for the change, but quite a few are not as pro-active and you should not assume that we will modify all packages. To complete any migration, we must have the possibility to just do the change and manually fix up packages where nothing has been stated. (The approach might vary depending on the risks, etc but you get the idea) support. build-arch/build-indep should just be *done* rather than asking packages to say whether they support it, IMO. Which means that policy must state MUST for those targets to exist. In which case the Build-Options-Supported is implicit and derived from the Standards-Version. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: On Mon, 11 May 2009, Russ Allbery wrote: That seems orthogonal. Either way, you have to get most package maintainers to modify their packages and test to be sure that you can change the default build flags. Either way, the results of that change will produce artifacts that you can look for to see how many packages are currently building with the new flags. Either way, there is a way for maintainers to opt out of default flags. The fact that we can filter out some default flags doesn't make it a better approach IMO. If you just want to disable hardening for your package, it would be a pain to have to filter out a possibly evolving list of default flags. Why would you want to disable all hardening instead of filtering out the flag that breaks the package? And yes, it's best when all package maitainers test their package for the change, but quite a few are not as pro-active and you should not assume that we will modify all packages. To complete any migration, we must have the possibility to just do the change and manually fix up packages where nothing has been stated. Which again I agree with but it's orthogonal to what you're talking about. You can do that either way. Which means that policy must state MUST for those targets to exist. In which case the Build-Options-Supported is implicit and derived from the Standards-Version. In which case again you're not using Build-Options-Supported. Besides, I'm skeptical that's the right way to handle that transition. It seems to me like it would be a lot more effective to add a Lintian test first, warn everyone, do a mass bug filing, and otherwise follow the same practice we always use for global archive changes. That's the procedure that we follow when we're trying to do something that changes large numbers of packages. Basing it off Standards-Version to my mind just creates FTBFS landmines in the archive that will be exploding for years to come as people update old packages and don't think to add build-arch. I suppose that has the advantage of spreading the pain out across many years, but wouldn't it be better to get it over with? -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, May 10, 2009 at 11:30:43PM -0500, Manoj Srivastava wrote: I'm really surprised to see this approach getting traction. To me, this seems like a significant, unprecedented departure from the kinds of interfaces we've mandated in Policy in the past (i.e., environment variables, executables and command-line options). While one build I am happy to see this gain traction, really. And We have not done so in the past is a very illogical criteria to judge the technical merits of a proposal, I hope no one puts much weight on it. No, it's *surprising* because it's a departure from the mechanisms used in Policy in the past. It's *bad* because using 'include' semantics for config files is Worst Practice. helper or another may mandate Makefile includes, there's never been anything of the sort in Policy, and I don't think it's good to add such a thing now. I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. I find it a fine and flexible idea to implement configuration files using include mechanisms, including Perl load directives. Gives the end user full control (and yes, with full control comes great responsibility, Peter Parker would say) Why is a user who wants that kind of full control running Debian instead of Slackware? I expect Debian to provide sane delineation between supported and unsupported changes to the system, not encourage rampant foot-shooting. If there's any intention at all that Policy eventually mandate use of these Makefile includes, then at a minimum I think Policy needs to *very* tightly constrain what dpkg is allowed to put in those files, to avoid future incompatibilities. I think what dpkg sets in the env variables is a fine start. And policy should not be a stick to hit the dpkg developers on the head with. I say we let an implementation blossom in the wild, see what works, and then codif that in policy. Policy is not the place to do design in. I said that tightly-constrained config files should be a condition of mandating such includes. If you're not going to specify a reasonable interface, then fine - it doesn't need to be in Policy. But unfortunately, if we're going to support site files, we're in no position to enforce such requirements there; so packages are still subject to breakage from admins populating their site file with random settings (or syntax errors?). Right. The admins can break package compilation, and then they get to keep all the parts. I see no harm in this. We should get away from the Cathedral mode of looking at our end users as morons who need to have their hands held, and start trusting them to know what they do. If a site want s to do things that Debian Power Up On High consider to be The Wrong Thing, I think we should give the sites enough rope. Right - I'll be sure to forward all the resulting bug reports to you, then. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, May 10, 2009 at 11:41:30PM -0500, Manoj Srivastava wrote: The only builds Debian supports are not just the buildd ones. As members of the free software community, we should also cater to end users building, tweaking, and rebuilding our software. The only builds Debian supports are the ones that are distributed from the archive. Debian doesn't support any other builds. Some other builds will work and be compatible with Debian. Others won't, and if they don't, it's left to the discretion of the individual maintainer whether to help. Or does Debian support all of the unmodified source packages that are rebuilt for Ubuntu? That would be news to me. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Steve Langasek wrote: On Sun, May 10, 2009 at 11:41:30PM -0500, Manoj Srivastava wrote: The only builds Debian supports are not just the buildd ones. As members of the free software community, we should also cater to end users building, tweaking, and rebuilding our software. The only builds Debian supports are the ones that are distributed from the archive. Debian doesn't support any other builds. Some other builds will work and be compatible with Debian. Others won't, and if they don't, it's left to the discretion of the individual maintainer whether to help. We support more than just building for the archive: E.g. we want to support user to rebuild packages with no optimization and debug symbols, to help us to find out bugs. See also http://debian.org/doc/debian-policy/ch-source.html#s-debianrules-options ciao cate -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Russ Allbery wrote: Giacomo A. Catenazzi c...@debian.org writes: Manoj Srivastava wrote: The only builds Debian supports are not just the buildd ones. As members of the free software community, we should also cater to end users building, tweaking, and rebuilding our software. You are a very special case: a developer since very long time, with a enormous knowledge of debian policy (and dpkg internal). But I really think that most people outside DD use dpkg-buildpackage because it is the easier way (without need to remember a lot of details). I think also that most of DDs use dpkg-buildpackage. The number of people using different methods is fairly irrelevant to Manoj's point. The question is more fundamental: are packages built by a makefile that we call debian/rules, or are they built by the program dpkg-buildpackage using debian/rules as a configuration file for that program? I see. Anyway I don't see it as configuration file, but as environment for debian/rules. Note: currently we use fakeroot in order to setup other things of environment. It is not mandatory (we can build deb as superuser), but also dpkg-buildpackage will not be mandatory (if user set some environment variable). Could maintainer do a sensible choice of CFLAGS? I don't think so. The flags are too much dependent on architecture then the theoretical meaning. - Support of some flags on some architecture is broken (according to documentation) - optimization is not safe on some architectures (broken compiler) or it worse the code (broken default choices) - debug symbol format changes in different architectures: we need to use -g, -ggdb or what? For these reasons I see CFLAGS more as an architectural choice than a maintainer choice. But I'm not an expert on the field, and I've no packages on core Debian, where flags could be more interesting. And in the thread, it seems that there are such cases. I'm currious to see what are such cases (maintainer MUST override flags) and the reason. In every case, I think the .dsc should include some build information, to block earlier some build errors (e.g. include with custom setting, or dpkg-buildpackage non Debian-standard settings). We will do errors, so we need also a way to find out such errors. ciao cate -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Tue, May 12, 2009 at 10:12:20AM +0200, Giacomo A. Catenazzi wrote: Steve Langasek wrote: On Sun, May 10, 2009 at 11:41:30PM -0500, Manoj Srivastava wrote: The only builds Debian supports are not just the buildd ones. As members of the free software community, we should also cater to end users building, tweaking, and rebuilding our software. The only builds Debian supports are the ones that are distributed from the archive. Debian doesn't support any other builds. Some other builds will work and be compatible with Debian. Others won't, and if they don't, it's left to the discretion of the individual maintainer whether to help. We support more than just building for the archive: E.g. we want to support user to rebuild packages with no optimization and debug symbols, to help us to find out bugs. See also http://debian.org/doc/debian-policy/ch-source.html#s-debianrules-options a) this is a Policy recommendation, not a requirement b) if anyone rebuilt a package in such a way for their own purposes, then filed a bug report that the package wasn't working and it came to light in the process of triaging the bug that the user had done this, the bug would almost certainly be given the lowest priority by the package maintainer and the user would probably be told to use the official binaries. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 11, 2009 at 10:01:14AM -0700, Russ Allbery wrote: If they're built by the program, then anyone who wants to properly build the software, even if they don't want to go all the way to the package, will need to use the program, since people will write debian/rules such that it assumes the program is in use. They'll assume default CFLAGS are set and so forth. I don't think this is the right direction to go, but I'm not going to stomp off in a huff if we go that direction or anything. :) But I do want to be sure that we're all clear on what we're saying if we do take that approach and make dpkg-buildpackage the only supported way to build packages. I think it's likely that if we go that route, with it providing the defaults, we'll find over time that some packages will either not build or will mis-build with debian/rules build and no one will notice or be particularly interested in fixing it. That's a fair point, and if preserving the behavior of debian/rules as a standalone is important to others, then I'm happy for us to find a solution that meets this requirement *as long as* it also avoids the pain of letting mandated config files arbitrarily modify the behavior of debian/rules. Robert Collins' suggestion in 1241989292.8026.20.ca...@lifeless-64 seems like a good approach for this, then (modulo the syntax bits, and putting the tool in the dpkg-* namespace instead of debhelper's namespace). -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Tue, 12 May 2009, Russ Allbery wrote: The fact that we can filter out some default flags doesn't make it a better approach IMO. If you just want to disable hardening for your package, it would be a pain to have to filter out a possibly evolving list of default flags. Why would you want to disable all hardening instead of filtering out the flag that breaks the package? Because no-one has identified the precise flag that breaks the package? Or because you really want no hardening at all because you don't want to have to deal with subtle breakages caused by a supplementary hardening flags added later? I know that we always have many different opinions when it comes to implement some new stuff and I'd rather have some supplementary flexibility that is not needed rather than finding atferwards that we need more of it because when it concerns the whole archive, it's painful. And yes, it's best when all package maitainers test their package for the change, but quite a few are not as pro-active and you should not assume that we will modify all packages. To complete any migration, we must have the possibility to just do the change and manually fix up packages where nothing has been stated. Which again I agree with but it's orthogonal to what you're talking about. You can do that either way. We suppose all packages already have the right include that imports the default cflags. So now, you decide it's time to use hardened flags by default so you just change the default flags. Is that right? How did we enable hardening flags on individual package before (in your scenario)? (Do we have a copy of all hardened flags in each package? if not where do they come from since the default flags don't contain them?) What were maintainers supposed to do in advance to disable some hardening flags that break their packages given that they don't even know for sure what the default flags will look like. Having stuff in place and just switching a test looks more predictable than not having it and simply relying on some d-d-a mail stating: in X months the default flags will be Y, try it out and make sure your package don't break when we do it. (But maybe I guessed this all wrong and you had something else in mind) It seems to me like it would be a lot more effective to add a Lintian test first, warn everyone, do a mass bug filing, and otherwise follow the same practice we always use for global archive changes. That's the procedure that we follow when we're trying to do something that changes large numbers of packages. Basing it off Standards-Version to my mind just creates FTBFS landmines in the archive that will be exploding for years to come as people update old packages and don't think to add build-arch. I suppose that has the advantage of spreading the pain out across many years, but wouldn't it be better to get it over with? Not sure at all. In part because only a small percentage of the packages would benefit from it (those that build arch: all and arch: any and where the work to compile doc for the arch: all is expensive). So targetting a MUST in policy is overkill in my opinion. Maybe the right thing to do is to make it mandatory only for packages that have both arch-indep and arch-specific binary packages. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Giacomo A. Catenazzi c...@debian.org writes: Could maintainer do a sensible choice of CFLAGS? I don't think so. The flags are too much dependent on architecture then the theoretical meaning. Well, maintainers are doing so now and have been since pretty much the beginning of the packaging format, so clearly that answer is yes insofar as you think the current Debian archive is sensible. dpkg-buildpackage setting CFLAGS is a recent change. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Steve Langasek vor...@debian.org writes: Robert Collins' suggestion in 1241989292.8026.20.ca...@lifeless-64 seems like a good approach for this, then (modulo the syntax bits, and putting the tool in the dpkg-* namespace instead of debhelper's namespace). I like this idea on first glance. It matches what we currently recommend with dpkg-architecture. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: On Tue, 12 May 2009, Russ Allbery wrote: Why would you want to disable all hardening instead of filtering out the flag that breaks the package? Because no-one has identified the precise flag that breaks the package? Then filter out the ones that might cause the problem. Last I heard, we're only talking about two or three flags here and they weren't changing particularly fast. Or because you really want no hardening at all because you don't want to have to deal with subtle breakages caused by a supplementary hardening flags added later? It sounds like you're thinking that we could potentially change the hardening flags down the road without going through the same checking and care that we did when introducing them in the first place. I'm fairly sure that's unrealistic. I know that we always have many different opinions when it comes to implement some new stuff and I'd rather have some supplementary flexibility that is not needed rather than finding atferwards that we need more of it because when it concerns the whole archive, it's painful. I think that having a big switch that turns on (or off) some unrestricted set of hardening flags that may change in the future is not horribly useful flexibility. We suppose all packages already have the right include that imports the default cflags. So now, you decide it's time to use hardened flags by default so you just change the default flags. Is that right? Correct. Obviously you'd have to do a lot of preparation before doing that sort of global change. How did we enable hardening flags on individual package before (in your scenario)? If DEB_BUILD_OPTIONS contains hardening, the package would add the contents of DEB_BUILD_HARDENING_CFLAGS to CFLAGS and DEB_BUILD_HARDENING_LDFLAGS to LDFLAGS. (For example. There are other ways it could be done, but that's the one that comes to mind.) At some point we'd add a nohardening DEB_BUILD_OPTIONS setting that would disable hardening flags for people who wanted to do a build without hardening and then add the hardening flags to the default flags iff that option isn't set. (Assuming we decide to make hardening the default.) What were maintainers supposed to do in advance to disable some hardening flags that break their packages given that they don't even know for sure what the default flags will look like. Surely that's not a situation anyone would be in. It supposes that we'll be in a situation where we're about to make hardening the global default and yet we don't know what flags will be in it. If we're in that situation, we screwed up, and we need to go back and finalize the flags before we talk about changing the defaults. Not sure at all. In part because only a small percentage of the packages would benefit from it (those that build arch: all and arch: any and where the work to compile doc for the arch: all is expensive). So targetting a MUST in policy is overkill in my opinion. That would argue for not doing it for all packages and instead providing some way to say it's supported, and yeah, that's the case where I can see having Build-Options-Supported. Although I also think it's becoming increasingly easy to add the build-arch and build-indep targets. If one added them to debhelper's rule minimization and cdbs, you'd already catch a large percentage of the archive now. Either way, we shouldn't base it off Standards-Version; that bothers me much more than Build-Options-Supported does. :) Maybe the right thing to do is to make it mandatory only for packages that have both arch-indep and arch-specific binary packages. It's really fairly rare that this matters that much, so yeah, the more I think about it, the more you're probably right and this is a case where the package should be able to communicate back to the build system that it supports doing something. I have a ton of packages with both arch-independent and arch-specific binary packages, but only one of them would gain anything from this split. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, 10 May 2009, Manoj Srivastava wrote: I would prefer Debian to remain a full fledged member of the free software community, and continue to not let build behaviour diverge whether or not dpkg-buildpackage was used -- which can be a substancial resource hog for multiple binary source packages. resource hog?? If you refer to the fact, that dpkg-buildpackage cleans and rebuilds everything and that it can take a lot of time, then please stop using arguments that do not hold at all. you can call arbitrary debian/rules targets with dpkg-buildpackage (without calling the clean rule before-hand) and I pointed that to you multiple times already. It's in dpkg 1.15.x (in experimental right now, in sid this week) and it's the --target option. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Monday 11 May 2009 00:06:09 Steve Langasek wrote: Or maybe I've misunderstood, and there are Debian developers who are building official packages for *upload* by calling debian/rules by hand, and that's what people are concerned about preserving while still getting the benefits of these distro build flags? I hadn't considered that possibility, because I can't imagine anyone wanting to build packages that way instead of using dpkg-buildpackage, which does it all in a single command. So I really don't consider that an important use case, weighed against the concerns I outlined. I don't either, but it would probably be better to spell that out explicitly somewhere. For example, you possibly get something different depending on whether you call debian/rules, dpkg-buildpackage, debuild, or pbuilder. And the difference is hard to explain or analyze. Er, both debuild and pbuilder invoke dpkg-buildpackage. So it seems clear to me that the only difference would be when calling debian/rules directly, and at that point you're opting out of lots of other conveniences, not just distro build policy. Well, debuild calls dpkg-buildpackage most of the time, unless you give a specific target (which would again possibly be of interest to those who are interested in calling debian/rules by hand for some reason). And that is also something newish. Plus, you can set separate environment variables for debuild. And probably also for pbuilder. And you can set environment variables or possibly site files within the pbuilder chroot. And there is also the option of pbuilder calling debuild -- who knows what that really does. So again some of this would become clearer if the actually supported build methods are more clearly spelled out. And then if someone could fold all of the functionality of debuild into dpkg-buildpackage, there would be even less distraction and variation. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, 10 May 2009, Manoj Srivastava wrote: If there's any intention at all that Policy eventually mandate use of these Makefile includes, then at a minimum I think Policy needs to *very* tightly constrain what dpkg is allowed to put in those files, to avoid future incompatibilities. I think what dpkg sets in the env variables is a fine start. And policy should not be a stick to hit the dpkg developers on the head with. I say we let an implementation blossom in the wild, see what works, and then codif that in policy. BTW, just to make things clear. It's likely that those Makefile snippet (if we decide to go that way) become quite more elaborated as we try integrating support for things like hardening-wrapper (see #489771). Expect stuff like if debian/control has Build-Options-Supported: hardening then use that set of flags, otherwise that one. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 11 May 2009, Peter Eisentraut wrote: Well, debuild calls dpkg-buildpackage most of the time, unless you give a specific target (which would again possibly be of interest to those who are interested in calling debian/rules by hand for some reason). And that is also something newish. Any pointer to this feature? So again some of this would become clearer if the actually supported build methods are more clearly spelled out. And then if someone could fold all of the functionality of debuild into dpkg-buildpackage, there would be even less distraction and variation. It's planned, see #476221. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Monday 11 May 2009 09:49:31 Raphael Hertzog wrote: On Mon, 11 May 2009, Peter Eisentraut wrote: Well, debuild calls dpkg-buildpackage most of the time, unless you give a specific target (which would again possibly be of interest to those who are interested in calling debian/rules by hand for some reason). And that is also something newish. Any pointer to this feature? Um, didn't you yourself orginally report this? (bug #476100) Anyway, the current man pages of debuild says this: An alternative way of using debuild is to use one or more of the parameters binary, binary-arch, binary-indep and clean, in which case debuild will attempt to gain root privileges and then run debian/rules with the given parameters. So again some of this would become clearer if the actually supported build methods are more clearly spelled out. And then if someone could fold all of the functionality of debuild into dpkg-buildpackage, there would be even less distraction and variation. It's planned, see #476221. That sounds like the ticket. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Goswin von Brederlow wrote: Holger Levsen hol...@layer-acht.org writes: Hi, On Sonntag, 10. Mai 2009, Raphael Hertzog wrote: With the include approach, we lack this feature and bad/broken local overrides can't be detected if we only have the build log at hand. which reminds me that we dont have build logs for probably a lot more than 25% (*) of the most used packages in the archive. The ones build manually by the developer... I would really like to see that binaries for all archs are (re-)build on builds and that those logs are kept archived and linked from the PTS. (And arch all too of course). regards, Holger (*) wild guess, asuming that most packages are upload on amd64/i386/powerpc and mostly used on i386. Debuild already creates a build log. I think it would be nice to include that file in the changes file and have DAK forward it to buildd.debian.org for archival. git-buildpackage, svn-buildpackage, ... or even dpkg-buildpackage could do this too. but I would like to have more :-) Currently I prefer to build package with --quiet flag, so that I see easily problems on building package: I see that a lot of my sponsoree forgot to check *carefully* logs, to look for deprecated features, or important but non fatal warnings. So I would like to have a short log (e.g. what I put in stdout/stderr, with ./configure --quiet), so that people will have no excuses for not be carefukll, but also to have access to configure.log (or/and other build time log), on build failure. Is it to much to ask? ciao cate -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Manoj Srivastava wrote: On Sun, May 10 2009, Steve Langasek wrote: On Sun, May 10, 2009 at 11:37:46PM +0300, Peter Eisentraut wrote: On Sunday 10 May 2009 13:56:04 Steve Langasek wrote: I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. Guillem pointed out one problem: Either you do it via a make include (which you have issues with), or you stop supporting calling debian/rules directly (inconvenient, probably prone to break things) I don't agree that dpkg-buildpackage sets additional environment variables to implement a distro/site policy for builds implies calling debian/rules directly is unsupported. Or maybe I've misunderstood, and there are Debian developers who are building official packages for *upload* by calling debian/rules by hand, and that's what people are concerned about preserving while still getting the benefits of these distro build flags? The only builds Debian supports are not just the buildd ones. As members of the free software community, we should also cater to end users building, tweaking, and rebuilding our software. You are a very special case: a developer since very long time, with a enormous knowledge of debian policy (and dpkg internal). But I really think that most people outside DD use dpkg-buildpackage because it is the easier way (without need to remember a lot of details). I think also that most of DDs use dpkg-buildpackage. Note: it is also used to do make clean before *any* build, because the non traced dependencies. E.g. on old time, some kernel configurations needed a make clean. I don't think I want to remember when I need to run make clean, so any changes on compiler, on configuration or other heavy changes I run make clean. I think for the same reason people (but people very deep in debian packaging) will use dpkg-buildpackage: not the optimal way, but it is safer, and with less surprises. A new point, still neglected on this thread: there are other nasty cases: as we saw in an other thread on debian-policy, the current locale changes the way wide-characters (and wide strings) are encoded in binaries (UTF-16 if current charset in current locale could be encoded in 16-bit; UTF-32 on other cases, and fortunately when no charset is specified on current locale, thus also on C locale). Wide characters are not common, but they are specified not only in POSIX but also in standard C (also in C89/C90 version IIRC). I don't see a good way that: - developers will do the right things (how many of you doesn know such strangeness?) [ add a flag to CFLAG in every debian/rules is IMHO not the right thing ] - it is documented (e.g. on logs), actually we need to parse binaries to find what type of encoding did the original uploader (not so complex because we have only currently two encoding, but anyway...] - security team and MNU doesn't break package because of nasty interaction because of different locale. other than using dpkg-buildpackage (and letting dpkg-buildpackage to set this (and probably other nearly-unknow paramenters) ciao cate -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 11 2009, Raphael Hertzog wrote: On Sun, 10 May 2009, Manoj Srivastava wrote: I would prefer Debian to remain a full fledged member of the free software community, and continue to not let build behaviour diverge whether or not dpkg-buildpackage was used -- which can be a substancial resource hog for multiple binary source packages. resource hog?? That is indeed what I said. If you refer to the fact, that dpkg-buildpackage cleans and rebuilds everything and that it can take a lot of time, then please stop using arguments that do not hold at all. you can call arbitrary debian/rules targets with dpkg-buildpackage (without calling the clean rule before-hand) and I pointed that to you multiple times already. It's in dpkg 1.15.x (in experimental right now, in sid this week) and it's the --target option. Nice rant about a facility that is not yet in proper distribution (ustable, testing, or stable). So, until a date in the future at this point, using dpkg-buildpackageis a resource hog, and no, you have not told me multiple times. manoj -- And remember: Evil will always prevail, because Good is dumb. Spaceballs Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: BTW, just to make things clear. It's likely that those Makefile snippet (if we decide to go that way) become quite more elaborated as we try integrating support for things like hardening-wrapper (see #489771). Expect stuff like if debian/control has Build-Options-Supported: hardening then use that set of flags, otherwise that one. I still think Build-Options-Supported is fundamentally the wrong way to implement that. You have to modify every package to add it anyway, in which case you can just as easily support it in the package's debian/rules the way that we already support various other DEB_BUILD_OPTIONS settings. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Giacomo A. Catenazzi c...@debian.org writes: Manoj Srivastava wrote: The only builds Debian supports are not just the buildd ones. As members of the free software community, we should also cater to end users building, tweaking, and rebuilding our software. You are a very special case: a developer since very long time, with a enormous knowledge of debian policy (and dpkg internal). But I really think that most people outside DD use dpkg-buildpackage because it is the easier way (without need to remember a lot of details). I think also that most of DDs use dpkg-buildpackage. The number of people using different methods is fairly irrelevant to Manoj's point. The question is more fundamental: are packages built by a makefile that we call debian/rules, or are they built by the program dpkg-buildpackage using debian/rules as a configuration file for that program? If they're built by the program, then anyone who wants to properly build the software, even if they don't want to go all the way to the package, will need to use the program, since people will write debian/rules such that it assumes the program is in use. They'll assume default CFLAGS are set and so forth. I don't think this is the right direction to go, but I'm not going to stomp off in a huff if we go that direction or anything. :) But I do want to be sure that we're all clear on what we're saying if we do take that approach and make dpkg-buildpackage the only supported way to build packages. I think it's likely that if we go that route, with it providing the defaults, we'll find over time that some packages will either not build or will mis-build with debian/rules build and no one will notice or be particularly interested in fixing it. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 11, 2009 at 03:43:41PM +0200, Giacomo A. Catenazzi wrote: You are a very special case: a developer since very long time, with a enormous knowledge of debian policy (and dpkg internal). But I really think that most people outside DD use dpkg-buildpackage because it is the easier way (without need to remember a lot of details). I think also that most of DDs use dpkg-buildpackage. My experiences are quite the contrary: people who are not deeply involved with Debian tend to run debian/rules directly, and the only Debian-specific command they know is dpkg -i --force-depends... Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 04, 2009 at 07:35:18AM +0200, Guillem Jover wrote: * We can set the architecture and default flags (from policy) on the makefile to be included, and packagers will be able to do the change and fix any possible problems (progressive opt-in), but once it's included by all packages, then we can do system-wide default changes in the same we change toolchains (mass rebuild, bug filing, change when bug count goes down). The makefile has the advantage that the distro default can be temporarily changed for the mass build w/o needing to totally override the build flags. I'm really surprised to see this approach getting traction. To me, this seems like a significant, unprecedented departure from the kinds of interfaces we've mandated in Policy in the past (i.e., environment variables, executables and command-line options). While one build helper or another may mandate Makefile includes, there's never been anything of the sort in Policy, and I don't think it's good to add such a thing now. I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. If there's any intention at all that Policy eventually mandate use of these Makefile includes, then at a minimum I think Policy needs to *very* tightly constrain what dpkg is allowed to put in those files, to avoid future incompatibilities. But unfortunately, if we're going to support site files, we're in no position to enforce such requirements there; so packages are still subject to breakage from admins populating their site file with random settings (or syntax errors?). -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, May 10, 2009, Steve Langasek wrote: I'm really surprised to see this approach getting traction. To me, this seems like a significant, unprecedented departure from the kinds of interfaces we've mandated in Policy in the past (i.e., environment variables, executables and command-line options). While one build helper or another may mandate Makefile includes, there's never been anything of the sort in Policy, and I don't think it's good to add such a thing now. I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. If there's any intention at all that Policy eventually mandate use of these Makefile includes, then at a minimum I think Policy needs to *very* tightly constrain what dpkg is allowed to put in those files, to avoid future incompatibilities. But unfortunately, if we're going to support site files, we're in no position to enforce such requirements there; so packages are still subject to breakage from admins populating their site file with random settings (or syntax errors?). Full ack, thanks for putting it so clearly -- Loïc Minier -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, 10 May 2009, Steve Langasek wrote: I'm really surprised to see this approach getting traction. To me, this seems like a significant, unprecedented departure from the kinds of interfaces we've mandated in Policy in the past (i.e., environment variables, executables and command-line options). While one build helper or another may mandate Makefile includes, there's never been anything of the sort in Policy, and I don't think it's good to add such a thing now. I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. Another negative aspect of the include approach is the lack of tracability. Right now when the user overrides a build flag it appears in the build log since dpkg-buildpackage prints it out in the log: dpkg-buildpackage: use CFLAGS from environment: -O3 -Wall With the include approach, we lack this feature and bad/broken local overrides can't be detected if we only have the build log at hand. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, May 10, 2009 at 09:54:11PM +0200, Raphael Hertzog wrote: On Sun, 10 May 2009, Steve Langasek wrote: I'm really surprised to see this approach getting traction. To me, this seems like a significant, unprecedented departure from the kinds of interfaces we've mandated in Policy in the past (i.e., environment variables, executables and command-line options). While one build helper or another may mandate Makefile includes, there's never been anything of the sort in Policy, and I don't think it's good to add such a thing now. I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. I have sympathy for Steve viewpoint however it lacks an alternative proposal. However I cannot only strongly disagree with Raphael argument below: Another negative aspect of the include approach is the lack of tracability. Right now when the user overrides a build flag it appears in the build log since dpkg-buildpackage prints it out in the log: dpkg-buildpackage: use CFLAGS from environment: -O3 -Wall With the include approach, we lack this feature and bad/broken local overrides can't be detected if we only have the build log at hand. There is nothing that prevents a future dpkg-buildpackage to cat to stdout the relevant files at startup so that they appear in the buildlog. Cheers, -- Bill. ballo...@debian.org Imagine a large red swirl here. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sunday 10 May 2009 13:56:04 Steve Langasek wrote: I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. Guillem pointed out one problem: Either you do it via a make include (which you have issues with), or you stop supporting calling debian/rules directly (inconvenient, probably prone to break things), or you require every package to handle it itself (unreliable, stupid) -- or you have the current situation, which is somewhere in the middle. For example, you possibly get something different depending on whether you call debian/rules, dpkg-buildpackage, debuild, or pbuilder. And the difference is hard to explain or analyze. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Hi, On Sonntag, 10. Mai 2009, Raphael Hertzog wrote: With the include approach, we lack this feature and bad/broken local overrides can't be detected if we only have the build log at hand. which reminds me that we dont have build logs for probably a lot more than 25% (*) of the most used packages in the archive. The ones build manually by the developer... I would really like to see that binaries for all archs are (re-)build on builds and that those logs are kept archived and linked from the PTS. (And arch all too of course). regards, Holger (*) wild guess, asuming that most packages are upload on amd64/i386/powerpc and mostly used on i386. signature.asc Description: This is a digitally signed message part.
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, 2009-05-10 at 23:37 +0300, Peter Eisentraut wrote: On Sunday 10 May 2009 13:56:04 Steve Langasek wrote: I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. Guillem pointed out one problem: Either you do it via a make include (which you have issues with), or you stop supporting calling debian/rules directly (inconvenient, probably prone to break things), or you require every package to handle it itself (unreliable, stupid) -- or you have the current situation, which is somewhere in the middle. For example, you possibly get something different depending on whether you call debian/rules, dpkg-buildpackage, debuild, or pbuilder. And the difference is hard to explain or analyze. What about having a dh_config - e.g. CFLAGS := dh_config CFLAGS That can look it up via a config file, environment variable etc, and returns the decided answer. Should be consistent for all ways of invoking, and we can put a stock set of calls to this in a makefile fragment. including the code to do config files is different to doing config files by including :). -Rob signature.asc Description: This is a digitally signed message part
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, May 10, 2009 at 11:37:46PM +0300, Peter Eisentraut wrote: On Sunday 10 May 2009 13:56:04 Steve Langasek wrote: I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. Guillem pointed out one problem: Either you do it via a make include (which you have issues with), or you stop supporting calling debian/rules directly (inconvenient, probably prone to break things) I don't agree that dpkg-buildpackage sets additional environment variables to implement a distro/site policy for builds implies calling debian/rules directly is unsupported. Or maybe I've misunderstood, and there are Debian developers who are building official packages for *upload* by calling debian/rules by hand, and that's what people are concerned about preserving while still getting the benefits of these distro build flags? I hadn't considered that possibility, because I can't imagine anyone wanting to build packages that way instead of using dpkg-buildpackage, which does it all in a single command. So I really don't consider that an important use case, weighed against the concerns I outlined. For example, you possibly get something different depending on whether you call debian/rules, dpkg-buildpackage, debuild, or pbuilder. And the difference is hard to explain or analyze. Er, both debuild and pbuilder invoke dpkg-buildpackage. So it seems clear to me that the only difference would be when calling debian/rules directly, and at that point you're opting out of lots of other conveniences, not just distro build policy. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Steve Langasek vor...@debian.org writes: On Sun, May 10, 2009 at 11:37:46PM +0300, Peter Eisentraut wrote: On Sunday 10 May 2009 13:56:04 Steve Langasek wrote: I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. Guillem pointed out one problem: Either you do it via a make include (which you have issues with), or you stop supporting calling debian/rules directly (inconvenient, probably prone to break things) I don't agree that dpkg-buildpackage sets additional environment variables to implement a distro/site policy for builds implies calling debian/rules directly is unsupported. Or maybe I've misunderstood, and there are Debian developers who are building official packages for *upload* by calling debian/rules by hand, and that's what people are concerned about preserving while still getting the benefits of these distro build flags? I hadn't considered that possibility, because I can't imagine anyone wanting to build packages that way instead of using dpkg-buildpackage, which does it all in a single command. So I really don't consider that an important use case, weighed against the concerns I outlined. And yet people do. Also don't forget that many people call debuild, get an error, edit some file, run debian/rules foo to see if it got fixed. Now suddenly that quick check if it got fixed behaves differently. For example, you possibly get something different depending on whether you call debian/rules, dpkg-buildpackage, debuild, or pbuilder. And the difference is hard to explain or analyze. Er, both debuild and pbuilder invoke dpkg-buildpackage. So it seems clear to me that the only difference would be when calling debian/rules directly, and at that point you're opting out of lots of other conveniences, not just distro build policy. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Bill Allombert bill.allomb...@math.u-bordeaux1.fr writes: On Sun, May 10, 2009 at 09:54:11PM +0200, Raphael Hertzog wrote: On Sun, 10 May 2009, Steve Langasek wrote: I'm really surprised to see this approach getting traction. To me, this seems like a significant, unprecedented departure from the kinds of interfaces we've mandated in Policy in the past (i.e., environment variables, executables and command-line options). While one build helper or another may mandate Makefile includes, there's never been anything of the sort in Policy, and I don't think it's good to add such a thing now. I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. I have sympathy for Steve viewpoint however it lacks an alternative proposal. However I cannot only strongly disagree with Raphael argument below: Another negative aspect of the include approach is the lack of tracability. Right now when the user overrides a build flag it appears in the build log since dpkg-buildpackage prints it out in the log: dpkg-buildpackage: use CFLAGS from environment: -O3 -Wall With the include approach, we lack this feature and bad/broken local overrides can't be detected if we only have the build log at hand. There is nothing that prevents a future dpkg-buildpackage to cat to stdout the relevant files at startup so that they appear in the buildlog. Cheers, $(info CFLAGS = $(CFLAGS)) in the makefile sniplet. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Holger Levsen hol...@layer-acht.org writes: Hi, On Sonntag, 10. Mai 2009, Raphael Hertzog wrote: With the include approach, we lack this feature and bad/broken local overrides can't be detected if we only have the build log at hand. which reminds me that we dont have build logs for probably a lot more than 25% (*) of the most used packages in the archive. The ones build manually by the developer... I would really like to see that binaries for all archs are (re-)build on builds and that those logs are kept archived and linked from the PTS. (And arch all too of course). regards, Holger (*) wild guess, asuming that most packages are upload on amd64/i386/powerpc and mostly used on i386. Debuild already creates a build log. I think it would be nice to include that file in the changes file and have DAK forward it to buildd.debian.org for archival. git-buildpackage, svn-buildpackage, ... or even dpkg-buildpackage could do this too. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 11, 2009 at 02:02:47AM +0200, Goswin von Brederlow wrote: Debuild already creates a build log. I think it would be nice to include that file in the changes file and have DAK forward it to buildd.debian.org for archival. git-buildpackage, svn-buildpackage, ... or even dpkg-buildpackage could do this too. Full ACK. So long as the build logs don't get held up by dak for processing with the rest of the uploaded files and just get forwarded to buildd.debian.org, this could make the buildd code a bit simpler. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 11, 2009 at 8:02 AM, Goswin von Brederlow goswin-...@web.de wrote: Debuild already creates a build log. I think it would be nice to include that file in the changes file and have DAK forward it to buildd.debian.org for archival. git-buildpackage, svn-buildpackage, ... or even dpkg-buildpackage could do this too. It would be much nicer to discard maintainer-built packages and build everything on the buildds. Then we get build logs as well as the opportunity to replicate Ubuntu's automatically created debug debs. -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, May 10 2009, Steve Langasek wrote: On Mon, May 04, 2009 at 07:35:18AM +0200, Guillem Jover wrote: * We can set the architecture and default flags (from policy) on the makefile to be included, and packagers will be able to do the change and fix any possible problems (progressive opt-in), but once it's included by all packages, then we can do system-wide default changes in the same we change toolchains (mass rebuild, bug filing, change when bug count goes down). The makefile has the advantage that the distro default can be temporarily changed for the mass build w/o needing to totally override the build flags. I'm really surprised to see this approach getting traction. To me, this seems like a significant, unprecedented departure from the kinds of interfaces we've mandated in Policy in the past (i.e., environment variables, executables and command-line options). While one build I am happy to see this gain traction, really. And We have not done so in the past is a very illogical criteria to judge the technical merits of a proposal, I hope no one puts much weight on it. helper or another may mandate Makefile includes, there's never been anything of the sort in Policy, and I don't think it's good to add such a thing now. I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. I find it a fine and flexible idea to implement configuration files using include mechanisms, including Perl load directives. Gives the end user full control (and yes, with full control comes great responsibility, Peter Parker would say) If there's any intention at all that Policy eventually mandate use of these Makefile includes, then at a minimum I think Policy needs to *very* tightly constrain what dpkg is allowed to put in those files, to avoid future incompatibilities. I think what dpkg sets in the env variables is a fine start. And policy should not be a stick to hit the dpkg developers on the head with. I say we let an implementation blossom in the wild, see what works, and then codif that in policy. Policy is not the place to do design in. But unfortunately, if we're going to support site files, we're in no position to enforce such requirements there; so packages are still subject to breakage from admins populating their site file with random settings (or syntax errors?). Right. The admins can break package compilation, and then they get to keep all the parts. I see no harm in this. We should get away from the Cathedral mode of looking at our end users as morons who need to have their hands held, and start trusting them to know what they do. If a site want s to do things that Debian Power Up On High consider to be The Wrong Thing, I think we should give the sites enough rope. manoj -- Living your life is a task so difficult, it has never been attempted before. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, May 10 2009, Raphael Hertzog wrote: On Sun, 10 May 2009, Steve Langasek wrote: I'm really surprised to see this approach getting traction. To me, this seems like a significant, unprecedented departure from the kinds of interfaces we've mandated in Policy in the past (i.e., environment variables, executables and command-line options). While one build helper or another may mandate Makefile includes, there's never been anything of the sort in Policy, and I don't think it's good to add such a thing now. I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. BTW, the goal of policy is to enshrine the right thing, not to enshrine tradition, or The Way Things Have Always Been Done. In this case, the proposal seems to meet the goals: i) Provide a way for project wide build variables and flags to be implemented ii) Allow a set to override these values, augment them, or set new ones. iii) Allow a package to override or augment the site values. iv) Allow a use to override or augment any of the above. How a proposal meets the goals should be immaterial, as long as it does so correctly. The fact that we need caution in setting project wide values for these things is a given, and a separate issue. Another negative aspect of the include approach is the lack of tracability. Right now when the user overrides a build flag it appears in the build log since dpkg-buildpackage prints it out in the log: dpkg-buildpackage: use CFLAGS from environment: -O3 -Wall With the include approach, we lack this feature and bad/broken local overrides can't be detected if we only have the build log at hand. Sure they can. Have the site file bundles in with the report. Broken setups can be easily read there, manoj -- Bernard Shaw is an excellent man; he has not an enemy in the world, and none of his friends like him either. -- Oscar Wilde Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, May 10 2009, Steve Langasek wrote: On Sun, May 10, 2009 at 11:37:46PM +0300, Peter Eisentraut wrote: On Sunday 10 May 2009 13:56:04 Steve Langasek wrote: I thought it was generally recognized that it's a Bad Idea to implement config files using your interpreter's 'include' functionality, but that's basically what we have here. Guillem pointed out one problem: Either you do it via a make include (which you have issues with), or you stop supporting calling debian/rules directly (inconvenient, probably prone to break things) I don't agree that dpkg-buildpackage sets additional environment variables to implement a distro/site policy for builds implies calling debian/rules directly is unsupported. Or maybe I've misunderstood, and there are Debian developers who are building official packages for *upload* by calling debian/rules by hand, and that's what people are concerned about preserving while still getting the benefits of these distro build flags? The only builds Debian supports are not just the buildd ones. As members of the free software community, we should also cater to end users building, tweaking, and rebuilding our software. If the behaviour of the package is substantively different when we use or not use dpkg-buildpackage, it would be a clear violation of the principle of least surprise. I hadn't considered that possibility, because I can't imagine anyone wanting to build packages that way instead of using dpkg-buildpackage, which does it all in a single command. So I really don't consider that an important use case, weighed against the concerns I outlined. If you take the self centered approach that Debian cares only about the binary packages, and does not cater to end users also building packages, sure, it is not an important use case. I would prefer Debian to remain a full fledged member of the free software community, and continue to not let build behaviour diverge whether or not dpkg-buildpackage was used -- which can be a substancial resource hog for multiple binary source packages. manoj -- When you were born, a big chance was taken for you. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 04 2009, Peter Eisentraut wrote: On Monday 04 May 2009 23:53:15 Manoj Srivastava wrote: On Mon, May 04 2009, Peter Eisentraut wrote: Please be sure to use FOO = bar instead of :=, unless you have determined that you really wanted :=. In most cases it won't make a difference, but in some it does, and then it would behave contrary to how make usually treats variables. Why, in your opinion, would we want _not_ to use :=? What does delayed evaluation buy us? That is up for debate, to some degree, I guess. I just want to make sure that a conscious decision is made either way. (In my experience, many uses of := are made without knowledge about what it does.) That is surprising (and somewhat disheartening), it is not a complex concept. Now double-colon rules, that's something else. I think delayed evaluation is sort of the default way in which make treats variables, and so to avoid surprises and confusion, we should go with that one unless there is a specific reason otherwise. Err, no. If you use =, there is one behaviour, if you use :=, there is another. I would not call either one default, apart from the fact that non-gnu makes do not have :=. In practical terms, using delayed evaluation makes the outcome mostly independent of the order of the assignments. Which might become quite relevant when you design an include-a-bit-here, override-a-bit-there scheme that is supposed to be robust against all the nonsense that 1 source packages might do with it afterwards. We do *NOT* want settings from the distribution, from the site, the package, and the user happen in a manner mostly independent of the order -- making it harder for the user to see wtf is going on. Indeed, this is one case where we *must* have as little ambiguity as possible, where we are trying to provide _default_ values in a hierarchical fashion. Simply expanded variables generally make complicated makefile programming more predictable because they work like variables in most programming languages. They allow you to redefine a variable using its own value (or its value processed in some way by one of the expansion functions) and to use the expansion functions much more efficiently. Also note that someone who wants to be careful not to overwrite values supplied elsewhere might use ?=, which creates a delayed expansion type variable. Have you been reading at all? Has it not been stated repeatedly that we _want_ a specific order of overrides for the variable values? We _want_ the site set values to override the distribution defaults -- and then we want the package to override that, and the command line trumps all. There is no careful not to override meme here -- we want emphatically to overwrite what the outer layer has set. All the dilly dallying with delayed evaluation just confuses the issue, and makes make work harder, for no other reason than a vague don't use := cause I don't remember what it does fear. In any case, we should be careful to define and document it one way or the other. Otherwise stuff like CPPFLAGS += -DFOO=$(BAR) has unclear behavior, depending on how or whether CPPFLAGS was previously set up. (And note that it will change if initially you don't define CPPFLAGS at all and in a later version make a := definition for it -- delayed variables being the default.) There is nothing unclear about it. You are appending to the value of the preprocessor flags in that line -- at the time of use (that is, when the flags are used or evaluated). However, whenever you use a :=, the variable is cleared, and set to the value on the right hand side. So the := setting does what it is supposed to: ignore anything we knew about the variable before that point in the Makefile. If you do not know what := does, I suppose that can be confusing, and/or terrifying. You know, I get it that you have trouble understanding what = and := do in make. But this knee jerk don't use := is not helping. There is nothing mystical about := manoj -- The penalty for laughing in a courtroom is six months in jail; if it were not for this penalty, the jury would never hear the evidence. -- H. L. Mencken Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 04 2009, Guillem Jover wrote: I'll try to summarize the discussion (althought it might be biased to my preferred option) and some facts, so that we can get to a conclusion: +1 I agree with almost everything said in this email. manoj -- What use is your matted hair, you fool? What use is your antelope skin? You are tangled inside, and you are just making the outside pretty. 394 Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 2009-05-04 at 07:35 +0200, Guillem Jover wrote: On Fri, 2009-03-13 at 21:04:30 +0100, Raphael Hertzog wrote: we have an unfortunate situation where the practice in dpkg-buildpackage and the policy do not match fully. ... So I think for next dpkg upload we should make dpkg-buildpackage stop setting any flags by default, and switch the setting to go through the command line arguments to override the package options instead of the environment, so this would become the user override part. The DEB_VENDOR env var should not be set either, and we should work on getting a dpkg-vendor instead, before people try to start using the variable. Then if we get consensus that this is the righ path, agree on the makefile names (as once decided it will be a pain to change later on in all packages), we'll need to ship the distro defaults file somewhere and start fixing packages to include that makefile. The files could look something like: ,-- /usr/share/dpkg/build-options.mk # distro defaults FOO := distro -include /etc/dpkg/build-options.mk `-- ,-- /etc/dpkg/build-options.mk # site overrides #FOO := site `-- ,-- debian/rules -include /usr/share/dpkg/build-options.mk # package overrides FOO := pkg `-- ,-- command line # user overrides $ make -f debian/rules FOO=cmd That was really a very well-spent two months! +1 from me :-) Regards, Andrew. andrew (AT) morphoss (DOT) com+64(272)DEBIAN You will be advanced socially, without any special effort on your part. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Guillem Jover wrote: [ BCCed debian-dpkg for the proposed dpkg changes. ] * Lots of packages (roughly around 4k) do not honour env vars for build flags, as they follow the examples from policy (or dh-make and similar) and thus are setting them unconditionally, which env vars do not override. Not to forget these packages where flags in the environment made the package FTBFS as they were overriding flags set by makefiles or other build tools by some default values which did not include all necessary options. * We can set the architecture and default flags (from policy) on the makefile to be included, and packagers will be able to do the change and fix any possible problems (progressive opt-in), but once it's included by all packages, then we can do system-wide default changes in the same we change toolchains (mass rebuild, bug filing, change when bug count goes down). The makefile has the advantage that the distro default can be temporarily changed for the mass build w/o needing to totally override the build flags. You need to take care here to use names which will not clash with variable names upstream may be using, otherwise you'd add new problems while adding new vars. So I think for next dpkg upload we should make dpkg-buildpackage stop setting any flags by default, and switch the setting to go through the Please do so! +1 from me, too. Thanks for the good work! Bernd -- Bernd Zeimetz Debian GNU/Linux Developer GPG Fingerprint: 06C8 C9A2 EAAD E37E 5B2C BE93 067A AD04 C93B FF79 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 04 May 2009, Guillem Jover wrote: env var should not be set either, and we should work on getting a dpkg-vendor instead, before people try to start using the variable. FYI, I already started this. start fixing packages to include that makefile. The files could look something like: ,-- /usr/share/dpkg/build-options.mk # distro defaults FOO := distro -include /etc/dpkg/build-options.mk `-- Note that recent changes have aimed at avoiding fork of dpkg by integrating upstream some distro-specific logic. We should do the same for this file then... ie accept that external vendors provide us their defaults and have the dpkg source package pick the right one based on the current vendor. ,-- debian/rules -include /usr/share/dpkg/build-options.mk # package overrides FOO := pkg `-- Probably without the leading -, otherwise we have a risk to not have any sane default at all. And we should add a build-dependency instead. Or maybe that choice can be left to the maintainer. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 04 2009, Raphael Hertzog wrote: On Mon, 04 May 2009, Guillem Jover wrote: ,-- debian/rules -include /usr/share/dpkg/build-options.mk # package overrides FOO := pkg `-- Probably without the leading -, otherwise we have a risk to not have any sane default at all. I disagree. Without the leading -, it would make it impossible to build on a machine with older dpkg -- and why be unnecessarily unfriendly to backporters? If you want sane defaults, by all means provide sane defaults yourself. And we should add a build-dependency instead. And restrict the machines where the package can be built? I think not. Or maybe that choice can be left to the maintainer. This is better than forcing people into restricting the platforms where their sources can be built, but still a cop out. Guillem's proposal is technically better. manoj -- You k'n hide de fier, but w'at you gwine do wid de smoke? Joel Chandler Harris, proverbs of Uncle Remus Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
mandate build-arch (was Re: Environment variables, debian/rules and dpkg-buildpackage)
If we're doing something that ultimately requires modification of every debian/rules file *anyway*, can we please make build-arch/build-indep mandatory targets, so that dpkg-buildpackage -B can *finally* (eight years later!) be changed to call build-arch? zw not subscribed to d-devel; please cc: -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Monday 04 May 2009 08:35:18 Guillem Jover wrote: I like this proposal. A small nit: ,-- /usr/share/dpkg/build-options.mk # distro defaults FOO := distro Please be sure to use FOO = bar instead of :=, unless you have determined that you really wanted :=. In most cases it won't make a difference, but in some it does, and then it would behave contrary to how make usually treats variables. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, May 04 2009, Peter Eisentraut wrote: On Monday 04 May 2009 08:35:18 Guillem Jover wrote: I like this proposal. A small nit: ,-- /usr/share/dpkg/build-options.mk # distro defaults FOO := distro Please be sure to use FOO = bar instead of :=, unless you have determined that you really wanted :=. In most cases it won't make a difference, but in some it does, and then it would behave contrary to how make usually treats variables. Why, in your opinion, would we want _not_ to use :=? What does delayed evaluation buy us? manoj puzzled -- When you live in a sick society, just about everything you do is wrong. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Monday 04 May 2009 23:53:15 Manoj Srivastava wrote: On Mon, May 04 2009, Peter Eisentraut wrote: Please be sure to use FOO = bar instead of :=, unless you have determined that you really wanted :=. In most cases it won't make a difference, but in some it does, and then it would behave contrary to how make usually treats variables. Why, in your opinion, would we want _not_ to use :=? What does delayed evaluation buy us? That is up for debate, to some degree, I guess. I just want to make sure that a conscious decision is made either way. (In my experience, many uses of := are made without knowledge about what it does.) I think delayed evaluation is sort of the default way in which make treats variables, and so to avoid surprises and confusion, we should go with that one unless there is a specific reason otherwise. In practical terms, using delayed evaluation makes the outcome mostly independent of the order of the assignments. Which might become quite relevant when you design an include-a-bit-here, override-a-bit-there scheme that is supposed to be robust against all the nonsense that 1 source packages might do with it afterwards. Also note that someone who wants to be careful not to overwrite values supplied elsewhere might use ?=, which creates a delayed expansion type variable. In any case, we should be careful to define and document it one way or the other. Otherwise stuff like CPPFLAGS += -DFOO=$(BAR) has unclear behavior, depending on how or whether CPPFLAGS was previously set up. (And note that it will change if initially you don't define CPPFLAGS at all and in a later version make a := definition for it -- delayed variables being the default.) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
[ BCCed debian-dpkg for the proposed dpkg changes. ] Hi! On Fri, 2009-03-13 at 21:04:30 +0100, Raphael Hertzog wrote: we have an unfortunate situation where the practice in dpkg-buildpackage and the policy do not match fully. Ok, had finally the time to read and think about this. I've to say first of all that I've never quite liked dpkg-buildpackage as a way to set distro-wide flags, and less so by doing it by default and kind of deprecating debian/rules in the way. But at the time this got implemented I didn't have the time to ponder about better alternatives or discuss about it (although the makefile include crossed my mind at some point, and we talked about it with Raphaël before he started this thread). In retrospect I guess I should have chimed in on the initial discussion or the subsequent complaints, but nothing that cannot be repaired now anyway. I'll try to summarize the discussion (althought it might be biased to my preferred option) and some facts, so that we can get to a conclusion: * We'd like to have at least this overriding hierarchy (the implementation could be a bit more complex, but I think that's not important now, as this one already exposes problems in the dpkg-buildpackage proposal): - Distro defaults. - Site overrides over the above (can be partial filters). - Package overrides over the above (can be partial filters). - User overrides over the above (total overrides). * dpkg-buildpackage currently sets the build flags through env vars. * Lots of packages (roughly around 4k) do not honour env vars for build flags, as they follow the examples from policy (or dh-make and similar) and thus are setting them unconditionally, which env vars do not override. ,-- chk-flags -- #!/bin/sh set -eu regex=$@ cd /srv/lintian.debian.org/laboratory/source/ ls | xargs -I% grep -H -l $regex %/debfiles/rules | wc -l `-- ./chk-flags '^[^#]*\(CXX\|F\|CPP\|LD\|C\)FLAGS[[:space:]]*:\?=' * Thus lots of packages in the archive have to be modified anyway regardless of either solution (centralization through dpkg-buildpackage or makefile include). If we have to modify them, we can make them include a makefile. Adding such include line at the top of debian/rules should certainly imply less changes than the proposed changes in http://lists.debian.org/debian-devel/2009/03/msg00920.html. * dpkg-buildpackage has been setting env vars for dpkg-architecture output for a long time (2001), but those flags are a bit different than the other build flags. First the example code in dh-make uses ?= which makes the env vars take preference, and they are (or should be) always initialized in the debian/rules file as recommended by dpkg-architecture(1). And second the *_BUILD_* ones should alway match the current system, and the *_HOST_* ones should be changed by changing the toolchain, so there's no reason to override them in the distro or packaging (if there's a need then dpkg should be fixed instead). * Externalizing the setup of the build flags means that debian/rules stops being a working standalone interface for building packages, as mandated by policy, and imposes an additional layer to rely on for building packages * Setting system (distro and site) build vars through command line (or forced environment 'make -e') does not allow the pkg to override them, nor do partial filtering. * Using 'make -e' is not really a good idea as it's not fine grained. So the correct solution when total override is desired is to set the vars via the command line. * Setting system (distro and site) build vars through command line, implies we can only use limited make syntax for those vars. * We can set the architecture and default flags (from policy) on the makefile to be included, and packagers will be able to do the change and fix any possible problems (progressive opt-in), but once it's included by all packages, then we can do system-wide default changes in the same we change toolchains (mass rebuild, bug filing, change when bug count goes down). The makefile has the advantage that the distro default can be temporarily changed for the mass build w/o needing to totally override the build flags. So I think for next dpkg upload we should make dpkg-buildpackage stop setting any flags by default, and switch the setting to go through the command line arguments to override the package options instead of the environment, so this would become the user override part. The DEB_VENDOR env var should not be set either, and we should work on getting a dpkg-vendor instead, before people try to start using the variable. Then if we get consensus that this is the righ path, agree on the makefile names (as once decided it will be a pain to change later on in all packages), we'll need to ship the distro defaults file somewhere and start fixing packages to include that makefile. The files could look something like: ,-- /usr/share/dpkg/build-options.mk #
Re: Environment variables, debian/rules and dpkg-buildpackage
On Tue, 17 Mar 2009, Manoj Srivastava wrote: On Tue, Mar 17 2009, Stefano Zacchiroli wrote: It seems to me that you are indeed close, but with the exception of this required include in all our debian/rules, which will be a PITA to achieve. Modulo debhelper, cdbs, et.al can help. Debhelper can't help. Only CDBS can (as its design is precisely based on included Makefiles). And will be mostly useless. dpkg can set these variables until it is blue in the face, and it has zero effect on my packages until I change my rules file. Or change upstream Makefiles to pay attention to the cflags set in the env. This is a major change for some of my packages, and without them the dpkg proposal does nothing. That's true for a lot of packages but not all packages. It also means that there is some usage of this approach in the archive (otherwise if all packages ignored the environment variables, there would never have been packages that FTBFS when this change was introduced in Lenny). So according to your rule that policy should standardize common practice and not mandate something completely new, the env variable proposal is in more widespread usage. (Note that I don't find this rule particularly useful and that's why I didn't mention it before but since you ask about points where you are biased I thought it could be worth telling) See, I think we need have package opt-in in any case. Not in all cases. CDBS could also be updated to cope with the env var and debhelper 7 packages using rules.tiny could also make use of it without a modification. Also, the dpkg proposal blurs the distinction between the site wide policy and project defaults, as far as the user or the package maintainer is concerned. This is a deficiency. While I agree that the distinction is blurred because in the rules files you don't know where the env var comes from, I don't agree that it's a deficiency. The rules files receives a value and should use it. It doesn't need to know whether it's the site-wide default or the distribution default. Yup. The major differences are: With the dpkg proposal: Bad: o) people must always use dpkg to build packages, or the packages come out different If the policy document the fact that the rules files need some input variables, then it's not a big deal: if you really care about being able to call debian/rules directly you can always adapt your rules files accordingly like I suggested at the end of this mail: http://lists.debian.org/debian-devel/2009/03/msg00920.html We could even write such a Makefile that mimics more closely what dpkg-buildpackage does for people that care about it. But I don't really want to impose a Makefile snippet to everybody. o) The user or the package maintainer can no longer tell the difference between site policy and project policy The user is intelligent, he can read the output of dpkg-buildpackage that tells him where the data comes from (or he can read the doc and the configuration file). In any case, it's comparable to having to read a Makefile snippet. For the package maintainer, why should he care ? The variable is provided in a manner where he can or can't override it, but that's all that matters. o) The environment variables are set even if the package is not ready for them, We already took that hit. It's not an issue. o) rules file still need to be modified to take advantage of the variables set -- none of my rules files will be affected by any settings in dpkg right now. The fact that your rules files need change doesn't meant that all need it. With the Make snippets proposal: Bad: o) Every package has to change (but,every package has to change anyway, since currently dpkg can set the variables till it is blue in the face and nothing will take effect in my packages) Good: o) the person building the package is not constrained to using dpkg; o) The site admin can add on to the variables set on that site, and is not constrained by what dpkg handles o) the process is opt in, and packages opt in to using the new mechanism when they are ready. o) The end user can override project, site, or package policies, individually, or in any combination If I am biased (likely), please tel me where I have gone wrong above. If the policy documents the environment that needs to be setup, nobody is forced to use dpkg-buildpackage to do it. But it's probably more convenient to use a helper compared to manually setting the expected variable. dpkg-buildpackages should be considered like a helper and I'm available to fix any problem that makes it painful to use (if you find its usage too much constraining). I would like to point out that we speak mainly of CFLAGS and al. but that I also included DEB_VENDOR in the set of variables. This variable is not used currently (as I just introduced it with dpkg 1.15.0) but I expect it
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: On Tue, 17 Mar 2009, Manoj Srivastava wrote: On Tue, Mar 17 2009, Stefano Zacchiroli wrote: It seems to me that you are indeed close, but with the exception of this required include in all our debian/rules, which will be a PITA to achieve. Modulo debhelper, cdbs, et.al can help. Debhelper can't help. Only CDBS can (as its design is precisely based on included Makefiles). With the magic %: dh $@ debhelper could help. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Wed, Mar 18 2009, Raphael Hertzog wrote: On Tue, 17 Mar 2009, Manoj Srivastava wrote: On Tue, Mar 17 2009, Stefano Zacchiroli wrote: It seems to me that you are indeed close, but with the exception of this required include in all our debian/rules, which will be a PITA to achieve. Modulo debhelper, cdbs, et.al can help. Debhelper can't help. Only CDBS can (as its design is precisely based on included Makefiles). And will be mostly useless. dpkg can set these variables until it is blue in the face, and it has zero effect on my packages until I change my rules file. Or change upstream Makefiles to pay attention to the cflags set in the env. This is a major change for some of my packages, and without them the dpkg proposal does nothing. That's true for a lot of packages but not all packages. It also means that there is some usage of this approach in the archive (otherwise if all packages ignored the environment variables, there would never have been packages that FTBFS when this change was introduced in Lenny). Any package that follows policy §4.9 examples to set CFLAGS based on DEB_BUILD_OPTIONS will be immune to CFLAGS et al set in the environment. Also, any upstream Makefile that sets CFLAGS (don't most ones that use automake do that?) will also be not affected, unless even more hackery is done. At this point, what dpkg does to these variables not enough to justify any such effort. So according to your rule that policy should standardize common practice and not mandate something completely new, the env variable proposal is in more widespread usage. That was not my rule, just a misunderstanding of what I said. The rules is that policy a) standardizes practices needed for integration, and no more. b) when it must add stuff, policy tries to add the best, and minimal, solution. c) Often the viability of s solution is not clear until it has been put into practice, and the kinks ironed out. This does not mean shoddy practices will be enshrined in policy. (Note that I don't find this rule particularly useful and that's why I didn't mention it before but since you ask about points where you are biased I thought it could be worth telling) An incorrect rendering of the rule does not help, sorry. Also, the dpkg proposal blurs the distinction between the site wide policy and project defaults, as far as the user or the package maintainer is concerned. This is a deficiency. While I agree that the distinction is blurred because in the rules files you don't know where the env var comes from, I don't agree that it's a deficiency. A missing feature is a deficiency. How critical a deficiency it is is a matter of opinion, and we apparently differ. The rules files receives a value and should use it. It doesn't need to know whether it's the site-wide default or the distribution default. Yup. The major differences are: With the dpkg proposal: Bad: o) people must always use dpkg to build packages, or the packages come out different If the policy document the fact that the rules files need some input variables, then it's not a big deal: if you really care about being able to call debian/rules directly you can always adapt your rules files accordingly like I suggested at the end of this mail: http://lists.debian.org/debian-devel/2009/03/msg00920.html We could even write such a Makefile that mimics more closely what dpkg-buildpackage does for people that care about it. But I don't really want to impose a Makefile snippet to everybody. Frankly, dpkg does very little: All it does is to discard the policy suggestion (which I think is a bad idea) and only sets FLAGS to -g -O0 or -g -O2 I do not think that this is much of an assistance, if this is as far as dpkg goes. Setting CPPFLAGS, COPTS, CWARNS, CFLAGS, CMACHINE (and things for other languages) might be better, but trivially just setting CFLAGS to select between -O0 and -O2 is not enough to justify mangling build systems to o) The user or the package maintainer can no longer tell the difference between site policy and project policy The user is intelligent, he can read the output of dpkg-buildpackage that tells him where the data comes from (or he can read the doc and the configuration file). In any case, it's comparable to having to read a Makefile snippet. With the makefile snippet, the Makefile can tell the difference, and there is a standard method to tell what variables are set in the site config. For the package maintainer, why should he care ? The variable is provided in a manner where he can or can't override it, but that's all that matters. Because it gives the option to override one or the other -- a generic default that is project wide might be less applicable, but site policies can still be followed.
Re: Environment variables, debian/rules and dpkg-buildpackage
On Wed, Mar 18, 2009 at 09:53:46AM +0100, Raphael Hertzog wrote: So according to your rule that policy should standardize common practice and not mandate something completely new, the env variable proposal is in more widespread usage. For ten years, the common practice was that dpkg-buildpackage did not set any variable. We cannot standardize on the env variable proposal because such proposal has never be made. Instead dpkg-buildpackage was broken in Lenny, and should be fixed ASAP. Now we have packages that do not build correctly with dpkg-buildpackage, others that do not build correctly with debian/rules binary, and all handle env var differently. Cheers, -- Bill. ballo...@debian.org Imagine a large red swirl here. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Wed, Mar 18, 2009 at 06:23:55PM +0100, Bill Allombert wrote: For ten years, the common practice was that dpkg-buildpackage did not set any variable. We cannot standardize on the env variable proposal because such proposal has never be made. Instead dpkg-buildpackage was broken in Lenny, and should be fixed ASAP. Now we have packages that do not build correctly with dpkg-buildpackage, others that do not build correctly with debian/rules binary, and all handle env var differently. I concur. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Wed, Mar 18, 2009 at 06:23:55PM +0100, Bill Allombert wrote: On Wed, Mar 18, 2009 at 09:53:46AM +0100, Raphael Hertzog wrote: So according to your rule that policy should standardize common practice and not mandate something completely new, the env variable proposal is in more widespread usage. For ten years, the common practice was that dpkg-buildpackage did not set any variable. We cannot standardize on the env variable proposal because such proposal has never be made. Instead dpkg-buildpackage was broken in Lenny, and should be fixed ASAP. Now we have packages that do not build correctly with dpkg-buildpackage, others that do not build correctly with debian/rules binary, and all handle env var differently. Exactly. dpkg-buildpackage setting environment variables is broken. Using make to set site/distro/user-specific options makes more sense. Not only is it more flexible and extensible, it will also work no matter how one builds the package since make will handle it. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Hello, On Wed, 18 Mar 2009, Bill Allombert wrote: On Wed, Mar 18, 2009 at 09:53:46AM +0100, Raphael Hertzog wrote: So according to your rule that policy should standardize common practice and not mandate something completely new, the env variable proposal is in more widespread usage. For ten years, the common practice was that dpkg-buildpackage did not set any variable. It does set the dpkg-architecture related variables since 2001, so that's not true. We cannot standardize on the env variable proposal because such proposal has never be made. Instead dpkg-buildpackage was broken in Lenny, and should be fixed ASAP. I can understand you were not happy with the way the change was done but saying dpkg-bp is broken is strong (and wrong). If you really believed that a major mistake was done at that time, you could have complained louder and you could have asked for a tech-ctte ruling. We also offered to retract the change to the release team but they did not deem it necessary. Now we have packages that do not build correctly with dpkg-buildpackage, others that do not build correctly with debian/rules binary, and all handle env var differently. Hence why I started this discussion, I'm not happy with the situation either. I also know that several maintainers object to adding a Makefile snippet so if we ever want to have a chance to standardize something in policy, we need to find a reasonable compromise like the one I tried to propose in http://lists.debian.org/debian-devel/2009/03/msg00920.html While I prefer the env var approach, I'm not opposed to the Makefile snippet approach, but I know that others are and I really want that we find some path out of this situation because the objectives behind all this are desirable. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Manoj Srivastava sriva...@debian.org writes: Also, any upstream Makefile that sets CFLAGS (don't most ones that use automake do that?) will also be not affected, unless even more hackery is done. At this point, what dpkg does to these variables not enough to justify any such effort. Most packages that use Automake also use Autoconf, and Autoconf picks up CFLAGS from the environment at the time configure is run, so most Automake packages would pick up an environmental default. However, I expect many packages are shielded at present by use of the default Policy-recommended explicit setting of CFLAGS. I know nearly all of mine (that care about CFLAGS at all) are. The main exception are Perl modules, for which I've mostly switched to debhelper v7 rule minimization. When this change was originally made in dpkg-buildpackage, it broke several packages that didn't have this shield. I'm not sure if there was a common way in which those packages are fixed or how many of them went back to setting CFLAGS explicitly. On Wed, Mar 18 2009, Raphael Hertzog wrote: While I agree that the distinction is blurred because in the rules files you don't know where the env var comes from, I don't agree that it's a deficiency. A missing feature is a deficiency. How critical a deficiency it is is a matter of opinion, and we apparently differ. And for what it's worth, I agree with Manoj here. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: I can understand you were not happy with the way the change was done but saying dpkg-bp is broken is strong (and wrong). If you really believed that a major mistake was done at that time, you could have complained louder and you could have asked for a tech-ctte ruling. We also offered to retract the change to the release team but they did not deem it necessary. Well, I tried to bring it up at the time, but it seemed clear that you felt strongly that it was the right way forward and there were only a few who objected (although I suspect that's because not that many people realized at the time all of the implications). I suppose I could have appealed it to the tech-ctte, but that seemed unnecessarily confrontational given that the actual negative effect on the archive at the moment was relatively minor. Most of my concern was over packages starting to rely on this behavior of dpkg-buildpackage, thus making it difficult to use a different solution. At the time, I didn't think of the makefile fragment idea that Manoj has proposed. If I had thought of that, I would have pushed that, since I think it addresses the core goal in a much cleaner fashion. At the time, I had no good alternative solution. I thought at the time about making a bigger deal about it, but I didn't really want to just start a flamewar and it seemed like a poor way to thank you for all of the other work that you were doing on dpkg-buildpackage. At the time, I think it was more important to worry about finishing the lenny work. I'm very glad that you've re-raised the discussion, and I really appreciate you doing so. Hopefully we can find a solution that makes everyone happy for squeeze. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Manoj Srivastava schrieb: On Mon, Mar 16 2009, Raphael Hertzog wrote: On Sun, 15 Mar 2009, Bill Allombert wrote: There is no documented semantic for CFLAGS et. al. in Debian policy. While some Makefile handle it in a certain way, this is not mandatory in any way. For example some configure scripts append options to CFLAGS while other will not change it if it is defined. This is precisely what I want to change. We should provide a reasonable way for the user/admin to give default flags and for the packager to override/extend the default set of flags. So here are what I think we need, based on the most common use cases: A) Provide a way to specify project wide defaults for env variables B) Allow a site admin to selectively override these, and provide site wide defaults C) Allow a package to set some flags that would otherwise break the package, overriding site and/or project defaults as needed D) Allow the user to specify and override any of the above. E) Provide a way to specify project wide defaults and override any of the above. This is much needed to for rebuild checks ignoring package specific work arounds. Currently workarounds tend to live forever, and there is no way to automatically override those. I do not see any reason to limit ourselves to one tool to set the env variables, especially since it prevents the package maintainer from distinguishing between the project wide defaults anduser set values, and has no easy way for a site to set site wide values. Hey, I might like hardening flags the project does not cater to when I rebuild the archive on my buildd -- and the snippet approach allows for it. maybe, but for E) a package should honour these overwrite flags. Matthias -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Matthias Klose d...@debian.org writes: Manoj Srivastava schrieb: A) Provide a way to specify project wide defaults for env variables B) Allow a site admin to selectively override these, and provide site wide defaults C) Allow a package to set some flags that would otherwise break the package, overriding site and/or project defaults as needed D) Allow the user to specify and override any of the above. E) Provide a way to specify project wide defaults and override any of the above. This is much needed to for rebuild checks ignoring package specific work arounds. Currently workarounds tend to live forever, and there is no way to automatically override those. Wouldn't the user override work for you in this case? If you're doing archive-wide test builds overriding any package settings, that seems to me to be exactly the same case as a user overriding all package settings to force a particular set of build flags and you could do this with the same mechanism. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Raphael Geissert wrote: Stefano Zacchiroli wrote: ... and pretty please, do not choose a solution that will require adding an -include to 15'000 thousands debian/rules; we will finish doing that by Lenny+50, the earliest. It would take some time, yes; but packages using cdbs would only require a binNMU once cdbs includes that file on its own. BinNMU are not needed. The default CFLAGS won't have changed. A rebuild to verify that none are FTBFS would be enough. IMO this is what looks like The Right Solution; are we willing to ignore it just because it would take some time? I don't know your criteria for The Right Solution, so this opinion doesn't buy me much. What other approach (which fulfils all the pros already mentioned by Manoj) do you suggest? What are the pros mentioned by Manoj that are specific to the Makefile snippet approach except the fact that we can continue to call debian/rules directly on all packages ? At this point, I have the feeling it's a matter of individual taste mainly. Every time we have accepted that we have ended up with 2 different approachs (python-support vs python-central) and we have not won much. Is there a way to reconcile both approaches ? Maybe this one: we let the caller pass the required build options as environment variables or command line options but the package maintainer has the possibility to include a Makefile snippet that would set any missing build-option to the default value chosen by the distribution so that he can continue to call debian/rules directly without having to take care of setting the environment variables. Such a Makefile snippet could then be like this: CFLAGS ?= -Wall -g ifeq $(origin CFLAGS) file ifneq (,$(filter noopt,$(DEB_BUILD_OPTIONS))) CFLAGS += -O0 else CFLAGS += -O2 endif endif How does that sound ? Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Tue, Mar 17 2009, Raphael Hertzog wrote: What are the pros mentioned by Manoj that are specific to the Makefile snippet approach except the fact that we can continue to call debian/rules directly on all packages ? That by itself is reason enough, I think. Secondly, I have never seen the dpkg approach address the site-wide configuration: How is that to be done? At this point, I have the feeling it's a matter of individual taste mainly. Every time we have accepted that we have ended up with 2 different approachs (python-support vs python-central) and we have not won much. If you address these issues, I can accept your staatement: a) The site admin can set, and optionally override, the project wide defaults b) The package can set, and optionally override the site defaults, or the global defaults, or either, or none c) The user may set defaults, ot override the global, site, or package defaults, in any order. Is there a way to reconcile both approaches ? If you thik the dpkg approach can also provide the above functionality of the Makefile snippetr approach, then yes, the only advandate of the Makefile approach is that one is not dependent on dpkg to build packages (which, in my eyes, is a critical advantage by itself). manoj -- grep me no patterns and I'll tell you no lines. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Raphael Hertzog wrote: On Mon, 16 Mar 2009, Manoj Srivastava wrote: And you are missing the point that making people type stuff on the command line for site specific stuff looses out to being able to edit a conffile instead. Who said the command line was for site-specific stuff? In my proposition the hierarchy could be something like this: - distribution default (encoded in dpkg-buildpackage or in a /etc/dpkg/origins/ file) The latter is better, since it does not force people to iuse dpkg, and does not force blended distributions and derivatives to modify dpkg every time they want a change. - site-specific (/etc/dpkg/dpkg-buildpackage.conf) - package specific (inside debian/rules) - user specific (command line options) In my opinion, the origin of the variable is not important, the only important part is to specifiy in policy how the value is communicated to the rules file. I ahd not read this changed proposal until now. This is close enough to what I was saying to be workable -- as long as the project defaults are in a file, and not hard coded into a script. The only other caveat I would put in is implementation details, with my Make hat on: Use the := and = variants in make to set variables that help differentiate the policies that the variables define; --8---cut here---start-8--- default_foo:= PROJECT_VALUE # set in /etc/dpkg/origins -include /etc/dpkg/default.mk # set in /etc/buildpackage.mk site_foo := SITE_VALUE # set in /etc/buildpackage.mk ifneq (,$strip($(site_foo)))# also set in /etc/buildpackage.mk foo = $(site_foo) else ifneq (,$strip($(default_foo))) foo = $(default_foo) endif endif # in debian/rules -include /etc/buildpackage.mk # Set foo here, if necessary --8---cut here---end---8--- With such a template, we can override either the default, or site, policy, either in the debian/rules, or on the command line. I think we are close. manoj -- Death is nature's way of saying `Howdy'. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Tue, Mar 17, 2009 at 11:36:07AM -0500, Manoj Srivastava wrote:
# in debian/rules
-include /etc/buildpackage.mk
It seems to me that you are indeed close, but with the exception of
this required include in all our debian/rules, which will be a PITA to
achieve. AFAIU Raphael's proposal, the site-specific configuration
will be achieved via dpkg-buildpackage config's file, without needing
to include anything explicitly in debian/rules, while you are
insisting in the explicit include.
While I understand (though I disagree) with your reasons for having
that include, that looks like a major differences in the two positions
still.
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Environment variables, debian/rules and dpkg-buildpackage
On Tue, Mar 17 2009, Stefano Zacchiroli wrote: On Tue, Mar 17, 2009 at 11:36:07AM -0500, Manoj Srivastava wrote: # in debian/rules -include /etc/buildpackage.mk It seems to me that you are indeed close, but with the exception of this required include in all our debian/rules, which will be a PITA to achieve. Modulo debhelper, cdbs, et.al can help. AFAIU Raphael's proposal, the site-specific configuration will be achieved via dpkg-buildpackage config's file, without needing to include anything explicitly in debian/rules, while you are insisting in the explicit include. And will be mostly useless. dpkg can set these variables until it is blue in the face, and it has zero effect on my packages until I change my rules file. Or change upstream Makefiles to pay attention to the cflags set in the env. This is a major change for some of my packages, and without them the dpkg proposal does nothing. See, I think we need have package opt-in in any case. Also, the dpkg proposal blurs the distinction between the site wide policy and project defaults, as far as the user or the package maintainer is concerned. This is a deficiency. While I understand (though I disagree) with your reasons for having that include, that looks like a major differences in the two positions still. Yup. The major differences are: With the dpkg proposal: Bad: o) people must always use dpkg to build packages, or the packages come out different o) The user or the package maintainer can no longer tell the difference between site policy and project policy o) The environment variables are set even if the package is not ready for them, o) rules file still need to be modified to take advantage of the variables set -- none of my rules files will be affected by any settings in dpkg right now. Good: o) There is no transition period involved -- except that rules files still need to be modified, so there is a transition period anyway. With the Make snippets proposal: Bad: o) Every package has to change (but,every package has to change anyway, since currently dpkg can set the variables till it is blue in the face and nothing will take effect in my packages) Good: o) the person building the package is not constrained to using dpkg; o) The site admin can add on to the variables set on that site, and is not constrained by what dpkg handles o) the process is opt in, and packages opt in to using the new mechanism when they are ready. o) The end user can override project, site, or package policies, individually, or in any combination If I am biased (likely), please tel me where I have gone wrong above. manoj -- Unfair competition: Selling cheaper than we do. Kelvin Throop III, The Management Dictionary Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, 15 Mar 2009, Bill Allombert wrote: There is no documented semantic for CFLAGS et. al. in Debian policy. While some Makefile handle it in a certain way, this is not mandatory in any way. For example some configure scripts append options to CFLAGS while other will not change it if it is defined. This is precisely what I want to change. We should provide a reasonable way for the user/admin to give default flags and for the packager to override/extend the default set of flags. You are conflating breakage that were reported with breakage that appeared but were not reported. […] Changing CFLAGS behind the back of the maintainers is potentially changing how the package is build in a undefined way. A package built in such way should be assumed broken. So you should assume that most Lenny packages are broken. I know the difference but while I don't deny that some unused packages might be affected by this, it's not my concern right now (those packages are probably candidate for removal if nobody used them to detect that they are not working anymore). Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, 15 Mar 2009, Stephen Gran wrote: This one time, at band camp, Raphael Hertzog said: Care to elaborate what kind of flexibility you need in this specific case ? I don't. I'm imagining that some of our downstreams may. It's precisely one of our downstreams that pushed the dpkg-buildpackage change (Ubuntu). I don't think most downstreams care a lot about the exact implementation, but they clearly want a way to alter defauld build flags at the distribution level. This all started with: https://wiki.ubuntu.com/DistCompilerFlags Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog wrote: Note that I'm not asking to mandate the tool. I would like to mandate the fact that packages can rely on some environment variables being set to some values. Note that packages will not necessarily pickup the environment variables. autoconf using packages will probably do automatically, but other build-systems still require the maintainer to pick up the changes from the environment and passing them to the build command (eg, scons doesn't have any standard way for that, its up to the package to define a variable). So you don't actually guarantee that all packages use the default flags. -- Felipe Sateler -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: On Fri, 13 Mar 2009, Manoj Srivastava wrote: 3. dpkg-buildpackage is probably the wrong place to put this solution in. Why? The fact that dpkg-buildpackage's setting the variables is not easily configurable, and presents to make as though it was set on the commandline, and thus making it hard for the *USER* to set the flag variables via env variables, is, in my opinion, a major bug. This is wrong. dpkg-buildpackage preserves the value set by the user if any. It simply sets default values to all those environment variables if they have none. I think he ment that you can not know wether the setting comes from dpkg-buildpackage or the user. If it comes from dpkg-buildpackage then debian/rules should be free to override it as needed. If it comes from the user then that is another story. At least that is my take on it. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Fri, Mar 13, 2009 at 09:04:30PM +0100, Raphael Hertzog wrote:
* either we modify policy to mandate the set of environment variables that
dpkg-buildpackage sets
snip
In terms of efforts, the first solution is the easiest. But we aim
at the _right_ solution so feel free to design something that makes
the second solution viable.
FWIW I'd love to see the first solution implemented. My main reason,
as a maintainer of packages which mostly escape the autotools / C
language pattern, is to see written somewhere the intended semantics
in Debian of the various environment variable. That would offer an
explanation to maintainers about how to handle such variables even if
their upstreams are not using standard makefiles.
Regarding the objection of mandating dpkg-buildpackage as *the* tool
to build, it looks moot to me. The main point would be defining the
API and the intended meaning of variables. Then if it will happen that
dpkg-buildpackage sets them properly, let's be happy about that or
implement alternative tools.
Same goes for the configuration file objection: dpkg-buildpackage can
have one.
... and pretty please, do not choose a solution that will require
adding an -include to 15'000 thousands debian/rules; we will finish
doing that by Lenny+50, the earliest.
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Raphael Hertzog wrote: On Sun, 15 Mar 2009, Bill Allombert wrote: There is no documented semantic for CFLAGS et. al. in Debian policy. While some Makefile handle it in a certain way, this is not mandatory in any way. For example some configure scripts append options to CFLAGS while other will not change it if it is defined. This is precisely what I want to change. We should provide a reasonable way for the user/admin to give default flags and for the packager to override/extend the default set of flags. So here are what I think we need, based on the most common use cases: A) Provide a way to specify project wide defaults for env variables B) Allow a site admin to selectively override these, and provide site wide defaults C) Allow a package to set some flags that would otherwise break the package, overriding site and/or project defaults as needed D) Allow the user to specify and override any of the above. In the non-snippet method proposed, there is no way for the package maintainer to override project defaults yet cater user set variable settings, since the information is lost. Also, I think that just CFLAGS is probably too broad, I might want to override only some of the categories of flags (optimization, machine related, warning, debugging, etc) and not others; pulling warning and optimization out as separate subflags might be a good compromise. All of this can be done with Makefile snippets, and do not allow you to mandate any set of tools, without which the environment set would be different. You can use dpkg-vuildpackage, $MAKE) -f deban/rules, write a wrapper Makefile that include ./debian/rules -- it all would just work. I do not see any reason to limit ourselves to one tool to set the env variables, especially since it prevents the package maintainer from distinguishing between the project wide defaults anduser set values, and has no easy way for a site to set site wide values. Hey, I might like hardening flags the project does not cater to when I rebuild the archive on my buildd -- and the snippet approach allows for it. manoj -- s = (char*)(long)retval; /* ouch */ Larry Wall in doio.c from the perl source code Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Raphael Hertzog wrote: On Mon, 16 Mar 2009, Bdale Garbee wrote: I think he ment that you can not know wether the setting comes from dpkg-buildpackage or the user. If it comes from dpkg-buildpackage then debian/rules should be free to override it as needed. If it comes from the user then that is another story. At least that is my take on it. This is a great point. It must be possible to craft a rules file that overrides system or distribution wide defaults and which can be over-ridden by an individual building the package. I don't see why it's needed, it's the job of the rules file to adjust the flags if there's a reason to deviate (like using -O0 instead of -O2 for some arch with compiler problems) but in general the package should not override completely the default flags but just complete them and/or adjust/fix them if needed. But while overriding the project wide default makes sense, overriding the human doing hte building does not. However, if the caller really wish that his build options prevail in all cases, he can use make -e (and dpkg-buildpackage has the -R option that let him call debian/rules as make -e -f debian/rules instead). We do not want to override *EVERYTHING* in the Makefile, just the CFLAGS. This blunt instrument of the -e flag is not a good solution. If necessary we can add a new option --force-flags to dpkg-bp or similar to make this even easier. Making a bad solution easy is not a good thing either. That seems to require the ability for the code in a rules file to distinguish between things in the environment because they're defaults and things in the environment that are attempting to override defaults. Apparently there's no way to know from where the variable value come in make. That's true for environment variables like for command line variables. (at least according to my lookup of info make) Err, I think you need to get more conversant with Make. Yes, there is no way to know whether or not the environment variables were set by dpkg or the user; but it is itrivial to make it so that the project wide defaults can be overridden by the site wide stuff, and each being overridden by the package at will. We can even make it so that the package can tell which one set the value, and override only the project wide one and not the site variable. So if we really want the rules files to be able to know, we have to add yet another requirement to create this information. This is not desirable IMO. Given how trivial it is, I think it is eminently desirable. It is just that dpkg is not the right place for such subtleties. manoj -- Though many hands make light work, too many cooks spoil the broth. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Manoj Srivastava wrote: In the non-snippet method proposed, there is no way for the package maintainer to override project defaults yet cater user set variable settings, since the information is lost. You're not trying very hard to look from both sides: whether the default value comes from the environment or from an included Makefile, in both cases the user can override it with command-line arguments. Granted it means that dpkg-buildpackage would have to pass user-overriden flags on the command line instead of using the environment, but that can be done if people really want this possibility. So the policy would mandate a set of variables that could be communicated either via the environment or via the command-line depending on how authoritative the user wants to be. That would be fine for me too. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Manoj Srivastava wrote: However, if the caller really wish that his build options prevail in all cases, he can use make -e (and dpkg-buildpackage has the -R option that let him call debian/rules as make -e -f debian/rules instead). We do not want to override *EVERYTHING* in the Makefile, just the CFLAGS. This blunt instrument of the -e flag is not a good solution. Then he can add the required parameter on the command line instead of using the environment. make -f debian/rules CFLAGS=... target You don't seem to grasp the fact that mandating some variables to be preset doesn't forbid us to rely on Makefile features if we chose to. Apparently there's no way to know from where the variable value come in make. That's true for environment variables like for command line variables. (at least according to my lookup of info make) Err, I think you need to get more conversant with Make. Yes, there is no way to know whether or not the environment variables were set by dpkg or the user; but it is itrivial to make it so that the project wide defaults can be overridden by the site wide stuff, and each being overridden by the package at will. We can even make it so that the package can tell which one set the value, and override only the project wide one and not the site variable. It is trivial to do with whatever solution we retain. The problematic part that I pointed out is how to know in the rules file if the CFLAGS value comes from the command-line, from the environment or from the makefile itself. Do you have a solution to that ? If no, then there was no point in suggesting me to get more conversant with Make. If yes, show it to me. Suppose you have FOO ?= bar in the Makefile, write me the rest of the Makefile so that I have this: $ FOO=foo make FOO was set in the environment $ make FOO=foo FOO was set on the command-line $ make FOO was set in the Makefile Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Raphael Hertzog wrote: On Mon, 16 Mar 2009, Manoj Srivastava wrote: In the non-snippet method proposed, there is no way for the package maintainer to override project defaults yet cater user set variable settings, since the information is lost. You're not trying very hard to look from both sides: whether the default value comes from the environment or from an included Makefile, in both cases the user can override it with command-line arguments. Granted it means that dpkg-buildpackage would have to pass user-overriden flags on the command line instead of using the environment, but that can be done if people really want this possibility. So the policy would mandate a set of variables that could be communicated either via the environment or via the command-line depending on how authoritative the user wants to be. That would be fine for me too. That still fails on two points: a) Not using dpkg would mean different build environments b) there is no way for a site or buildd admin to provide site wide defaults that override dpkg ones but the user can override Also, a dding new project wide builkd defaults is tied to dpkg development and releases (which is not really a great idea either manoj -- ...a most excellent barbarian ... Genghis Kahn! _Bill And Ted's Excellent Adventure_ Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16, 2009 at 06:37:40PM +0100, Raphael Hertzog wrote: Suppose you have FOO ?= bar in the Makefile, write me the rest of the Makefile so that I have this: $ FOO=foo make FOO was set in the environment $ make FOO=foo FOO was set on the command-line $ make FOO was set in the Makefile $ cat Makefile FOO ?= bar all: ifeq $(origin FOO) command line $(info FOO was set on the command-line) endif ifeq $(origin FOO) environment $(info FOO was set in the environment) endif ifeq $(origin FOO) file $(info FOO was set in the Makefile) endif -- James GPG Key: 1024D/61326D40 2003-09-02 James Vega james...@debian.org signature.asc Description: Digital signature
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16, 2009 at 09:14:32AM +0100, Raphael Hertzog wrote: On Sun, 15 Mar 2009, Stephen Gran wrote: This one time, at band camp, Raphael Hertzog said: Care to elaborate what kind of flexibility you need in this specific case ? I don't. I'm imagining that some of our downstreams may. It's precisely one of our downstreams that pushed the dpkg-buildpackage change (Ubuntu). I don't think most downstreams care a lot about the exact implementation, but they clearly want a way to alter defauld build flags at the distribution level. I do not see a need for dpkg-buildpackage to set CFLAGS etc. in the environment *by default*. What is needed instead is a policy spelling how a package should react when a particular variable is set in the environment. If the variable is unset the package should be built the default way. This would still allow downstream to set them (and objectively, nothing has ever prevented downstream to set them in the first place). At this point it seems clear to me that: 1) dpkg-buildpackage should be changed not to set CFLAGS etc. by default. 2) we need to document how packages are supposed to behave should CFLAGS etc. be set in the environment at build-time 3) dpkg-buildpackage should provide a configurable way to set CFLAGS etc. to some values if this is considered useful. dpkg-buildpackage setting CFLAGS per default is both useless and harmful. On the other hand, this proposal still allow downstream to use dpkg-buildpackage in the same way as with lenny, and preserve the same level of functionality Cheers, -- Bill. ballo...@debian.org Imagine a large red swirl here. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: You're not trying very hard to look from both sides: whether the default value comes from the environment or from an included Makefile, in both cases the user can override it with command-line arguments. Granted it means that dpkg-buildpackage would have to pass user-overriden flags on the command line instead of using the environment, but that can be done if people really want this possibility. I think it's clearly mandatory to implement a hierarchy of settings: * Debian defaults * Local distribution overrides * Local package overrides * User settings where each overrides the previous ones. Personally, I'd rather see this done via an included make fragment. I think the logic in debian/rules is simpler in that case, and I don't agree with the argument that it's that much harder to implement than relying on environment variables. In practice, huge numbers of Debian packages already unconditionally set CFLAGS in debian/rules and hence override any environment variable settings. All those packages are going to have to be modified anyway. I don't see a way of meaningfully deploying this change without modifying most of the archive. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Raphael Hertzog wrote: On Mon, 16 Mar 2009, Manoj Srivastava wrote: However, if the caller really wish that his build options prevail in all cases, he can use make -e (and dpkg-buildpackage has the -R option that let him call debian/rules as make -e -f debian/rules instead). We do not want to override *EVERYTHING* in the Makefile, just the CFLAGS. This blunt instrument of the -e flag is not a good solution. Then he can add the required parameter on the command line instead of using the environment. make -f debian/rules CFLAGS=... target You don't seem to grasp the fact that mandating some variables to be preset doesn't forbid us to rely on Makefile features if we chose to. Heh. I never thought I would be lectured on the use of make. Cute. And you are missing the point that making people type stuff on the command line for site specific stuff looses out to being able to edit a conffile instead. dpkg might suffice if we were not wanting the ability to set site wide build defaults a la gentoo use flags -- but it seems silly to give up on that i we are designing a solution now. Apparently there's no way to know from where the variable value come in make. That's true for environment variables like for command line variables. (at least according to my lookup of info make) Err, I think you need to get more conversant with Make. Yes, there is no way to know whether or not the environment variables were set by dpkg or the user; but it is itrivial to make it so that the project wide defaults can be overridden by the site wide stuff, and each being overridden by the package at will. We can even make it so that the package can tell which one set the value, and override only the project wide one and not the site variable. It is trivial to do with whatever solution we retain. The problematic part that I pointed out is how to know in the rules file if the CFLAGS value comes from the command-line, from the environment or from the makefile itself. Do you have a solution to that ? If no, then there was no point in Of course I do. suggesting me to get more conversant with Make. If yes, show it to me. *Sigh*. --8---cut here---start-8--- File: make.info, Node: Origin Function, Next: Flavor Function, Prev: Eval Function, Up: Functions --8---cut here---end---8--- This is precisely what the origin function is for. Also, you might not need it with the snippet example: --8---cut here---start-8--- default_foo := BAR site_foo := baz # by default, this is empty -- set by site admin ifndef (,$(strip $(site_foo))) foo = $(site_foo) else foo = $(default_foo) endif --8---cut here---end---8--- Now the variable foo is set, but we can tell it was set in a makefile, and set by default or the site admin. And make a decision in the rules file based on that. (Note the use of := versus = above; the difference is significant, really). We can also tell if a variable was set in the environment, or the command line -- but the whole include makefile snippet means we do not have to replicate what dpkg does on the command line if we do not want to use that single point of entry. Suppose you have FOO ?= bar in the Makefile, write me the rest of the Makefile so that I have this: $ FOO=foo make FOO was set in the environment $ make FOO=foo FOO was set on the command-line $ make FOO was set in the Makefile Goodness me. manoj -- The world is coming to an end ... SAVE YOUR BUFFERS!!! Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Russ Allbery wrote: Raphael Hertzog hert...@debian.org writes: You're not trying very hard to look from both sides: whether the default value comes from the environment or from an included Makefile, in both cases the user can override it with command-line arguments. Granted it means that dpkg-buildpackage would have to pass user-overriden flags on the command line instead of using the environment, but that can be done if people really want this possibility. I think it's clearly mandatory to implement a hierarchy of settings: * Debian defaults * Local distribution overrides * Local package overrides * User settings where each overrides the previous ones. We can do better than that with the Make snippets. Say, as an user, I think my admin is nuts, and I want o override his setting, and revert to the project wide default, for all packages I build on this machine. Easy (just set site_foo in the env, call debian/rules with -e). Or, as an admin, I want to always override the project wide settings (hey, I like hardening flags), but on some machines I have taken time out to set up site wide defaults -- and I do not want to over ride those. Again, easy to do with the example snippet I posted. All kinds of flexibility become ours once we embrace toe sour^H^H^Hsnippets, I mean. Personally, I'd rather see this done via an included make fragment. I think the logic in debian/rules is simpler in that case, and I don't agree with the argument that it's that much harder to implement than relying on environment variables. In practice, huge numbers of Debian packages already unconditionally set CFLAGS in debian/rules and hence override any environment variable settings. All those packages are going to have to be modified anyway. I don't see a way of meaningfully deploying this change without modifying most of the archive. The advantage of the snippet approach is where packages might break if the builtin defaults are changed will not be broken by the snippet approach, since the change is opt-in. manoj -- It is a city built of bones, and daubed with flesh and blood, in which old age and death, pride and hypocrisy are the inhabitants. 150 Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
I think it's clearly mandatory to implement a hierarchy of settings: * Debian defaults * Local distribution overrides * Local package overrides * User settings where each overrides the previous ones. I think we all mostly agree on that. I see only two remarks: - the package can either fully override the default settings or filter the provided build options: i.e. add/remove/replace build options. (and I think that the filter option should be the recommended approach) - the user should also be able to provide a default build option that the package can override in case the user wants to trust the maintainer's opinion on what build options are sane or not for this specific package (it might allow him to have a better success rate if you want to recompile lots of packages with some options that are known to not work in some cases) Do we want that hierarchy set in stone in policy or do we simply want to define how the rules file is supposed to retrieve the relevant build options ? What would the policy change look like if we select the Makefile snippet approach ? In practice, huge numbers of Debian packages already unconditionally set CFLAGS in debian/rules and hence override any environment variable settings. All those packages are going to have to be modified anyway. I don't see a way of meaningfully deploying this change without modifying most of the archive. It would be nice to have figures here. Note that we have packages switching to debhelper 7 rules files as simple as /usr/share/doc/debhelper/examples/rules.tiny so the number of packages explicitely setting CFLAGS might drop. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Manoj Srivastava wrote: The advantage of the snippet approach is where packages might break if the builtin defaults are changed will not be broken by the snippet approach, since the change is opt-in. Once a package relies on values provided by a Makefile snippet, it might also break when the values provided by the snippet change. Otherwise we lose the advantage of facilitating distribution-wide changes of default build options. I don't see how you reconcile everything. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Manoj Srivastava wrote: And you are missing the point that making people type stuff on the command line for site specific stuff looses out to being able to edit a conffile instead. Who said the command line was for site-specific stuff? In my proposition the hierarchy could be something like this: - distribution default (encoded in dpkg-buildpackage or in a /etc/dpkg/origins/ file) - site-specific (/etc/dpkg/dpkg-buildpackage.conf) - package specific (inside debian/rules) - user specific (command line options) In my opinion, the origin of the variable is not important, the only important part is to specifiy in policy how the value is communicated to the rules file. Do you have a solution to that ? If no, then there was no point in Of course I do. This is precisely what the origin function is for. Then it would have been nice to point it out directly instead of making fun of me while I made it clear that I have read the whole section about variables (where there was no reference to this origin function). Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: I think we all mostly agree on that. I see only two remarks: - the package can either fully override the default settings or filter the provided build options: i.e. add/remove/replace build options. (and I think that the filter option should be the recommended approach) Yes. - the user should also be able to provide a default build option that the package can override in case the user wants to trust the maintainer's opinion on what build options are sane or not for this specific package (it might allow him to have a better success rate if you want to recompile lots of packages with some options that are known to not work in some cases) I don't think this is particularly important at a level of granularity less than the system configuration level (in other words, having the user do this by changing a conffile would be fine with me). I base this on my experience as a user overriding compilation flags long before I'd ever used Linux, where if need be I'll just cut and paste what the package currently uses and change it slightly when building regular packages. This is pretty much par for the course for overriding configuration like that. Do we want that hierarchy set in stone in policy or do we simply want to define how the rules file is supposed to retrieve the relevant build options ? I think we should say something about the hierarchy of options in Policy at least as non-normative text as an explanation for what on earth we're trying to accomplish, so that ten years from now we can figure out what we were thinking. What would the policy change look like if we select the Makefile snippet approach ? Roughly: All packages should add: include /etc/dpkg-dev/build-options.mk to their debian/rules file. This file will set the following options: CFLAGS LDFLAGS ... Packages should use those settings by default (with appropriate changes or propagation into other variables as needed by the build system of the package -- insert explanation of what each of them means here), but the package may override or change those settings if they're not appropriate for this particular package. Users when building a package may override both the defaults and debian/rules by setting the appropriate variable on the make command line with, for example: make -f debian/rules CFLAGS=blah Not in Policy: dpkg-buildpackage could certainly have a useful option for doing the last, and I think that would be a worthwhile feature. It would be nice to have figures here. I don't have time to write the code myself, but it should be fairly trivial to find most packages that set CFLAGS explicitly by looking at the rules file in the Lintian lab on gluck. You'll miss more complex makefile include systems, but you'll get most of the cases. Note that we have packages switching to debhelper 7 rules files as simple as /usr/share/doc/debhelper/examples/rules.tiny so the number of packages explicitely setting CFLAGS might drop. Yes, agreed. Note that all CDBS packages can easily switch to whatever we want, including makefile snippets, with a rev to CDBS. debhelper 7 with minimal rules will probably need to add the include file directly, since debhelper isn't in as good of a position to force the makefile inclusion. (There are ways in which it could, but they all seem very ugly to me.) -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Stefano Zacchiroli wrote: ... and pretty please, do not choose a solution that will require adding an -include to 15'000 thousands debian/rules; we will finish doing that by Lenny+50, the earliest. It would take some time, yes; but packages using cdbs would only require a binNMU once cdbs includes that file on its own. IMO this is what looks like The Right Solution; are we willing to ignore it just because it would take some time? Anyway, any package that was last updated before dpkg-buildpackage started to set the environment vars could easily be discarded, because it was never affected by the ENV vars mess. What other approach (which fulfils all the pros already mentioned by Manoj) do you suggest? Cheers, Raphael Geissert -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Fri, 13 Mar 2009, Manoj Srivastava wrote:
There are several things I dislike about the first option.
1. I do not like that policy would turn around 15 years of convention
and suddenly outlaw $(MAKE) -f debian/rules foo. This will break
many of my packages; and I do not think that that is
justified. There are better solutions than the ones proposed.
Packages are not suddenly broken, no. The fact the we require that some
environment variables are set to specific values when we call debian/rules
won't break packages.
(It did break some packages when dpkg-buildpackage started setting those
variables (in the lenny cycle) but they have all been fixed now.)
Note also that all buildd use dpkg-buildpackage and a majority of DD
also use dpkg-buildpackage, directly or indirectly with debuild or
something similar.
2. As a packager, it is easy enough to over ride the setting
externally, by setting my own variables; but this makes it hard for
the end user to set the variables. I can certainly see cases where
one-size-does-not-fit all cases; so making it easy to change the
flags on a per package, per site, and a per compilation case can be,
and should be addressed.
Yes.
3. dpkg-buildpackage is probably the wrong place to put this solution
in.
Why?
The fact that dpkg-buildpackage's setting the variables is not
easily configurable, and presents to make as though it was set
on the commandline, and thus making it hard for the *USER* to set the
flag variables via env variables, is, in my opinion, a major bug.
This is wrong. dpkg-buildpackage preserves the value set by the user if
any. It simply sets default values to all those environment variables if
they have none.
Did you care to check before doing such a bold assertion ?
For reference, here's the code that sets the build options:
my $default_flags = defined $build_opts-{noopt} ? -g -O0 : -g -O2;
my %flags = ( CPPFLAGS = '',
CFLAGS = $default_flags,
CXXFLAGS = $default_flags,
FFLAGS = $default_flags,
LDFLAGS = '',
);
foreach my $flag (keys %flags) {
if ($ENV{$flag}) {
printf(_g(%s: use %s from environment: %s\n), $progname,
$flag, $ENV{$flag});
} else {
$ENV{$flag} = $flags{$flag};
printf(_g(%s: set %s to default value: %s\n), $progname,
$flag, $ENV{$flag});
}
if ($ENV{${flag}_APPEND}) {
$ENV{$flag} .= .$ENV{${flag}_APPEND};
}
}
As a simple, off the cuff design, I can see hewing closer to the
Gentoo use flags solution, and putting in /etc/debian-build.mk makefile
snippet, which contains the default setting of the flags.
We probably don't want to directly include a conffile in our rules files.
I will rather opt for a file in /usr/share/dpkg/ that would try to include a
file from /etc/dpkg to keep the possibility for the local admin to override
default build settings.
Of course, this requires that every package rules file will need
the -include /etc/debian-build.mk at top, which makes it less than
ideal, but I am sure we can come up with a better solution.
Like what? It is the biggest disadvantage compared to the other solution
and the main reason that I still prefer the other approach.
I consider the issue of being able to set these flags on these
levels fairly important; if I have missed some use case of where one
might want to set defaults otherwise (buildd's can set arch specific
flags just like site admins can), please feel free to point it out.
The site-wide configuration of default flags is currently lacking
in dpkg-buildpackage but I can add that support if needed. We only need
to add a configuration file for this.
Cheers,
--
Raphaël Hertzog
Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny :
http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sat, 14 Mar 2009, Stephen Gran wrote: This one time, at band camp, Raphael Hertzog said: * either we modify policy to mandate the set of environment variables that dpkg-buildpackage sets [...] In terms of efforts, the first solution is the easiest. But we aim at the _right_ solution so feel free to design something that makes the second solution viable. I'm more than slightly uncomfortable with mandating a particular tool as the only way to build packages (and mandating something that's different than current policy at that). Note that I'm not asking to mandate the tool. I would like to mandate the fact that packages can rely on some environment variables being set to some values. I'd be much happier with a Makefile snippet and a conversion period. I know it's more work and slower to migrate to, but it seems to me to have the advantage of being the most flexible. Care to elaborate what kind of flexibility you need in this specific case ? Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
This one time, at band camp, Raphael Hertzog said: On Sat, 14 Mar 2009, Stephen Gran wrote: Note that I'm not asking to mandate the tool. I would like to mandate the fact that packages can rely on some environment variables being set to some values. I'd be much happier with a Makefile snippet and a conversion period. I know it's more work and slower to migrate to, but it seems to me to have the advantage of being the most flexible. Care to elaborate what kind of flexibility you need in this specific case ? I don't. I'm imagining that some of our downstreams may. Cheers, -- - | ,''`.Stephen Gran | | : :' :sg...@debian.org | | `. `'Debian user, admin, and developer | |`- http://www.debian.org | - signature.asc Description: Digital signature
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, Mar 15, 2009 at 05:33:31PM +0100, Raphael Hertzog wrote: On Fri, 13 Mar 2009, Manoj Srivastava wrote: There are several things I dislike about the first option. 1. I do not like that policy would turn around 15 years of convention and suddenly outlaw $(MAKE) -f debian/rules foo. This will break many of my packages; and I do not think that that is justified. There are better solutions than the ones proposed. Packages are not suddenly broken, no. The fact the we require that some environment variables are set to specific values when we call debian/rules won't break packages. (It did break some packages when dpkg-buildpackage started setting those variables (in the lenny cycle) but they have all been fixed now.) You are conflating breakage that were reported with breakage that appeared but were not reported. There is no documented semantic for CFLAGS et. al. in Debian policy. While some Makefile handle it in a certain way, this is not mandatory in any way. For example some configure scripts append options to CFLAGS while other will not change it if it is defined. For example a package might be miscompiled if -fno-strict-aliasing is not given to gcc but a setting CFLAGS might cause this setting not to be added by configure, leading to a subtly broken package, that will not be detected at build time. Changing CFLAGS behind the back of the maintainers is potentially changing how the package is build in a undefined way. A package built in such way should be assumed broken. Cheers, -- Bill. ballo...@debian.org Imagine a large red swirl here. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Fri, Mar 13 2009, Manoj Srivastava wrote: There are several things I dislike about the first option. 1. I do not like that policy would turn around 15 years of convention and suddenly outlaw $(MAKE) -f debian/rules foo. This will break many of my packages; and I do not think that that is justified. There are better solutions than the ones proposed. 2. As a packager, it is easy enough to over ride the setting externally, by setting my own variables; but this makes it hard for the end user to set the variables. I can certainly see cases where one-size-does-not-fit all cases; so making it easy to change the flags on a per package, per site, and a per compilation case can be, and should be addressed. 3. dpkg-buildpackage is probably the wrong place to put this solution in. As a simple, off the cuff design, I can see hewing closer to the Gentoo use flags solution, and putting in /etc/debian-build.mk makefile snippet, which contains the default setting of the flags. A) Project wide flags can be set by making that file part of an essential package (perhaps dpkg). B) The site admin can make any changes they want, like adding SELinux or hardening flags in that file. Regular conffile handlig will allow site admins to be aware of any project wide changes. This allows each site to set variables for that site. C) Each package can include that file (-include, so no errors if file does not exist), and then override whatever variables they want in a package specific variation. D) The user can set the variable on the command line, which will override the Makefile versions of the variable, for a one off per compilation change. The fact that dpkg-buildpackage's setting the variables is not easily configurable, and presents to make as though it was set on the commandline, and thus making it hard for the *USER* to set the flag variables via env variables, is, in my opinion, a major bug. Of course, this requires that every package rules file will need the -include /etc/debian-build.mk at top, which makes it less than ideal, but I am sure we can come up with a better solution. I consider the issue of being able to set these flags on these levels fairly important; if I have missed some use case of where one might want to set defaults otherwise (buildd's can set arch specific flags just like site admins can), please feel free to point it out. The external file is nice in the fact that it decouples the installation of the essential package and the rules files; this might be useful for backported packages as well. In the rules file, one may check for the presence of the file to be included, and take different action based on whether it is present or not. Packages may continue to move over to supporting the common included file on their own timetable, by just not including the snippet until the ya re ready. And the snippet may take care of tsome of the DEB_BUILD_OPTIONS stuff. So. Here is a little snippet that sets some of the flags (this is a demo only, I am not setting all the flags here that dpkg sets) --8---cut here---start-8--- # set CC to $(DEB_HOST_GNU_TYPE)-gcc only if a cross-build is detected ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE)) CC=$(DEB_HOST_GNU_TYPE)-gcc else CC = cc endif # Policy 10.1 says to make this the default CFLAGS = -Wall -g ifneq (,$(filter noopt,$(DEB_BUILD_OPTIONS))) CFLAGS += -O0 else CFLAGS += -O2 endif ## ifneq (,$(findstring debug,$(DEB_BUILD_OPTIONS))) ## endif ifeq (,$(findstring nostrip,$(DEB_BUILD_OPTIONS))) STRIP += -s LDFLAGS += -s INT_INSTALL_TARGET = install else INT_INSTALL_TARGET = install endif ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS))) NUMJOBS = $(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS))) MAKEFLAGS += -j$(NUMJOBS) endif --8---cut here---end---8--- manoj -- The computing field is always in need of new cliches. Alan Perlis Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
