Re: Bug#466550: Pristine source from upstream VCS repository
Manoj Srivastava sriva...@debian.org writes: I would not be against a recommendation in policy to implement direct-from-vcs upstream tarballs to be created vbia get-orig-source, and everyone else just use debian/watch and debian/urepack files. Okay, now I'm officially confused. I don't see how the patch [0] I've submitted for this issue does not satisfy what you're saying. Ideally, I'd like to see you produce a patch for bug#466550 that demonstrates what you're saying, so I can see the difference. I can understand if that's too much effort though. [0] URL:http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=466550#64 -- \ “Oh, I realize it's a penny here and a penny there, but look at | `\ me: I've worked myself up from nothing to a state of extreme | _o__) poverty.” —Groucho Marx | Ben Finney -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, 15 Mar 2009, Bill Allombert wrote: There is no documented semantic for CFLAGS et. al. in Debian policy. While some Makefile handle it in a certain way, this is not mandatory in any way. For example some configure scripts append options to CFLAGS while other will not change it if it is defined. This is precisely what I want to change. We should provide a reasonable way for the user/admin to give default flags and for the packager to override/extend the default set of flags. You are conflating breakage that were reported with breakage that appeared but were not reported. […] Changing CFLAGS behind the back of the maintainers is potentially changing how the package is build in a undefined way. A package built in such way should be assumed broken. So you should assume that most Lenny packages are broken. I know the difference but while I don't deny that some unused packages might be affected by this, it's not my concern right now (those packages are probably candidate for removal if nobody used them to detect that they are not working anymore). Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Sun, 15 Mar 2009, Stephen Gran wrote: This one time, at band camp, Raphael Hertzog said: Care to elaborate what kind of flexibility you need in this specific case ? I don't. I'm imagining that some of our downstreams may. It's precisely one of our downstreams that pushed the dpkg-buildpackage change (Ubuntu). I don't think most downstreams care a lot about the exact implementation, but they clearly want a way to alter defauld build flags at the distribution level. This all started with: https://wiki.ubuntu.com/DistCompilerFlags Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Bug#466550: Pristine source from upstream VCS repository
Ben Finney ben+deb...@benfinney.id.au writes: Ben Hutchings b...@decadent.org.uk writes: # Upstream homepage links to a file called puzzles.tar.gz which # redirects to puzzles-r$version.tar.gz. uscan can't check that. # However, this is a nightly snapshot numbered according to the SVN # revision number, so we can extract the version number from its web # view. version=3 opts=filenamemangle=s/.*\?rev=(\d+).*/sgt-puzzles_$1.orig.tar.gz/,downloadurlmangl...@.*\?rev=(\d+)@http://www.chiark.greenend.org.uk/~sgtatham/puzzles/puzzles-r$1.tar.gz@; \ http://svn.tartarus.org/sgt/puzzles/ /sgt\?rev=(\d+)\view=rev OK, so it's not exactly pretty... It also fails in those instances where “available from a VCS” does not mean “available as a working tree of files via HTTP”. Some VCS repositories make *only* the revision data available, for various reasons. For ffmpeg (and mplayer AFAIUI), upstream provides only revision data, and is heavily using svn:externals, which forced me doing horrible things like this: http://git.debian.org/?p=pkg-multimedia/ffmpeg-debian.git;a=blob;f=debian/get-orig-source.sh;hb=HEAD http://git.debian.org/?p=pkg-multimedia/mplayer.git;a=blob;f=debian/get-orig-source.sh;hb=HEAD Does someone manage to port that script to uscan? ;-) -- Gruesse/greetings, Reinhard Tartler, KeyID 945348A4 -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog wrote: Note that I'm not asking to mandate the tool. I would like to mandate the fact that packages can rely on some environment variables being set to some values. Note that packages will not necessarily pickup the environment variables. autoconf using packages will probably do automatically, but other build-systems still require the maintainer to pick up the changes from the environment and passing them to the build command (eg, scons doesn't have any standard way for that, its up to the package to define a variable). So you don't actually guarantee that all packages use the default flags. -- Felipe Sateler -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: On Fri, 13 Mar 2009, Manoj Srivastava wrote: 3. dpkg-buildpackage is probably the wrong place to put this solution in. Why? The fact that dpkg-buildpackage's setting the variables is not easily configurable, and presents to make as though it was set on the commandline, and thus making it hard for the *USER* to set the flag variables via env variables, is, in my opinion, a major bug. This is wrong. dpkg-buildpackage preserves the value set by the user if any. It simply sets default values to all those environment variables if they have none. I think he ment that you can not know wether the setting comes from dpkg-buildpackage or the user. If it comes from dpkg-buildpackage then debian/rules should be free to override it as needed. If it comes from the user then that is another story. At least that is my take on it. MfG Goswin -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#519835: debian-policy: Please add new sections to policy
As Joerg has just said on d-d-a, some new sections have been added to the archive. I've attached a patch for policy to bring it up-to-date. The list become complex, considering also the priorities of sections. Could we ask ftp-master to give us a fixed-URL to the list of sections, the meaning and priorities? We definitely need to add that link (informative, in a footnote). ciao cate -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#466550: Pristine source from upstream VCS repository
On Mon, Mar 16 2009, Ben Finney wrote: Manoj Srivastava sriva...@debian.org writes: I would not be against a recommendation in policy to implement direct-from-vcs upstream tarballs to be created vbia get-orig-source, and everyone else just use debian/watch and debian/urepack files. Okay, now I'm officially confused. I don't see how the patch [0] I've submitted for this issue does not satisfy what you're saying. Ideally, I'd like to see you produce a patch for bug#466550 that demonstrates what you're saying, so I can see the difference. I can understand if that's too much effort though. I can see how this discussion could have gotten confusing. Use cases: A) Get upstream version from the Debian archive B) Get a specific version from upstream (perhaps to package, or to verify the version in the debian archive) C) Get the latest version from upstream (usually to package it) In cases B and C above, the upstream distribution can be a tarball or a VCS. Let me see if I can capture the current status again. I am starting with a modified form of Kapil's statement early in the report: 1. Once pkg_ver.orig.tar.gz enters the Debian archive this is considered the authoritative Debian version from which all the binary Debian packages will be built (for that version of the package). A signature/checksum is used (in the upload and the Sources.gz file) so as to detect any contamination. apt-get source is enough to get the latest Debian source from the archive (and wget for older sources) 2. Whenever upstream releases a new version, one needs to create a pkg_nver.orig.tar.gz for the newer version. In most cases this is merely a matter of downloading and renaming an upstream tar.gz. 3. If re-packaging of upstream sources was required in order to create this .orig.tar.gz, then this should be documented in the copyright file (with some further explication in README.Debian-source perhaps). 4. If upstream distributes tarballs, the uscan and uupdate programs are adequate and there is no significant need for a get-orig-source target. 5. If the upstream distribution is in the form of a VCS, then uscan does not cater to it. This seems to be the case where get-orig-source can fill a need. There are these three variables that govern the logic: package in Debian already: Yes/No Upstream code Mangling Required: Yes/No Upstream has tarballs: Yes/No Version to Get: Latest/Current In tabular/Karnaugh map form (X are the don't care states): |--+-++--++--| | Already | Version | Has tarballs | Only VCS | | Packaged | to get | Mangle | Pristine | Mangle | Pristine | |--+-++--++--| | Yeslatest | uscan | uscan| GOS| GOS | | Yescurrent | uscan | uscan| GOS| GOS | | No latest | uscan | uscan| GOS| GOS | | No current | X| X | X | X | |--+-++--++--| By logic minimization, the answer is clear :) While the target was originally designed for cases where we had to mangle upstream sources, after this discussion and analysis I am coming to the conclusion that uscan has matured to cover all cases where upstream distributes a tarball; making the target obsolete. The places where we do not have an existing solution is if upstream distributes sources _only_ in a VCS. Now, your patch states: --8---cut here---start-8--- + This target generates the original source archive for + the package, such that its contents exactly match the + original source archive used to generate the package + for Debian. + The commands for this target fetch the original source + package, corresponding to the Debian package version, + from a canonical archive site (for example, via FTP, + WWW, or a public VCS repository), do any necessary rearrangement to turn it into the original source + archive file format, and leave it in the current + directory. See ref id=pkg-sourcearchives for + policy details of the original source archive. --8---cut here---end---8--- There are some places where I differ: a) You ask this target only refer to the version in the changelog, and not the latest version b) You ask the file is left in the current directory, instead of ../ where uscan leaves it c) This patch makes the target work for cases where uscan would be enough -- watch files are useful for DEHS and the PTS and stuff, so we want to recommend watch files anyway, duplicating
Bug#519941: 10.2 Libraries recommends use of /etc/ld.so.conf instead of /etc/ld.so.conf.d
Package: debian-policy Severity: normal Hi, Debian policy 10.2 Libraries says: | Packages containing shared libraries that may be linked to by other | packages' binaries, but which for some compelling reason can not be | installed in /usr/lib directory, may install the shared library files | in subdirectories of the /usr/lib directory, in which case they should | arrange to add that directory in /etc/ld.so.conf in the package's | post-installation script, and remove it in the package's post-removal | script. I believe this should be changed to dropping a file into /etc/ld.so.conf.d instead. Giving a policy for the filename might be a good idea too so there won't be conflicts. How about /etc/ld.so.conf/package.conf? MfG Goswin -- System Information: Debian Release: lenny/sid APT prefers unstable APT policy: (500, 'unstable'), (400, 'unstable-i386'), (1, 'experimental-i386'), (1, 'experimental') Architecture: amd64 (x86_64) Kernel: Linux 2.6.26-xen-1 (SMP w/2 CPU cores) Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) Shell: /bin/sh linked to /bin/bash -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Fri, Mar 13, 2009 at 09:04:30PM +0100, Raphael Hertzog wrote:
* either we modify policy to mandate the set of environment variables that
dpkg-buildpackage sets
snip
In terms of efforts, the first solution is the easiest. But we aim
at the _right_ solution so feel free to design something that makes
the second solution viable.
FWIW I'd love to see the first solution implemented. My main reason,
as a maintainer of packages which mostly escape the autotools / C
language pattern, is to see written somewhere the intended semantics
in Debian of the various environment variable. That would offer an
explanation to maintainers about how to handle such variables even if
their upstreams are not using standard makefiles.
Regarding the objection of mandating dpkg-buildpackage as *the* tool
to build, it looks moot to me. The main point would be defining the
API and the intended meaning of variables. Then if it will happen that
dpkg-buildpackage sets them properly, let's be happy about that or
implement alternative tools.
Same goes for the configuration file objection: dpkg-buildpackage can
have one.
... and pretty please, do not choose a solution that will require
adding an -include to 15'000 thousands debian/rules; we will finish
doing that by Lenny+50, the earliest.
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
z...@{upsilon.cc,pps.jussieu.fr,debian.org} -- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...| ..: | Je dis tu à tous ceux que j'aime
signature.asc
Description: Digital signature
Re: Re: Environment variables, debian/rules and dpkg-buildpackage
I think he ment that you can not know wether the setting comes from dpkg-buildpackage or the user. If it comes from dpkg-buildpackage then debian/rules should be free to override it as needed. If it comes from the user then that is another story. At least that is my take on it. This is a great point. It must be possible to craft a rules file that overrides system or distribution wide defaults and which can be over-ridden by an individual building the package. That seems to require the ability for the code in a rules file to distinguish between things in the environment because they're defaults and things in the environment that are attempting to override defaults. Bdale -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#519941: 10.2 Libraries recommends use of /etc/ld.so.conf instead of /etc/ld.so.conf.d
On Mon, Mar 16, 2009 at 11:40:36AM +0100, Goswin von Brederlow wrote: Debian policy 10.2 Libraries says: | Packages containing shared libraries that may be linked to by other | packages' binaries, but which for some compelling reason can not be | installed in /usr/lib directory, may install the shared library files | in subdirectories of the /usr/lib directory, in which case they should | arrange to add that directory in /etc/ld.so.conf in the package's | post-installation script, and remove it in the package's post-removal | script. I believe this should be changed to dropping a file into /etc/ld.so.conf.d instead. Giving a policy for the filename might be a good idea too so there won't be conflicts. How about /etc/ld.so.conf/package.conf? This recommendation needs to be elminated entirely. It is *not* ok for packages that provide libraries to stick extra linker paths in the global configuration, whether by modifying ld.so.conf or by adding to /etc/ld.so.conf.d. Either the libraries provided by the packages are meant to be public, in which case they should be installed to the standard library path instead of needlessly adding another directory that's going to be globally visible anyway; or they should not, and the cooperating packages should use rpath instead. Use of rpath should still be discouraged, but if someone is bound and determined to violate the FHS with their library paths in order to have private libraries, they should make them really private with rpath instead of using this compromise solution that takes the worst of each approach. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#519941: 10.2 Libraries recommends use of /etc/ld.so.conf instead of /etc/ld.so.conf.d
* Steve Langasek [Mon, 16 Mar 2009 07:52:10 -0700]: On Mon, Mar 16, 2009 at 11:40:36AM +0100, Goswin von Brederlow wrote: Debian policy 10.2 Libraries says: | Packages containing shared libraries that may be linked to by other | packages' binaries, but which for some compelling reason can not be | installed in /usr/lib directory, may install the shared library files | in subdirectories of the /usr/lib directory, in which case they should | arrange to add that directory in /etc/ld.so.conf in the package's | post-installation script, and remove it in the package's post-removal | script. I believe this should be changed to dropping a file into /etc/ld.so.conf.d instead. Giving a policy for the filename might be a good idea too so there won't be conflicts. How about /etc/ld.so.conf/package.conf? This recommendation needs to be elminated entirely. It is *not* ok for packages that provide libraries to stick extra linker paths in the global configuration, whether by modifying ld.so.conf or by adding to /etc/ld.so.conf.d. Either the libraries provided by the packages are meant to be public, in which case they should be installed to the standard library path instead of needlessly adding another directory that's going to be globally visible anyway; or they should not, and the cooperating packages should use rpath instead. Use of rpath should still be discouraged, but if someone is bound and determined to violate the FHS with their library paths in order to have private libraries, they should make them really private with rpath instead of using this compromise solution that takes the worst of each approach. +1 -- - Are you sure we're good? - Always. -- Rory and Lorelai -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Bdale Garbee wrote: I think he ment that you can not know wether the setting comes from dpkg-buildpackage or the user. If it comes from dpkg-buildpackage then debian/rules should be free to override it as needed. If it comes from the user then that is another story. At least that is my take on it. This is a great point. It must be possible to craft a rules file that overrides system or distribution wide defaults and which can be over-ridden by an individual building the package. I don't see why it's needed, it's the job of the rules file to adjust the flags if there's a reason to deviate (like using -O0 instead of -O2 for some arch with compiler problems) but in general the package should not override completely the default flags but just complete them and/or adjust/fix them if needed. However, if the caller really wish that his build options prevail in all cases, he can use make -e (and dpkg-buildpackage has the -R option that let him call debian/rules as make -e -f debian/rules instead). If necessary we can add a new option --force-flags to dpkg-bp or similar to make this even easier. That seems to require the ability for the code in a rules file to distinguish between things in the environment because they're defaults and things in the environment that are attempting to override defaults. Apparently there's no way to know from where the variable value come in make. That's true for environment variables like for command line variables. (at least according to my lookup of info make) So if we really want the rules files to be able to know, we have to add yet another requirement to create this information. This is not desirable IMO. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Raphael Hertzog wrote: On Sun, 15 Mar 2009, Bill Allombert wrote: There is no documented semantic for CFLAGS et. al. in Debian policy. While some Makefile handle it in a certain way, this is not mandatory in any way. For example some configure scripts append options to CFLAGS while other will not change it if it is defined. This is precisely what I want to change. We should provide a reasonable way for the user/admin to give default flags and for the packager to override/extend the default set of flags. So here are what I think we need, based on the most common use cases: A) Provide a way to specify project wide defaults for env variables B) Allow a site admin to selectively override these, and provide site wide defaults C) Allow a package to set some flags that would otherwise break the package, overriding site and/or project defaults as needed D) Allow the user to specify and override any of the above. In the non-snippet method proposed, there is no way for the package maintainer to override project defaults yet cater user set variable settings, since the information is lost. Also, I think that just CFLAGS is probably too broad, I might want to override only some of the categories of flags (optimization, machine related, warning, debugging, etc) and not others; pulling warning and optimization out as separate subflags might be a good compromise. All of this can be done with Makefile snippets, and do not allow you to mandate any set of tools, without which the environment set would be different. You can use dpkg-vuildpackage, $MAKE) -f deban/rules, write a wrapper Makefile that include ./debian/rules -- it all would just work. I do not see any reason to limit ourselves to one tool to set the env variables, especially since it prevents the package maintainer from distinguishing between the project wide defaults anduser set values, and has no easy way for a site to set site wide values. Hey, I might like hardening flags the project does not cater to when I rebuild the archive on my buildd -- and the snippet approach allows for it. manoj -- s = (char*)(long)retval; /* ouch */ Larry Wall in doio.c from the perl source code Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Bug#466550: Pristine source from upstream VCS repository
On Mon, Mar 16 2009, Stefano Zacchiroli wrote: In a sense, while uscan offers an implementation, the policy offer its API. The two are complementary and I don't see why I should loose one because of the other. Also, having an API, offers exactly encapsulation, in the sense that you can use uscan to implement it, but you are not forced to. What is the problem with that? It is like *the* point I'm missing in this whole discussion. I am opposed to bloating the policy with dictum that are unnecessary, but I see your point about the API. The API is essentially the watch file, and we already specify that in the policy. DEHS (as mentioned in policy) uses that file;p there is no need to add stuff into policy that we already have. manoj -- You get what you pay for. Gabriel Biel Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Raphael Hertzog wrote: On Mon, 16 Mar 2009, Bdale Garbee wrote: I think he ment that you can not know wether the setting comes from dpkg-buildpackage or the user. If it comes from dpkg-buildpackage then debian/rules should be free to override it as needed. If it comes from the user then that is another story. At least that is my take on it. This is a great point. It must be possible to craft a rules file that overrides system or distribution wide defaults and which can be over-ridden by an individual building the package. I don't see why it's needed, it's the job of the rules file to adjust the flags if there's a reason to deviate (like using -O0 instead of -O2 for some arch with compiler problems) but in general the package should not override completely the default flags but just complete them and/or adjust/fix them if needed. But while overriding the project wide default makes sense, overriding the human doing hte building does not. However, if the caller really wish that his build options prevail in all cases, he can use make -e (and dpkg-buildpackage has the -R option that let him call debian/rules as make -e -f debian/rules instead). We do not want to override *EVERYTHING* in the Makefile, just the CFLAGS. This blunt instrument of the -e flag is not a good solution. If necessary we can add a new option --force-flags to dpkg-bp or similar to make this even easier. Making a bad solution easy is not a good thing either. That seems to require the ability for the code in a rules file to distinguish between things in the environment because they're defaults and things in the environment that are attempting to override defaults. Apparently there's no way to know from where the variable value come in make. That's true for environment variables like for command line variables. (at least according to my lookup of info make) Err, I think you need to get more conversant with Make. Yes, there is no way to know whether or not the environment variables were set by dpkg or the user; but it is itrivial to make it so that the project wide defaults can be overridden by the site wide stuff, and each being overridden by the package at will. We can even make it so that the package can tell which one set the value, and override only the project wide one and not the site variable. So if we really want the rules files to be able to know, we have to add yet another requirement to create this information. This is not desirable IMO. Given how trivial it is, I think it is eminently desirable. It is just that dpkg is not the right place for such subtleties. manoj -- Though many hands make light work, too many cooks spoil the broth. Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Manoj Srivastava wrote: In the non-snippet method proposed, there is no way for the package maintainer to override project defaults yet cater user set variable settings, since the information is lost. You're not trying very hard to look from both sides: whether the default value comes from the environment or from an included Makefile, in both cases the user can override it with command-line arguments. Granted it means that dpkg-buildpackage would have to pass user-overriden flags on the command line instead of using the environment, but that can be done if people really want this possibility. So the policy would mandate a set of variables that could be communicated either via the environment or via the command-line depending on how authoritative the user wants to be. That would be fine for me too. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Manoj Srivastava wrote: However, if the caller really wish that his build options prevail in all cases, he can use make -e (and dpkg-buildpackage has the -R option that let him call debian/rules as make -e -f debian/rules instead). We do not want to override *EVERYTHING* in the Makefile, just the CFLAGS. This blunt instrument of the -e flag is not a good solution. Then he can add the required parameter on the command line instead of using the environment. make -f debian/rules CFLAGS=... target You don't seem to grasp the fact that mandating some variables to be preset doesn't forbid us to rely on Makefile features if we chose to. Apparently there's no way to know from where the variable value come in make. That's true for environment variables like for command line variables. (at least according to my lookup of info make) Err, I think you need to get more conversant with Make. Yes, there is no way to know whether or not the environment variables were set by dpkg or the user; but it is itrivial to make it so that the project wide defaults can be overridden by the site wide stuff, and each being overridden by the package at will. We can even make it so that the package can tell which one set the value, and override only the project wide one and not the site variable. It is trivial to do with whatever solution we retain. The problematic part that I pointed out is how to know in the rules file if the CFLAGS value comes from the command-line, from the environment or from the makefile itself. Do you have a solution to that ? If no, then there was no point in suggesting me to get more conversant with Make. If yes, show it to me. Suppose you have FOO ?= bar in the Makefile, write me the rest of the Makefile so that I have this: $ FOO=foo make FOO was set in the environment $ make FOO=foo FOO was set on the command-line $ make FOO was set in the Makefile Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#519941: 10.2 Libraries recommends use of /etc/ld.so.conf instead of /etc/ld.so.conf.d
Steve Langasek vor...@debian.org writes: This recommendation needs to be elminated entirely. It is *not* ok for packages that provide libraries to stick extra linker paths in the global configuration, whether by modifying ld.so.conf or by adding to /etc/ld.so.conf.d. Either the libraries provided by the packages are meant to be public, in which case they should be installed to the standard library path instead of needlessly adding another directory that's going to be globally visible anyway; or they should not, and the cooperating packages should use rpath instead. Use of rpath should still be discouraged, but if someone is bound and determined to violate the FHS with their library paths in order to have private libraries, they should make them really private with rpath instead of using this compromise solution that takes the worst of each approach. Seconded. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Raphael Hertzog wrote: On Mon, 16 Mar 2009, Manoj Srivastava wrote: In the non-snippet method proposed, there is no way for the package maintainer to override project defaults yet cater user set variable settings, since the information is lost. You're not trying very hard to look from both sides: whether the default value comes from the environment or from an included Makefile, in both cases the user can override it with command-line arguments. Granted it means that dpkg-buildpackage would have to pass user-overriden flags on the command line instead of using the environment, but that can be done if people really want this possibility. So the policy would mandate a set of variables that could be communicated either via the environment or via the command-line depending on how authoritative the user wants to be. That would be fine for me too. That still fails on two points: a) Not using dpkg would mean different build environments b) there is no way for a site or buildd admin to provide site wide defaults that override dpkg ones but the user can override Also, a dding new project wide builkd defaults is tied to dpkg development and releases (which is not really a great idea either manoj -- ...a most excellent barbarian ... Genghis Kahn! _Bill And Ted's Excellent Adventure_ Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16, 2009 at 06:37:40PM +0100, Raphael Hertzog wrote: Suppose you have FOO ?= bar in the Makefile, write me the rest of the Makefile so that I have this: $ FOO=foo make FOO was set in the environment $ make FOO=foo FOO was set on the command-line $ make FOO was set in the Makefile $ cat Makefile FOO ?= bar all: ifeq $(origin FOO) command line $(info FOO was set on the command-line) endif ifeq $(origin FOO) environment $(info FOO was set in the environment) endif ifeq $(origin FOO) file $(info FOO was set in the Makefile) endif -- James GPG Key: 1024D/61326D40 2003-09-02 James Vega james...@debian.org signature.asc Description: Digital signature
Bug#519941: 10.2 Libraries recommends use of /etc/ld.so.conf instead of /etc/ld.so.conf.d
On Mon, Mar 16, 2009 at 10:44:49AM -0700, Russ Allbery wrote: Steve Langasek vor...@debian.org writes: This recommendation needs to be elminated entirely. It is *not* ok for packages that provide libraries to stick extra linker paths in the global configuration, whether by modifying ld.so.conf or by adding to /etc/ld.so.conf.d. Either the libraries provided by the packages are meant to be public, in which case they should be installed to the standard library path instead of needlessly adding another directory that's going to be globally visible anyway; or they should not, and the cooperating packages should use rpath instead. Use of rpath should still be discouraged, but if someone is bound and determined to violate the FHS with their library paths in order to have private libraries, they should make them really private with rpath instead of using this compromise solution that takes the worst of each approach. Seconded. I also agree. Kurt -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16, 2009 at 09:14:32AM +0100, Raphael Hertzog wrote: On Sun, 15 Mar 2009, Stephen Gran wrote: This one time, at band camp, Raphael Hertzog said: Care to elaborate what kind of flexibility you need in this specific case ? I don't. I'm imagining that some of our downstreams may. It's precisely one of our downstreams that pushed the dpkg-buildpackage change (Ubuntu). I don't think most downstreams care a lot about the exact implementation, but they clearly want a way to alter defauld build flags at the distribution level. I do not see a need for dpkg-buildpackage to set CFLAGS etc. in the environment *by default*. What is needed instead is a policy spelling how a package should react when a particular variable is set in the environment. If the variable is unset the package should be built the default way. This would still allow downstream to set them (and objectively, nothing has ever prevented downstream to set them in the first place). At this point it seems clear to me that: 1) dpkg-buildpackage should be changed not to set CFLAGS etc. by default. 2) we need to document how packages are supposed to behave should CFLAGS etc. be set in the environment at build-time 3) dpkg-buildpackage should provide a configurable way to set CFLAGS etc. to some values if this is considered useful. dpkg-buildpackage setting CFLAGS per default is both useless and harmful. On the other hand, this proposal still allow downstream to use dpkg-buildpackage in the same way as with lenny, and preserve the same level of functionality Cheers, -- Bill. ballo...@debian.org Imagine a large red swirl here. -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: You're not trying very hard to look from both sides: whether the default value comes from the environment or from an included Makefile, in both cases the user can override it with command-line arguments. Granted it means that dpkg-buildpackage would have to pass user-overriden flags on the command line instead of using the environment, but that can be done if people really want this possibility. I think it's clearly mandatory to implement a hierarchy of settings: * Debian defaults * Local distribution overrides * Local package overrides * User settings where each overrides the previous ones. Personally, I'd rather see this done via an included make fragment. I think the logic in debian/rules is simpler in that case, and I don't agree with the argument that it's that much harder to implement than relying on environment variables. In practice, huge numbers of Debian packages already unconditionally set CFLAGS in debian/rules and hence override any environment variable settings. All those packages are going to have to be modified anyway. I don't see a way of meaningfully deploying this change without modifying most of the archive. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Raphael Hertzog wrote: On Mon, 16 Mar 2009, Manoj Srivastava wrote: However, if the caller really wish that his build options prevail in all cases, he can use make -e (and dpkg-buildpackage has the -R option that let him call debian/rules as make -e -f debian/rules instead). We do not want to override *EVERYTHING* in the Makefile, just the CFLAGS. This blunt instrument of the -e flag is not a good solution. Then he can add the required parameter on the command line instead of using the environment. make -f debian/rules CFLAGS=... target You don't seem to grasp the fact that mandating some variables to be preset doesn't forbid us to rely on Makefile features if we chose to. Heh. I never thought I would be lectured on the use of make. Cute. And you are missing the point that making people type stuff on the command line for site specific stuff looses out to being able to edit a conffile instead. dpkg might suffice if we were not wanting the ability to set site wide build defaults a la gentoo use flags -- but it seems silly to give up on that i we are designing a solution now. Apparently there's no way to know from where the variable value come in make. That's true for environment variables like for command line variables. (at least according to my lookup of info make) Err, I think you need to get more conversant with Make. Yes, there is no way to know whether or not the environment variables were set by dpkg or the user; but it is itrivial to make it so that the project wide defaults can be overridden by the site wide stuff, and each being overridden by the package at will. We can even make it so that the package can tell which one set the value, and override only the project wide one and not the site variable. It is trivial to do with whatever solution we retain. The problematic part that I pointed out is how to know in the rules file if the CFLAGS value comes from the command-line, from the environment or from the makefile itself. Do you have a solution to that ? If no, then there was no point in Of course I do. suggesting me to get more conversant with Make. If yes, show it to me. *Sigh*. --8---cut here---start-8--- File: make.info, Node: Origin Function, Next: Flavor Function, Prev: Eval Function, Up: Functions --8---cut here---end---8--- This is precisely what the origin function is for. Also, you might not need it with the snippet example: --8---cut here---start-8--- default_foo := BAR site_foo := baz # by default, this is empty -- set by site admin ifndef (,$(strip $(site_foo))) foo = $(site_foo) else foo = $(default_foo) endif --8---cut here---end---8--- Now the variable foo is set, but we can tell it was set in a makefile, and set by default or the site admin. And make a decision in the rules file based on that. (Note the use of := versus = above; the difference is significant, really). We can also tell if a variable was set in the environment, or the command line -- but the whole include makefile snippet means we do not have to replicate what dpkg does on the command line if we do not want to use that single point of entry. Suppose you have FOO ?= bar in the Makefile, write me the rest of the Makefile so that I have this: $ FOO=foo make FOO was set in the environment $ make FOO=foo FOO was set on the command-line $ make FOO was set in the Makefile Goodness me. manoj -- The world is coming to an end ... SAVE YOUR BUFFERS!!! Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#519941: 10.2 Libraries recommends use of /etc/ld.so.conf instead of /etc/ld.so.conf.d
On Mon, Mar 16, 2009 at 07:52:10AM -0700, Steve Langasek wrote: On Mon, Mar 16, 2009 at 11:40:36AM +0100, Goswin von Brederlow wrote: Debian policy 10.2 Libraries says: | Packages containing shared libraries that may be linked to by other | packages' binaries, but which for some compelling reason can not be | installed in /usr/lib directory, may install the shared library files | in subdirectories of the /usr/lib directory, in which case they should | arrange to add that directory in /etc/ld.so.conf in the package's | post-installation script, and remove it in the package's post-removal | script. I believe this should be changed to dropping a file into /etc/ld.so.conf.d instead. Giving a policy for the filename might be a good idea too so there won't be conflicts. How about /etc/ld.so.conf/package.conf? This recommendation needs to be elminated entirely. It is *not* ok for packages that provide libraries to stick extra linker paths in the global configuration, whether by modifying ld.so.conf or by adding to /etc/ld.so.conf.d. Either the libraries provided by the packages are meant to be public, in which case they should be installed to the standard library path instead of needlessly adding another directory that's going to be globally visible anyway; or they should not, and the cooperating packages should use rpath instead. Use of rpath should still be discouraged, but if someone is bound and determined to violate the FHS with their library paths in order to have private libraries, they should make them really private with rpath instead of using this compromise solution that takes the worst of each approach. I do not have a strong opinion on this because policy says for some compelling reason and we can imagine setting where rpath does not work. But anyway, the major offender is libatlas-base-dev: #cat /etc/ld.so.conf.d/atlas.conf /usr/lib/atlas Maybe we should start by reporting a bug to libatlas-base-dev ? Cheers, -- Bill. ballo...@debian.org Imagine a large red swirl here. -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, Mar 16 2009, Russ Allbery wrote: Raphael Hertzog hert...@debian.org writes: You're not trying very hard to look from both sides: whether the default value comes from the environment or from an included Makefile, in both cases the user can override it with command-line arguments. Granted it means that dpkg-buildpackage would have to pass user-overriden flags on the command line instead of using the environment, but that can be done if people really want this possibility. I think it's clearly mandatory to implement a hierarchy of settings: * Debian defaults * Local distribution overrides * Local package overrides * User settings where each overrides the previous ones. We can do better than that with the Make snippets. Say, as an user, I think my admin is nuts, and I want o override his setting, and revert to the project wide default, for all packages I build on this machine. Easy (just set site_foo in the env, call debian/rules with -e). Or, as an admin, I want to always override the project wide settings (hey, I like hardening flags), but on some machines I have taken time out to set up site wide defaults -- and I do not want to over ride those. Again, easy to do with the example snippet I posted. All kinds of flexibility become ours once we embrace toe sour^H^H^Hsnippets, I mean. Personally, I'd rather see this done via an included make fragment. I think the logic in debian/rules is simpler in that case, and I don't agree with the argument that it's that much harder to implement than relying on environment variables. In practice, huge numbers of Debian packages already unconditionally set CFLAGS in debian/rules and hence override any environment variable settings. All those packages are going to have to be modified anyway. I don't see a way of meaningfully deploying this change without modifying most of the archive. The advantage of the snippet approach is where packages might break if the builtin defaults are changed will not be broken by the snippet approach, since the change is opt-in. manoj -- It is a city built of bones, and daubed with flesh and blood, in which old age and death, pride and hypocrisy are the inhabitants. 150 Manoj Srivastava sriva...@debian.org http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
I think it's clearly mandatory to implement a hierarchy of settings: * Debian defaults * Local distribution overrides * Local package overrides * User settings where each overrides the previous ones. I think we all mostly agree on that. I see only two remarks: - the package can either fully override the default settings or filter the provided build options: i.e. add/remove/replace build options. (and I think that the filter option should be the recommended approach) - the user should also be able to provide a default build option that the package can override in case the user wants to trust the maintainer's opinion on what build options are sane or not for this specific package (it might allow him to have a better success rate if you want to recompile lots of packages with some options that are known to not work in some cases) Do we want that hierarchy set in stone in policy or do we simply want to define how the rules file is supposed to retrieve the relevant build options ? What would the policy change look like if we select the Makefile snippet approach ? In practice, huge numbers of Debian packages already unconditionally set CFLAGS in debian/rules and hence override any environment variable settings. All those packages are going to have to be modified anyway. I don't see a way of meaningfully deploying this change without modifying most of the archive. It would be nice to have figures here. Note that we have packages switching to debhelper 7 rules files as simple as /usr/share/doc/debhelper/examples/rules.tiny so the number of packages explicitely setting CFLAGS might drop. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Manoj Srivastava wrote: The advantage of the snippet approach is where packages might break if the builtin defaults are changed will not be broken by the snippet approach, since the change is opt-in. Once a package relies on values provided by a Makefile snippet, it might also break when the values provided by the snippet change. Otherwise we lose the advantage of facilitating distribution-wide changes of default build options. I don't see how you reconcile everything. Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
On Mon, 16 Mar 2009, Manoj Srivastava wrote: And you are missing the point that making people type stuff on the command line for site specific stuff looses out to being able to edit a conffile instead. Who said the command line was for site-specific stuff? In my proposition the hierarchy could be something like this: - distribution default (encoded in dpkg-buildpackage or in a /etc/dpkg/origins/ file) - site-specific (/etc/dpkg/dpkg-buildpackage.conf) - package specific (inside debian/rules) - user specific (command line options) In my opinion, the origin of the variable is not important, the only important part is to specifiy in policy how the value is communicated to the rules file. Do you have a solution to that ? If no, then there was no point in Of course I do. This is precisely what the origin function is for. Then it would have been nice to point it out directly instead of making fun of me while I made it clear that I have read the whole section about variables (where there was no reference to this origin function). Cheers, -- Raphaël Hertzog Contribuez à Debian et gagnez un cahier de l'admin Debian Lenny : http://www.ouaza.com/wp/2009/03/02/contribuer-a-debian-gagner-un-livre/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Raphael Hertzog hert...@debian.org writes: I think we all mostly agree on that. I see only two remarks: - the package can either fully override the default settings or filter the provided build options: i.e. add/remove/replace build options. (and I think that the filter option should be the recommended approach) Yes. - the user should also be able to provide a default build option that the package can override in case the user wants to trust the maintainer's opinion on what build options are sane or not for this specific package (it might allow him to have a better success rate if you want to recompile lots of packages with some options that are known to not work in some cases) I don't think this is particularly important at a level of granularity less than the system configuration level (in other words, having the user do this by changing a conffile would be fine with me). I base this on my experience as a user overriding compilation flags long before I'd ever used Linux, where if need be I'll just cut and paste what the package currently uses and change it slightly when building regular packages. This is pretty much par for the course for overriding configuration like that. Do we want that hierarchy set in stone in policy or do we simply want to define how the rules file is supposed to retrieve the relevant build options ? I think we should say something about the hierarchy of options in Policy at least as non-normative text as an explanation for what on earth we're trying to accomplish, so that ten years from now we can figure out what we were thinking. What would the policy change look like if we select the Makefile snippet approach ? Roughly: All packages should add: include /etc/dpkg-dev/build-options.mk to their debian/rules file. This file will set the following options: CFLAGS LDFLAGS ... Packages should use those settings by default (with appropriate changes or propagation into other variables as needed by the build system of the package -- insert explanation of what each of them means here), but the package may override or change those settings if they're not appropriate for this particular package. Users when building a package may override both the defaults and debian/rules by setting the appropriate variable on the make command line with, for example: make -f debian/rules CFLAGS=blah Not in Policy: dpkg-buildpackage could certainly have a useful option for doing the last, and I think that would be a worthwhile feature. It would be nice to have figures here. I don't have time to write the code myself, but it should be fairly trivial to find most packages that set CFLAGS explicitly by looking at the rules file in the Lintian lab on gluck. You'll miss more complex makefile include systems, but you'll get most of the cases. Note that we have packages switching to debhelper 7 rules files as simple as /usr/share/doc/debhelper/examples/rules.tiny so the number of packages explicitely setting CFLAGS might drop. Yes, agreed. Note that all CDBS packages can easily switch to whatever we want, including makefile snippets, with a rev to CDBS. debhelper 7 with minimal rules will probably need to add the include file directly, since debhelper isn't in as good of a position to force the makefile inclusion. (There are ways in which it could, but they all seem very ugly to me.) -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Environment variables, debian/rules and dpkg-buildpackage
Stefano Zacchiroli wrote: ... and pretty please, do not choose a solution that will require adding an -include to 15'000 thousands debian/rules; we will finish doing that by Lenny+50, the earliest. It would take some time, yes; but packages using cdbs would only require a binNMU once cdbs includes that file on its own. IMO this is what looks like The Right Solution; are we willing to ignore it just because it would take some time? Anyway, any package that was last updated before dpkg-buildpackage started to set the environment vars could easily be discarded, because it was never affected by the ENV vars mess. What other approach (which fulfils all the pros already mentioned by Manoj) do you suggest? Cheers, Raphael Geissert -- To UNSUBSCRIBE, email to debian-policy-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
