Bug#949006: debian-policy: Stop recommending stamp files in debian/rules
Jonathan Nieder: > Niels Thykier wrote: > >> debhelper cannot see "inside" the upstream build system. If you modify >> a .c file, debhelper won't notice and will currently just skip the >> entire build. Alternatively, debhelper will have to invoke the build >> system and rely on it to not be flawed. > > Yes, I think that would be a good behavior (invoking the build system > and if it's flawed, let the packager work with upstream on it). > Especially because the effect is directly on the packagers --- buildds > wouldn't be hurt by this, as you note. > > Is that the proposed debhelper change? Where can I sign up? :) > You can get this behaviour today with: dh $@ --without build-stamp (You probably want to implement "Rules-Requires-Root: no" first) My idea would be to use Rules-Requires-Root as conditional to whether the build-stamp should be enabled by default in a future compat level. The rationale for that conditional being that we do not want to invoke the build system as root by default. >> AFAICT, the current practise recommended by policy have the same issue >> (assuming you implement the stamp file or touch the "build" file). > > Right, using "touch build" or build-stamp is a last resort, for > dealing with irreperable upstream build systems. Having a proper > upstream build system is much better (and isn't all that rare). > > Thanks, > Jonathan > Thanks, ~Niels
Bug#949006: debian-policy: Stop recommending stamp files in debian/rules
Niels Thykier wrote: > debhelper cannot see "inside" the upstream build system. If you modify > a .c file, debhelper won't notice and will currently just skip the > entire build. Alternatively, debhelper will have to invoke the build > system and rely on it to not be flawed. Yes, I think that would be a good behavior (invoking the build system and if it's flawed, let the packager work with upstream on it). Especially because the effect is directly on the packagers --- buildds wouldn't be hurt by this, as you note. Is that the proposed debhelper change? Where can I sign up? :) > AFAICT, the current practise recommended by policy have the same issue > (assuming you implement the stamp file or touch the "build" file). Right, using "touch build" or build-stamp is a last resort, for dealing with irreperable upstream build systems. Having a proper upstream build system is much better (and isn't all that rare). Thanks, Jonathan
Bug#949006: debian-policy: Stop recommending stamp files in debian/rules
Jonathan Nieder: > Hi! > > Niels Thykier wrote: > >> [...] >> >> Notely, this trickery prevents you from hacking on the upstream parts >> and use "dpkg-buildpackage -b -nc -uc -us" to reduce the build times >> for subsequent builds. You would have to add some rm -f calls to >> delete "internal debhelper state files" as well between each >> dpkg-buildpackage call. > > The ideal is to have dependencies correctly set so that if something > changes, the build system knows exactly what needs to be rebuilt. Is > that achieveable in the debhelper context? > > Summary: I don't have a strong opinion about what policy should say to > do with poorly-designed makefiles, but let's make sure debhelper > doesn't enter that category. > > Thanks, > Jonathan > As I see it, there is no way for debhelper to sanely determine whether something should be rebuild or not without just rerunning that part. debhelper cannot see "inside" the upstream build system. If you modify a .c file, debhelper won't notice and will currently just skip the entire build. Alternatively, debhelper will have to invoke the build system and rely on it to not be flawed. AFAICT, the current practise recommended by policy have the same issue (assuming you implement the stamp file or touch the "build" file). Thanks, ~Niels
Bug#949006: debian-policy: Stop recommending stamp files in debian/rules
Hi! Niels Thykier wrote: > I would like to propose that we drop or replace the following > recommendation in Policy: > > """ > When a package has a configuration and build routine which takes a > long time, or when the makefiles are poorly designed, or when build > needs to run clean first, it is a good idea to touch build when the > build process is complete. This will ensure that if debian/rules build > is run again it will not rebuild the whole program. [11] > """ > > (And related footnote about using "build-stamp" instead of "build"). [...] > As I understand it, the primary purpose behind this recommendation > comes from the need for running "debian/rules build && fakeroot > debian/rules binary" and thereby repeating the build step in some > cases (as listed in the text). I don't have a strong opinion about what we should recommend to do with poorly-designed makefiles (e.g. wouldn't it make sense for us to work with upstream to make them less poorly designed?), but I think this bug report is not characterizing the motivation correctly. The motivation is the same as the motivation for rules to be idempotent in any Makefile: it allows one to safely run steps in any order, rerun them without unnecessary work, and so on. In other words, this is a quality of implementation issue that affects all developers working with the source package, not just buildds. If we switch to a policy that focuses only on buildds working, that would take away a large part of what I find appealing about working within Debian. > However, with the advent of Rules-Requires-Root many packages can now > be built with a direct "debian/rules binary" call and here the stamp > file is no longer useful for the above purpose. > > Furthermore, debhelper need some complexity in implementing/emulating > this behaviour. This may sound "easy" until you try to implement this > "correctly" for the following sequence of debian/rules calls: > > debian/rules build-arch > debian/rules binary-indep > > This has resulted in debhelper using arcane trickery such as log files > (up to compat 9) and its own "stamp-log" (compat 10+). Both > techniques have their limitations and causes frustrations for people > that has a "well-behaving" upstream build system as they have to work > around debhelper's trickery. > > Notely, this trickery prevents you from hacking on the upstream parts > and use "dpkg-buildpackage -b -nc -uc -us" to reduce the build times > for subsequent builds. You would have to add some rm -f calls to > delete "internal debhelper state files" as well between each > dpkg-buildpackage call. The ideal is to have dependencies correctly set so that if something changes, the build system knows exactly what needs to be rebuilt. Is that achieveable in the debhelper context? Summary: I don't have a strong opinion about what policy should say to do with poorly-designed makefiles, but let's make sure debhelper doesn't enter that category. Thanks, Jonathan
Bug#949006: debian-policy: Stop recommending stamp files in debian/rules
Source: debian-policy Severity: wishlist Hi, I would like to propose that we drop or replace the following recommendation in Policy: """ When a package has a configuration and build routine which takes a long time, or when the makefiles are poorly designed, or when build needs to run clean first, it is a good idea to touch build when the build process is complete. This will ensure that if debian/rules build is run again it will not rebuild the whole program. [11] """ (And related footnote about using "build-stamp" instead of "build"). I think a better solution would be to either drop the recommendation entirely or keep it only for packages with this issue and Rules-Requires-Root set to binary-targets. Either way would enable me to move some debhelper hacks out of the "default" in future compat level (at least for people with Rules-Requires-Root). # Rationale: As I understand it, the primary purpose behind this recommendation comes from the need for running "debian/rules build && fakeroot debian/rules binary" and thereby repeating the build step in some cases (as listed in the text). However, with the advent of Rules-Requires-Root many packages can now be built with a direct "debian/rules binary" call and here the stamp file is no longer useful for the above purpose. Furthermore, debhelper need some complexity in implementing/emulating this behaviour. This may sound "easy" until you try to implement this "correctly" for the following sequence of debian/rules calls: debian/rules build-arch debian/rules binary-indep This has resulted in debhelper using arcane trickery such as log files (up to compat 9) and its own "stamp-log" (compat 10+). Both techniques have their limitations and causes frustrations for people that has a "well-behaving" upstream build system as they have to work around debhelper's trickery. Notely, this trickery prevents you from hacking on the upstream parts and use "dpkg-buildpackage -b -nc -uc -us" to reduce the build times for subsequent builds. You would have to add some rm -f calls to delete "internal debhelper state files" as well between each dpkg-buildpackage call. Thanks, ~Niels
