Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
On Mon, 10 Feb 2014 22:13:45 +0100 Wouter Verhelst wrote: > Sigh. > > On Wed, Feb 05, 2014 at 12:59:23PM +, Neil Williams wrote: > > Using packages to support upstream development is a common problem /common problem/common source of problems/ > > and this is exactly where things get awkward. > > No, it is not a *problem*; it is a *method* of doing things. ... one which I've used consistently for more years than I've been a DD and had frequent problems with various assumptions in various tools and in more distributions than just Debian and its derivatives. Hopefully the clarification will show that I'm not questioning the methods of anyone (other than possibly my own). > It is not your place (nor mine) to question another person's methods > of doing things; especially not if said methods are done outside of > Debian, as is here the case. ... and in my ongoing work. If using distribution tools for upstream development was easy, we might not have had people developing tools like pypi, ruby gems, CPAN or a whole range of other non-distribution distributive tools. This isn't just a Debian problem. (Indeed fixing it in Debian isn't going to fix the problems because upstreams will rarely fixate on a single distribution across the entire team - for entirely sane reasons.) It is right for upstream to want to deploy to FHS compliant paths and use dependencies from the main distribution system etc. None of the distribution tools for any of the distributions actually make it easy to then develop within those paths without either rebuilding unreleased upstream packages or copying files into privileged paths. Both of these routes need sudo access which just makes things harder again. Why must every developer have sudo access on the development box? That is exactly why pypi and buildout have got so much traction. It annoys me that I have to use such methods for upstream work because dpkg-dev is too constrained by rules which *only* relate to "official" builds. Doing a quick native build of a non-native package for use and distribution within a known team is a *common requirement* for upstream teams. (e.g. it means that developers can push to a branch, get a quick native package built, uploaded locally, installed via an inotify and available to test without the unnecessary step of building an .orig.tar.gz in the middle.) It's not quite as clean or DRY as restarting a daemon directly from a user-privilege git clone but it is workable. Why should that require two branches of the packaging files? Developing using Debian is not just about development of Debian. Upstream teams use dpkg-dev too. Constraints which are entirely warranted for developing packages destined for ftp-master are directly harmful for developing packages destined for a repository on 192.168. Yes this could work with overrides but those overrides need to be build specific (not package specific or version specific). This is exactly why a ~/.gbp.conf is the right approach. > > Not true. There are options to use debuild or pdebuild or > > dpkg-buildpackage in-place. > > > > e.g. I use: > > > > [DEFAULT] > > #builder = git-pbuilder > > builder = debuild > > cleaner = fakeroot debian/rules clean > > pristine-tar = True > > > > [git-buildpackage] > > export-dir = ../build-area/ > > tarball-dir = ../tarballs/ > > Even if so, this increases the complexity of the system, and requires > people to learn yet another tool to Just Do what was previously > possible with no extra fluff. > > It's okay for a tool (like dpkg) to warn if something doesn't look > right. It's not okay for a tool (like dpkg) to pretend to be smarter > than the person operating said tool. True - however, there will always be a need for tools like git-bp and it is common to use aliases and JDTRT scripts to provide a consistent interface no matter what changes beneath. Thankfully, none of those hacks make it into Debian but that does mean that people within Debian don't get to see how the tools are actually used. Switching a non-native package to native arbitrarily is a necessary use of dpkg and it needs to be supported cleanly and in a way which is easy to override using a per-build configuration option. -- Neil Williams = http://www.linux.codehelp.co.uk/ signature.asc Description: PGP signature
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
Sigh. On Wed, Feb 05, 2014 at 12:59:23PM +, Neil Williams wrote: > Using packages to support upstream development is a common problem and > this is exactly where things get awkward. No, it is not a *problem*; it is a *method* of doing things. It is not your place (nor mine) to question another person's methods of doing things; especially not if said methods are done outside of Debian, as is here the case. Enforcing Debian Policy in the tools (i.e., not allowing to do things contrary to debian policy, even if that's wanted) is a *bad* idea, in all cases. [...] > > Also, using git-buildpackage is difficult. > > The build is done by sbuild, which does not call git-buildpackage. > > Not true. There are options to use debuild or pdebuild or > dpkg-buildpackage in-place. > > e.g. I use: > > [DEFAULT] > #builder = git-pbuilder > builder = debuild > cleaner = fakeroot debian/rules clean > pristine-tar = True > > [git-buildpackage] > export-dir = ../build-area/ > tarball-dir = ../tarballs/ Even if so, this increases the complexity of the system, and requires people to learn yet another tool to Just Do what was previously possible with no extra fluff. It's okay for a tool (like dpkg) to warn if something doesn't look right. It's not okay for a tool (like dpkg) to pretend to be smarter than the person operating said tool. -- This end should point toward the ground if you want to go to space. If it starts pointing toward space you are having a bad problem and you will not go to space today. -- http://xkcd.com/1133/ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
> "Bernhard" == Bernhard R Link writes: As I mentioned I have a packaging branch and an upstream branch. I wish to use debian revisions to reflect packaging changes. It's slightly more complex than changes to debian directory involve a debian revision change; changes to other things involve a upstream version change. As an example I produce both RPMs and Debs. Just as I don't want to increment the upstream version number because of a spec file change, I don't want to increment the upstream version number because I updtaded build-depends in debian/control. Especially when the debian directory isn't even on the upstream master branch. Incrementing the upstream version number (which appears in configure.ac among other places) so I could make changes to files that don't even appear on that branch is an undesirable work flow. I guess I could have a debian upstream version number that differed from the actual upstream version number. That seems undesirable from a user expectations standpoint as well as potentially impacting things in unexpected ways. The bug claims that it is a violation of policy to use 3.0(native) without a.orig.tar.something. I'm actually failing to find that in policy at all. I'm finding some SHOULD level recommendations, but certainly not MUST level recommendations, I can think of reasons why a maintainer might want to voiolate those shoulds. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
* Sam Hartman [140205 13:27]: > no, that means I have to maintain the artifact (namely the > .orig.tar.gz). > The archive software (both reprepro and dak were I to use that) require > that the .orig.tar.gz not change checksums. > > I don't want my build machines to be able to push back to my master > repository. > Nor do I want to have to release upstream versions if I lose state on my > build machines. > So this violates my requirements because I have to maintain an artifact > of dpkg-source (the .orig.tar.gz) and makesure its checksum never > changes. This answers the question why you want to use a 3.0 (native) package. But the real question here is: Why do you want to use a version with "-" for such a package? Bernhard R. Link -- F8AC 04D5 0B9B 064B 3383 C3DA AFFC 96D1 151D FFDC -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
> "Neil" == Neil Williams writes: That makes sense and I do something similar as appropriate. Even so, I do not wish to maintain the upstream tarball as a maintained artifact. There are cases where packaging release releases are made. Maintaining pristine-tar commits for daily builds is a worse solution than 3.0(native) or not including source packages in the resulting Debian archive. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
Le Wed, Feb 05, 2014 at 12:46:09PM +0100, Andreas Beckmann a écrit : > > All this sounds like it can be done with git-buildpackage Hello everybody, I have the impression that we are arguing because of solution in search for a problem. Some things worked with the previous version of dpkg, with no extra work for anybody. Who benefited directly from the change of behavior ? Nobody ? Then please revert it; it was not necessary. I propose a compromise. - First, somebody convinces Joey to switch the ikiwiki to a non-native format. If you can twist the arm of a highly reputed developer, it means that the problem that you attempted to solve (a stricter distinction between native and non-native packages) was really important. - Then, we consider that others can follow. - In the meantime, please revert the change in dpkg. There is no need to prevent packages in the 3.0 (native) format to use non-native version number. Alternatively, please rename the "3.0 (native)" format to "3.0 (tarball)" or anything elese, and we are done. Cheers, -- Charles Plessy Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
On Wed, 5 Feb 2014 12:21:30 + Sam Hartman wrote: > > "Andreas" == Andreas Beckmann writes: > > Andreas> On 2014-02-05 10:57, Sam Hartman wrote: > >> tarballs useful; anyone who is likely to want to build this > >> from source probably has a copy of git and can checkout a tag. > > Andreas> Such a tag corresponds to an upstrema version? > > yes. > > >> I'm happy to entertain other options rather than 3.0(native) > >> but my requirements are: > >> > >> * support for upstream version * support for debian revision > >> > >> * No need to have upstream sources available to > >> dpkg-buildpackage prior to running it > >> > >> * No need to maintain .orig.tar.gz artifacts produced by > >> dpkg-source and keep the checksums of these artifacts > >> consistent between packages with the same upstream versions. > > Andreas> All this sounds like it can be done with git-buildpackage > Andreas> --git-pristine-tar --git-pristine-tar-commit. Can be set > Andreas> in debian/gbp.conf. And maybe dpkg-source > Andreas> --single-debian-patch. > > no, that means I have to maintain the artifact (namely the > .orig.tar.gz). > The archive software (both reprepro and dak were I to use that) > require that the .orig.tar.gz not change checksums. Using packages to support upstream development is a common problem and this is exactly where things get awkward. For my own role within an upstream team, I'm considering using "unofficial" or "developer" upstream tarball releases. We'll probably use a date based tag 2014.02 etc for the main monthly release. Developer builds will have a shortened git hash appended (this happens to match our existing deployment method) like 2014.02.234fdga2 and incremental upstream releases will use tag.01 etc. so 2014.02.01 This has advantages that developers self-verify that the tarballs work which finds problems due to new files not being included in the tarball. It also retains the upstream packaging behaviour. > I don't want my build machines to be able to push back to my master > repository. > Nor do I want to have to release upstream versions if I lose state on > my build machines. > So this violates my requirements because I have to maintain an > artifact of dpkg-source (the .orig.tar.gz) and makesure its checksum > never changes. > > Also, using git-buildpackage is difficult. > The build is done by sbuild, which does not call git-buildpackage. Not true. There are options to use debuild or pdebuild or dpkg-buildpackage in-place. e.g. I use: [DEFAULT] #builder = git-pbuilder builder = debuild cleaner = fakeroot debian/rules clean pristine-tar = True [git-buildpackage] export-dir = ../build-area/ tarball-dir = ../tarballs/ -- Neil Williams = http://www.linux.codehelp.co.uk/ signature.asc Description: PGP signature
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
> "Andreas" == Andreas Beckmann writes: Andreas> On 2014-02-05 10:57, Sam Hartman wrote: >> tarballs useful; anyone who is likely to want to build this from >> source probably has a copy of git and can checkout a tag. Andreas> Such a tag corresponds to an upstrema version? yes. >> I'm happy to entertain other options rather than 3.0(native) but >> my requirements are: >> >> * support for upstream version * support for debian revision >> >> * No need to have upstream sources available to dpkg-buildpackage >> prior to running it >> >> * No need to maintain .orig.tar.gz artifacts produced by >> dpkg-source and keep the checksums of these artifacts consistent >> between packages with the same upstream versions. Andreas> All this sounds like it can be done with git-buildpackage Andreas> --git-pristine-tar --git-pristine-tar-commit. Can be set in Andreas> debian/gbp.conf. And maybe dpkg-source Andreas> --single-debian-patch. no, that means I have to maintain the artifact (namely the .orig.tar.gz). The archive software (both reprepro and dak were I to use that) require that the .orig.tar.gz not change checksums. I don't want my build machines to be able to push back to my master repository. Nor do I want to have to release upstream versions if I lose state on my build machines. So this violates my requirements because I have to maintain an artifact of dpkg-source (the .orig.tar.gz) and makesure its checksum never changes. Also, using git-buildpackage is difficult. The build is done by sbuild, which does not call git-buildpackage. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
On 2014-02-05 10:57, Sam Hartman wrote: > tarballs useful; anyone who is likely to want to build this from source > probably has a copy of git and can checkout a tag. Such a tag corresponds to an upstrema version? > I'm happy to entertain other options rather than 3.0(native) but my > requirements are: > > * support for upstream version > * support for debian revision > > * No need to have upstream sources available to dpkg-buildpackage prior > to running it > > * No need to maintain .orig.tar.gz artifacts produced by dpkg-source > and keep the checksums of these artifacts consistent between packages > with the same upstream versions. All this sounds like it can be done with git-buildpackage --git-pristine-tar --git-pristine-tar-commit. Can be set in debian/gbp.conf. And maybe dpkg-source --single-debian-patch. And if this doesn't work for you, we should enhance the tools e.g. git-buildpackage, to better support the desired workflows (i.e. what you really want to achieve), not the workarounds (the way you used achieve this today^Wyesterday^Wbefore dpkg 1.17). And your goal seems to be: "I have a git repository with upstream branch, tags, debian branch and I want an easy solution (command) to build conforming packages without having to worry about details like creating upstream tarballs." and not "I need to upload foo 1.2-3 with source format 3.0 (native)". Andreas -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
control: subscribe -1 > "Charles" == Charles Plessy writes: Charles> The 3.0 (native) format is useful when packaging a work Charles> that is developped and distributed in a Git repository. Charles> Please leave us this possibility. Let me describe the use case I have which is an expansion on the above. I have a bunch of software that I perform daily builds for out of version control (git in my case but the issue applies to other vcs as well). The software does have upstream versions but is not stable enough that upstream release tarballs are useful to anyone. Honestly at this point, I'm not sure anyone will ever find upstream tarballs useful; anyone who is likely to want to build this from source probably has a copy of git and can checkout a tag. There is a packaging branch and an upstream branch. Changes made on the packaging branch increment the debian revision; changes made on the ustream branch eventually involve an increment to the upstream version. Things get dumped into a Debian reprepro repository, and into Ubuntu PPAs. Eventually, things will get stable enough that I'll upload to a PPA. Prior to that, I need a way to build a Debian package including source from a directory without an upstream tar ball. 3.0(git) is not a reasonable option because archive management programs have very little support for it, and because package download tools probably aren't well tested with it. I'm happy to entertain other options rather than 3.0(native) but my requirements are: * support for upstream version * support for debian revision * No need to have upstream sources available to dpkg-buildpackage prior to running it * No need to maintain .orig.tar.gz artifacts produced by dpkg-source and keep the checksums of these artifacts consistent between packages with the same upstream versions. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
Dimitri John Ledkov writes ("Re: dpkg-dev: please reject native/non-native
version when building native/non-native source packages"):
> Patch is attached to the new bug filed about this issue
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=737634
> Proposed patch adds "--force-native" dpkg-source option for the "3.0
> (Native)" format, which ignores enforcing native version number check.
Thanks for that. (Sorry, please ignore my previous mail.)
I think that this behaviour needs to be the default. Otherwise there
are packages in the archive that cannot be processed with the new
tool using the default options, which I think is a
non-backwards-compatible change.
Ian.
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#737634: dpkg-dev: please reject native/non-native version when building native/non-native source packages
On 4 February 2014 13:38, Jakub Wilk wrote: > * Dimitri John Ledkov , 2014-02-04, 13:30: > >> Enforcing Debian Policy at dpkg-source -b . level, is not a good idea, >> especially when it breaks backwards compat for 3rd parties. We have lintian, >> and ftp-master lintian auto-rejects to clense the archive if so is desired. > > > Hear, hear. And I even doubt there's consensus it is desired. > Patch is attached to the new bug filed about this issue http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=737634 Proposed patch adds "--force-native" dpkg-source option for the "3.0 (Native)" format, which ignores enforcing native version number check. -- Regards, Dimitri. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
