Re: Debian choice of upstream tarballs for packaging

2021-09-26 Thread gregor herrmann
On Mon, 16 Aug 2021 09:18:34 +0800, Paul Wise wrote: > I noticed that sometimes Debian's choice of upstream source for > packaging can be suboptimal. This is especially apparent for the > different per-language upstream packaging ecosystems[1], where the > upstream packaging differs from the

Re: Re: Debian choice of upstream tarballs for packaging

2021-08-29 Thread Nicholas Guriev
> git archive --prefix=$(DEBPKGNAME)-$(VERSION)/ $(GIT_TAG) \ > | xz >../$(DEBPKGNAME)_$(VERSION).orig.tar.xz I think you should add +ds version suffix or similar to indicate repacking for Debian. Does it still make sense provided that upstream does not care much of tarballs?

Re: Debian choice of upstream tarballs for packaging

2021-08-29 Thread Simon Richter
Hi, On 27.08.21 22:56, Sean Whitton wrote: That is ecosystem dependent. Yes, that was my point. We're going to have upstreams who release tarballs and upstreams who release tags for some time. My expectation for that state would be "indefinitely", and I don't see that as a bad thing, we

Re: Debian choice of upstream tarballs for packaging

2021-08-27 Thread Sean Whitton
Hello, On Wed 25 Aug 2021 at 07:01PM +02, Thomas Goirand wrote: > If pushing the upstream git tags to Salsa, you're safe, and the way we > do in the OpenStack team, we still generate and upload tarballs to the > Debian archive matching each tags. Branches on salsa can be force-pushed, so it's

Re: Debian choice of upstream tarballs for packaging

2021-08-27 Thread Sean Whitton
Hello, On Wed 25 Aug 2021 at 12:00PM +02, Simon Richter wrote: > Hi, > > On 8/25/21 1:21 AM, Sean Whitton wrote: > >> From my point of view, signing git tags is no less well established a >> best practice than signing tarballs -- in fact, to me, it seems *more* >> well established. > > That is

Re: Debian choice of upstream tarballs for packaging

2021-08-27 Thread Sean Whitton
Hello zigo, On Wed 25 Aug 2021 at 04:11PM +02, Thomas Goirand wrote: > I wrote this many times, but I don't see why we should use any "upstream > tarball" when the Git repository itself contains the tarball with: > > git archive --prefix=$(DEBPKGNAME)-$(VERSION)/ $(GIT_TAG) \ > | xz

Re: Debian choice of upstream tarballs for packaging

2021-08-27 Thread Sean Whitton
Hello, On Fri 27 Aug 2021 at 07:58AM GMT, Paul Wise wrote: > On Wed, Aug 25, 2021 at 2:36 PM Simon Richter wrote: > >> "git archive" is reproducible > > I'm told by the Bootstrappable Builds folks that `git archive` isn't > deterministic in some cases to do with filtering, I lost the details >

Re: Debian choice of upstream tarballs for packaging

2021-08-27 Thread Paul Wise
On Wed, Aug 25, 2021 at 2:36 PM Simon Richter wrote: > "git archive" is reproducible I'm told by the Bootstrappable Builds folks that `git archive` isn't deterministic in some cases to do with filtering, I lost the details though. -- bye, pabs https://wiki.debian.org/PaulWise

Re: Debian choice of upstream tarballs for packaging

2021-08-27 Thread Paul Wise
On Wed, Aug 25, 2021 at 10:01 AM Simon Richter wrote: > FWIW, I'd love to see git bundles as a source archive format -- this > would allow shipping a (signed) tag, its commit, and the tree and blob > objects for that commit as a single file that can be built in a > reproducible way and allows

Re: Debian choice of upstream tarballs for packaging

2021-08-26 Thread Simon Richter
Hi Thomas, On 8/26/21 4:16 PM, Thomas Goirand wrote: "git archive" is reproducible, for simplicity I wouldn't use a prefix though. xz has some issues with reproducibility, AFAIK "-T2" makes it disable some internal heuristics that are based on the machine it is running on, and generates

Re: Debian choice of upstream tarballs for packaging

2021-08-26 Thread Thomas Goirand
On 8/25/21 4:35 PM, Simon Richter wrote: > Hi, > >> I wrote this many times, but I don't see why we should use any "upstream >> tarball" when the Git repository itself contains the tarball with: > >> git archive --prefix=$(DEBPKGNAME)-$(VERSION)/ $(GIT_TAG) \ >> | xz

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Brian Thompson
On 0825, Simon Richter wrote: >Hi, > >On 8/25/21 1:21 AM, Sean Whitton wrote: > >> From my point of view, signing git tags is no less well established a >>best practice than signing tarballs -- in fact, to me, it seems *more* >>well established. > >That is ecosystem dependent. > >FWIW, I'd love to

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Sam Hartman
> "Simon" == Simon Richter writes: Simon> Hi, Simon> On 8/16/21 3:18 AM, Paul Wise wrote: >> I'd like to suggest that we standardise on the upstream VCS for >> our orig.tar.gz files and phase out use of upstream packaging >> ecosystems. Simon> This is also an

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Simon Richter
Hi, On 25.08.21 18:42, Phil Morrell wrote: "git archive" is reproducible, for simplicity I wouldn't use a prefix though. For simplicity I *would* use a prefix, purely because that's what github/gitlab uses, so upstream can still choose to additionally sign the distributed tarball if they

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Thomas Goirand
On 8/25/21 5:01 PM, Theodore Ts'o wrote: > On Wed, Aug 25, 2021 at 04:11:37PM +0200, Thomas Goirand wrote: >> >> It's been *years* since I encounter a PyPi package that doesn't have a >> Git repo as its homepage (and unfortunately, 99% on Github). >> >> I wrote this many times, but I don't see why

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Phil Morrell
On Wed, Aug 25, 2021 at 04:35:51PM +0200, Simon Richter wrote: > > I wrote this many times, but I don't see why we should use any "upstream > > tarball" when the Git repository itself contains the tarball with: > > > git archive --prefix=$(DEBPKGNAME)-$(VERSION)/ $(GIT_TAG) \ > > | xz

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Theodore Ts'o
On Wed, Aug 25, 2021 at 04:11:37PM +0200, Thomas Goirand wrote: > > It's been *years* since I encounter a PyPi package that doesn't have a > Git repo as its homepage (and unfortunately, 99% on Github). > > I wrote this many times, but I don't see why we should use any "upstream > tarball" when

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Jeremy Stanley
On 2021-08-25 16:11:37 +0200 (+0200), Thomas Goirand wrote: [...] > I wrote this many times, but I don't see why we should use any "upstream > tarball" when the Git repository itself contains the tarball with: > > git archive --prefix=$(DEBPKGNAME)-$(VERSION)/ $(GIT_TAG) \ > | xz

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Simon Richter
Hi, I wrote this many times, but I don't see why we should use any "upstream tarball" when the Git repository itself contains the tarball with: git archive --prefix=$(DEBPKGNAME)-$(VERSION)/ $(GIT_TAG) \ | xz >../$(DEBPKGNAME)_$(VERSION).orig.tar.xz (which leads to a .xz, which is

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Thomas Goirand
On 8/16/21 3:18 AM, Paul Wise wrote: > Hi all, > > I noticed that sometimes Debian's choice of upstream source for > packaging can be suboptimal. This is especially apparent for the > different per-language upstream packaging ecosystems[1], where the > upstream packaging differs from the upstream

Re: Debian choice of upstream tarballs for packaging

2021-08-25 Thread Simon Richter
Hi, On 8/25/21 1:21 AM, Sean Whitton wrote: From my point of view, signing git tags is no less well established a best practice than signing tarballs -- in fact, to me, it seems *more* well established. That is ecosystem dependent. FWIW, I'd love to see git bundles as a source archive

Re: Debian choice of upstream tarballs for packaging

2021-08-24 Thread Phil Morrell
On Tue, Aug 24, 2021 at 04:21:50PM -0700, Sean Whitton wrote: > On Wed 18 Aug 2021 at 10:10AM +02, Simon Josefsson wrote: > > Signing tarballs is the current > > established best practice -- moving to VCS builds needs a set of new > > schemes to be established and deployed, and I don't see any

Re: Debian choice of upstream tarballs for packaging

2021-08-24 Thread Sean Whitton
Hello Simon, On Wed 18 Aug 2021 at 10:10AM +02, Simon Josefsson wrote: > 1) Trust paths. Some upstreams sign release tarballs with an OpenPGP > release key that Debian trust for making releases. Not all upstream > uses the same key to sign VCS tags/commits, and not all upstreams sign > VCS

Re: Debian choice of upstream tarballs for packaging

2021-08-19 Thread Jonas Smedegaard
Quoting Marc Haber (2021-08-19 09:12:27) > On Tue, 17 Aug 2021 19:16:05 +0200, Jonas Smedegaard > wrote: > >Quoting Marc Haber (2021-08-17 18:56:59) > >> There is, for example, one distribution that is based on Ubuntu > >> (maybe they thought that ubuntu would be too hard to install) and > >>

Re: Debian choice of upstream tarballs for packaging

2021-08-19 Thread Marc Haber
On Tue, 17 Aug 2021 19:16:05 +0200, Jonas Smedegaard wrote: >Quoting Marc Haber (2021-08-17 18:56:59) >> There is, for example, one distribution that is based on Ubuntu (maybe >> they thought that ubuntu would be too hard to install) and does not >> support upgrades. Their FAQ says "we cannot

Re: Debian choice of upstream tarballs for packaging

2021-08-18 Thread Paul Wise
On Wed, Aug 18, 2021 at 8:10 AM Simon Josefsson wrote: > 1) Trust paths. Agreed, this is the main exception I mentioned when starting this thread. > 2a) Gnulib. Presumably upstream could be convinced to encode this information into the VCS, perhaps into the standard autogen script that is

Re: Debian choice of upstream tarballs for packaging

2021-08-18 Thread Simon Richter
Hi, On 8/18/21 5:04 AM, Paul Wise wrote: This is also an additional burden on package maintainers: explaining how they arrived at that particular "upstream" package in a reproducible way Debian explaining how we arrived at a particular orig.tar.gz is well established; use a debian/watch

Re: Debian choice of upstream tarballs for packaging

2021-08-18 Thread Simon Josefsson
Paul Wise writes: > Hi all, > > I noticed that sometimes Debian's choice of upstream source for > packaging can be suboptimal. This is especially apparent for the > different per-language upstream packaging ecosystems[1], where the > upstream packaging differs from the upstream VCS in some

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Paul Wise
On Tue, Aug 17, 2021 at 12:17 PM Simon Richter wrote: > This is also an additional burden on package maintainers: explaining how > they arrived at that particular "upstream" package in a reproducible way Debian explaining how we arrived at a particular orig.tar.gz is well established; use a

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Sune Vuorela
On 2021-08-16, Paul Wise wrote: > While discussing PyPI with the Python team, it was pointed out that > sometimes the tarball contains things that cannot be regenerated from > just the VCS snapshot, such as information stored in the VCS history, > so perhaps the recommendation should be to prefer

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Jonas Smedegaard
Quoting Marc Haber (2021-08-17 18:56:59) > On Tue, 17 Aug 2021 15:52:27 +0200, Jonas Smedegaard > wrote: > >Quoting Simon Richter (2021-08-17 14:17:05) > >> Rather than accept defeat, I'd like Debian to push upstreams more > >> aggressively for higher quality releases, and also to make > >>

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Marc Haber
On Tue, 17 Aug 2021 15:52:27 +0200, Jonas Smedegaard wrote: >Quoting Simon Richter (2021-08-17 14:17:05) >> Rather than accept defeat, I'd like Debian to push upstreams more >> aggressively for higher quality releases, and also to make judgement >> calls on whether a particular package is even

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Antonio Terceiro
On Tue, Aug 17, 2021 at 06:51:35AM +, Paul Wise wrote: > On Mon, Aug 16, 2021 at 1:19 AM Paul Wise wrote: > > > 1. the ecosystems I'm talking about include cargo, npm, browser > > extensions, rubygems, pypi, CPAN etc. > > Examples of what current Debian practices are for these ecosystems:

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Jonas Smedegaard
Quoting Simon Richter (2021-08-17 14:17:05) > Rather than accept defeat, I'd like Debian to push upstreams more > aggressively for higher quality releases, and also to make judgement > calls on whether a particular package is even suitable for a stable > release instead of assuming that by

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Simon Richter
Hi, On 8/16/21 3:18 AM, Paul Wise wrote: I'd like to suggest that we standardise on the upstream VCS for our orig.tar.gz files and phase out use of upstream packaging ecosystems. This is also an additional burden on package maintainers: explaining how they arrived at that particular

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Luca Boccassi
On Tue, 2021-08-17 at 14:07 +0530, Pirate Praveen wrote: > > 2021, ഓഗസ്റ്റ് 17 12:18:00 PM IST, Paul Wise ൽ എഴുതി > > On Mon, Aug 16, 2021 at 8:25 PM Pirate Praveen wrote: > > > > > Many node modules don't tag their releases so its really hard to get > > > exact source code corresponding to an

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Pirate Praveen
2021, ഓഗസ്റ്റ് 17 12:18:00 PM IST, Paul Wise ൽ എഴുതി >On Mon, Aug 16, 2021 at 8:25 PM Pirate Praveen wrote: > >> Many node modules don't tag their releases so its really hard to get >> exact source code corresponding to an npmjs.com release. > >It is probably worth filing upstream issues when

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Paul Wise
On Mon, Aug 16, 2021 at 1:19 AM Paul Wise wrote: > 1. the ecosystems I'm talking about include cargo, npm, browser > extensions, rubygems, pypi, CPAN etc. Examples of what current Debian practices are for these ecosystems: (Amost?) all rust-* packages come from crates.io. Many/most browser

Re: Debian choice of upstream tarballs for packaging

2021-08-17 Thread Paul Wise
On Mon, Aug 16, 2021 at 8:25 PM Pirate Praveen wrote: > Many node modules don't tag their releases so its really hard to get > exact source code corresponding to an npmjs.com release. It is probably worth filing upstream issues when you discover that. > Also with mono repos becoming more

Re: Debian choice of upstream tarballs for packaging

2021-08-16 Thread Pirate Praveen
On 17/08/21 1:43 am, Sean Whitton wrote: > I agree with this, and already do it for all or almost all of the > packages I maintain. There will probably need to be lots of exceptions, > however. Many node modules don't tag their releases so its really hard to get exact source code

Re: Debian choice of upstream tarballs for packaging

2021-08-16 Thread Sean Whitton
Hello, On Mon 16 Aug 2021 at 09:18AM +08, Paul Wise wrote: > I noticed that sometimes Debian's choice of upstream source for > packaging can be suboptimal. This is especially apparent for the > different per-language upstream packaging ecosystems[1], where the > upstream packaging differs from

Re: Debian choice of upstream tarballs for packaging

2021-08-16 Thread Sam Hartman
> "Paul" == Paul Wise writes: Paul> Hi all, I noticed that sometimes Debian's choice of upstream Paul> source for packaging can be suboptimal. This is especially Paul> apparent for the different per-language upstream packaging Paul> ecosystems[1], where the upstream packaging

Debian choice of upstream tarballs for packaging

2021-08-15 Thread Paul Wise
Hi all, I noticed that sometimes Debian's choice of upstream source for packaging can be suboptimal. This is especially apparent for the different per-language upstream packaging ecosystems[1], where the upstream packaging differs from the upstream VCS in some significant ways, including missing