Re: [RFC] DPT Policy: Canonise recommendation against PyPi-provided upstream source tarballs
On 2021-06-26 18:51:26 -0400 (-0400), Louis-Philippe Véronneau wrote: [...] > To me, the most important thing is that all packages must at least > run the upstream testsuite when it exists (I'm planning on writing > a policy proposal saying this after the freeze). If PyPi releases > include them, I think it's fine (but they often don't). When you do write that, you'll of course want to clarify what "the upstream testsuite" really means too. Lots of projects have vast testing which is simply not feasible to replicate within Debian for a number of reasons. Running some battery of upstream tests makes sense, but testsuites which require root access outside a chroot, integration tests orchestrated across multiple machines, access to unusual sorts of accelerator or network hardware, and so on can easily comprise part of "the upstream testsuite." -- Jeremy Stanley signature.asc Description: PGP signature
Re: [RFC] DPT Policy: Canonise recommendation against PyPi-provided upstream source tarballs
On 2021-06-25 16 h 42, Nicholas D Steeves wrote: > Hi Team! > > I feel like there is probably consensus against the use of PyPi-provided > upstream source tarballs in preference for what will usually be a GitHub > release tarball, so I made an MR to this effect (moderate recommendation > rather than a "must" directive): > > > https://salsa.debian.org/python-team/tools/python-modules/-/merge_requests/16 I don't often use PyPi releases because of the issues mentioned in the MR, but I think Jeremy's point is valid. IMO, rewording the text so that it clearly says "should" and not "must" would fix the issues at hand, as long as people justify their usage of PyPi when it's "The Right Thing" in a file somewhere. To me, the most important thing is that all packages must at least run the upstream testsuite when it exists (I'm planning on writing a policy proposal saying this after the freeze). If PyPi releases include them, I think it's fine (but they often don't). -- ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ Louis-Philippe Véronneau ⢿⡄⠘⠷⠚⠋ po...@debian.org / veronneau.org ⠈⠳⣄ OpenPGP_signature Description: OpenPGP digital signature
Re: [RFC] DPT Policy: Canonise recommendation against PyPi-provided upstream source tarballs
On Fri, 25 Jun 2021 at 18:29:19 -0400, Nicholas D Steeves wrote:
> Take for example the
> case where upstream exclusively supports a Flatpak and/or Snap
> package...
Flatpak and Snap aren't source package formats (like Autotools "make dist"
or Meson "meson dist" or Python sdist), they're binary package formats
(like .deb or Python wheels).
I don't know Snap infrastructure well, but Flatpak apps are built from
a manifest that lists one or more source projects, referenced as either
a VCS commit with a known-good commit identifier (usually git) or an
archive with a known-good hash (usually tar and sha256). The manifest
format and the upstream-recommended Flathub "app store" infrastructure
try to push authors towards building from source, although as with
.deb, technically it's possible to release an archive containing binary
blobs and use it as the "source" (which is how proprietary apps like
com.valvesoftware.SteamLink work, similar to many packages in the non-free
archive area).
If the upstream only provides source via their VCS, then obviously we
have to use `git archive` or equivalent because we have no other way to
get a flat-file version, and the experimental dpkg-source format
"3.0 (git)" isn't currently allowed in the Debian archive. If the upstream
releases tarball artifacts and builds their Flatpak app from those, we can
use those too.
I think the problem case here is when the upstream releases something that
has the name and format we would associate with a source release, but
has contents that are somewhere between a pure source release and a binary
release. Autotools "make dist" has always been a bit like this (it contains
a pre-generated build system so that people can build on platforms where
m4 and perl aren't available, and it's common to include pre-generated
convenience copies of things like gtk-doc documentation); Python sdist
archives are sometimes similar. In both Autotools and setuptools, it's
also far too easy to have files in the VCS but accidentally omit them from
the source distribution, by not listing them in Autotools EXTRA_DIST or in
setuptools MANIFEST.in.
What I have generally done to resolve this problem is to use the upstream's
official source releases ("make dist" or sdist), and if they are missing
files that we want, send merge requests to add them to the next release
(for example https://gitlab.gnome.org/GNOME/gi-docgen/-/commit/5fcaba6f
and https://github.com/containers/bubblewrap/commit/1c775f43),
and if necessary work around missing files by shipping them in debian/
(for example https://salsa.debian.org/gnome-team/gi-docgen/-/commit/f16845d9).
Several upstreams of projects I work on, notably GNOME, have been
switching from Autotools to Meson, and one of the reasons I'm in favour
of this tendency is that the Meson "meson dist" archive is a lightly
filtered version of `git archive` (it excludes `.gitignore` and other
highly git-specific files, but includes everything else), making it
harder for upstreams to accidentally omit necessary source code from
their source releases.
smcv
Re: [RFC] DPT Policy: Canonise recommendation against PyPi-provided upstream source tarballs
On 2021-06-26 02:04, Paul Wise wrote: > I would like to see #2 split into two separate tarballs, one for the > exact copy of the git tree and one containing the data about the other > tarball. Then use dpkg-source v3 secondary tarballs to add the data > about the git repo to the Debian source package. IIRC, last time I tried multiple tarballs, I got stuck with pristine-tar. Not sure, if I didn't find out how to commit or if the problem was with checkout, though. Do you happen to know, if this is an issue? PS: Just for the record: I'm always(?) using upstream sources from git, not PyPi, because the latter typically are missing unit tests, which we want to run in Debian.
