Re: PDM - Python package manager for Debian

[Stefano Rivera]

Hi Dominik (2022.04.03_09:16:14_+)
> > I've compared the tarball from GH with the file from PyPi, the sdist on 
> > PyPi contains even less files than the GH tarball, but also no setup.* 
> > files.
> 
> Uh, PEP-517 actually allows that... I have never seen that in the wild.

It's becoming quite common now. Anything that doesn't use setuptools
won't have setup.* files. And now that there are options besides
setuptools, people are switching to them.

> Cool, so this really means we will ultimately have to package
> everybody's homegrown build system now .

Sensible build systems divide the frontend (CLI and developer tools)
from the backend (PEP-517 API), so that only the latter is needed at
build time (and in Debian).  But yes, I expect a bit of an explosion in
PEP-517 backends in the archive.  And none of them are particularly
stable, yet. Lots of churn.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  +1 415 683 3272

Re: PDM - Python package manager for Debian

[Dominik George]

Hi,


> I've compared the tarball from GH with the file from PyPi, the sdist on PyPi 
> contains even less files than the GH tarball, but also no setup.* files.

Uh, PEP-517 actually allows that... I have never seen that in the wild.

Cool, so this really means we will ultimately have to package everybody's 
homegrown build system now .

-nik

Re: PDM - Python package manager for Debian

[Carsten Schoenert]

Hi Nick,

Am 03.04.22 um 09:58 schrieb Dominik George:

Hi Carsten,


Or is there a way to get such packages build without a need for PDM to be 
around?


This should really not matter at all when packaging for Debian. The
source tarball should include a setup.cfg or setup.py file (i.e. be a
regular Python sdist), and if not developing on the package, you
should never meet PDM.


yeah, but upstream isn't shipping one or both files, it only ships a 
file pyproject.toml. And switched from Poetry to PDM recently


https://github.com/mkdocstrings/autorefs/commit/85f3920ec153e6aa3c206475977b8666ac340504


If the upstream sdist is not installable without PDM, this is
probably an upstream bug; but my guess is that you chose a Git export
instead of a real sdist as orig.tar.gz.


No, I did pick up the tagged version 0.4.1 as tarball from GitHub as 
usual to start with as I'm aware of this trap.


https://github.com/mkdocstrings/autorefs/tags

I've compared the tarball from GH with the file from PyPi, the sdist on 
PyPi contains even less files than the GH tarball, but also no setup.* 
files.


https://pypi.org/project/mkdocs-autorefs/#files


If helpful I can upload my current WIP scratch status to my namespace on 
Salsa.


So far I've read about PDM the past hours, it's quite new yes, but more 
upstream projects will switch or start with this package management 
system I guess. So somehow the Debian packaging of Python libraries will 
need to deal with this.


--
Regards
Carsten

Re: PDM - Python package manager for Debian

[Dominik George]

Hi Carsten,

> Or is there a way to get such packages build without a need for PDM to be 
> around?

This should really not matter at all when packaging for Debian. The source 
tarball should include a setup.cfg or setup.py file (i.e. be a regular Python 
sdist), and if not developing on the package, you should never meet PDM.

If the upstream sdist is not installable without PDM, this is probably an 
upstream bug; but my guess is that you chose a Git export instead of a real 
sdist as orig.tar.gz.

-nik

PDM - Python package manager for Debian

[Carsten Schoenert]

Hi,

while working further on doing packaging on NetBox I've started trying 
to package more upcoming new dependencies.


Namely I was looking into mkdocstrings [1], which is a new dep for the 
next minor version of NetBox (in order to build the documentation).


mkdocstrings comes (of course) with own new package build deps [2] on 
mkdocs-autorefs [3] and mkdocstrings-python-legacy [4].

pymdown-extensions is something Sandro Tosi was already working on.

To get the additional build dependencies packaged first I started on 
mkdocs-autorefs.

While doing this the package build breaks due missing PDM support.


dh binary --with python3 --buildsystem=pybuild
   dh_update_autotools_config -O--buildsystem=pybuild
   dh_autoreconf -O--buildsystem=pybuild
   dh_auto_configure -O--buildsystem=pybuild
   dh_auto_build -O--buildsystem=pybuild
I: pybuild plugin_pyproject:107: Building wheel for python3.9 with "build" 
module
I: pybuild base:237: python3.9 -m build --skip-dependency-check --no-isolation --wheel --outdir /build/mkdocs-autorefs-0.4.1/.pybuild/cpython3_3.9_autorefs 
* Building wheel...


Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pep517/wrappers.py", line 332, in 
_call_hook
raise BackendUnavailable(data.get('traceback', ''))
pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pep517/in_process/_in_process.py", line 
89, in _build_backend
obj = import_module(mod_path)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "", line 1030, in _gcd_import
  File "", line 1007, in _find_and_load
  File "", line 972, in _find_and_load_unlocked
  File "", line 228, in _call_with_frames_removed
  File "", line 1030, in _gcd_import
  File "", line 1007, in _find_and_load
  File "", line 972, in _find_and_load_unlocked
  File "", line 228, in _call_with_frames_removed
  File "", line 1030, in _gcd_import
  File "", line 1007, in _find_and_load
  File "", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pdm'

ERROR Backend 'pdm.pep517.api' is not available.
E: pybuild pybuild:367: build: plugin pyproject failed with: exit code=1: python3.9 -m build --skip-dependency-check --no-isolation --wheel --outdir /build/mkdocs-autorefs-0.4.1/.pybuild/cpython3_3.9_autorefs 
I: pybuild plugin_pyproject:107: Building wheel for python3.10 with "build" module
I: pybuild base:237: python3.10 -m build --skip-dependency-check --no-isolation --wheel --outdir /build/mkdocs-autorefs-0.4.1/.pybuild/cpython3_3.10_autorefs 
* Building wheel...


Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pep517/wrappers.py", line 332, in 
_call_hook
raise BackendUnavailable(data.get('traceback', ''))
pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pep517/in_process/_in_process.py", line 
89, in _build_backend
obj = import_module(mod_path)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "", line 1050, in _gcd_import
  File "", line 1027, in _find_and_load
  File "", line 992, in _find_and_load_unlocked
  File "", line 241, in _call_with_frames_removed
  File "", line 1050, in _gcd_import
  File "", line 1027, in _find_and_load
  File "", line 992, in _find_and_load_unlocked
  File "", line 241, in _call_with_frames_removed
  File "", line 1050, in _gcd_import
  File "", line 1027, in _find_and_load
  File "", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pdm'

ERROR Backend 'pdm.pep517.api' is not available.
E: pybuild pybuild:367: build: plugin pyproject failed with: exit code=1: python3.10 -m build --skip-dependency-check --no-isolation --wheel --outdir /build/mkdocs-autorefs-0.4.1/.pybuild/cpython3_3.10_autorefs 
dh_auto_build: error: pybuild --build -i python{version} -p "3.9 3.10" returned exit code 13

make: *** [debian/rules:10: binary] Error 13
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
I: copying local configuration
E: Failed autobuilding of package


I've tried to find out if we have in Debian PDM already included but 
haven't found any Python related packages. But my knowledge about core 
package build backends is really poor. And doing a packaging of such an 
module is for sure to much I could do.


Anyone here on the list who can shad some light about PDM [5] or is 
someone already working on this?
Or is there a way to get such packages build without a need for PDM to 
be around?
Sorry if this is a stupid questions, but I'm really not in the deep 
interns of Python package build systems.



[1] https://github.com/mkdocstrings/mkdocstrings
[2] 
https://github.com/mkdocstrings/mkdocstrings/blob/master/pyproject.toml#L37

[3] https://github.com/mkdocstrings/autorefs