Re: Migration from setuptools to pyproject
Dear Louis-Philippe,
Thankd for your answer ... I will try to give some precision, inlined in the
text...
On Wed, 3 May 2023 11:13:56 -0400
Louis-Philippe Véronneau wrote:
> On 2023-05-03 04 h 06, Jerome Kieffer wrote:
> > Dear Debian Packager
> >
> > I am the upstream developer of a few debian packages (fabio, pyfai...) and
> > those packages used to rely on numpy.distutils. Since setuptools 60,
> > this is not more possible so I changed the build system to use mesonpy
> > (pep517 compliant).
> >
> > I found it easy to tell the debian packaging helper pybuild to use the
> > `pyproject.toml` file to build the software instead of the `setup.py`:
> > `export PYBUILD_SYSTEM=pyproject` in the debian/rules file.
> >
> > But later on, the meson configuration finds cython as cython3 but complains
> > the compiler was not found:
> >
> > ```
> > dh build --with python3,sphinxdoc --buildsystem=pybuild
> > dh_update_autotools_config -O--buildsystem=pybuild
> > dh_autoreconf -O--buildsystem=pybuild
> > dh_auto_configure -O--buildsystem=pybuild
> > pybuild --configure -i python{version} -p 3.11
> > dh_auto_build -O--buildsystem=pybuild
> > pybuild --build -i python{version} -p 3.11
> > I: pybuild plugin_pyproject:107: Building wheel for python3.11 with "build"
> > module
> > I: pybuild base:240: python3.11 -m build --skip-dependency-check
> > --no-isolation --wheel --outdir
> > /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.pybuild/cpython3_3.11_pyfai
> > * Building wheel...
> > + meson setup --prefix=/usr
> > /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0
> > /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-0rgp966m/build
> >
> > --native-file=/home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-native-file.ini
> > -Ddebug=false -Doptimization=2
> > The Meson build system
> > Version: 1.0.1
> > Source dir: /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0
> > Build dir:
> > /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-0rgp966m/build
> > Build type: native build
> > Project name: pyFAI
> > Project version: 2023.5.1a0
> > C compiler for the host machine: ccache cc (gcc 12.2.0 "cc (Debian
> > 12.2.0-14) 12.2.0")
> > C linker for the host machine: cc ld.bfd 2.40
> > Cython compiler for the host machine: cython3 (cython 0.29.32)
> > Host machine cpu family: x86_64
> > Host machine cpu: x86_64
> > Program cython found: NO
> >
> > ../../meson.build:17:0: ERROR: Program 'cython' not found or not executable
> >
> > ```
> >
> > I had a similar problem for python3 and it was solved by installing
> > `python-is-python3`. But there is no such trick for cython ...
> >
> > Nota: `sudo ln -s /usr/bin/cython3 /usr/bin/cython` addresses the bug but I
> > am looking for a solution which is compliant with debian packaging !
> >
> > Thanks for your help.
> >
> > Jerome
> >
>
> Hi,
>
> It's hard to help you without seeing what the /debian dir looks like.
Sorry, I forgot to put a link to the (ongoing) work:
https://github.com/silx-kit/pyFAI/pull/1867/files
This directory contains the debian files needed for packaging on debian
12/testing/unstable. Many things are still copy-paste from the former
build system (numpy-distutils)
>
> In theory, you don't need to specify `PYBUILD_SYSTEM=pyproject` for
> pybuild to use the pyproject.toml file.
Actually, there is still the former setup.py which is still used for
packaging under debian11- and also conda on macos. All other packaging
apparently support pyproject/meson-python.
> As per the pybuild manpage, you need to have the
> `pybuild-plugin-pyproject` package as a build-dependency and the
> package you use as the build system, in your case, `python3-mesonpy`.
Thanks for reminding me.
> As for your cython problem, it doesn't seem like you are building in
> a clean environment. I would highly recommend you use something like
> sbuild to do so:
>
> https://wiki.debian.org/sbuild
>
The computer was freshly installed with debian12 ... so no, it is not a
build in a chroot/docker but it is fairly clean. My question was rather
how to remove one level of abstraction to debug the code rather than
how to add one to make it cleaner.
Cheers,
Jerome
Migration from setuptools to pyproject
Dear Debian Packager
I am the upstream developer of a few debian packages (fabio, pyfai...) and
those packages used to rely on numpy.distutils. Since setuptools 60,
this is not more possible so I changed the build system to use mesonpy
(pep517 compliant).
I found it easy to tell the debian packaging helper pybuild to use the
`pyproject.toml` file to build the software instead of the `setup.py`:
`export PYBUILD_SYSTEM=pyproject` in the debian/rules file.
But later on, the meson configuration finds cython as cython3 but complains the
compiler was not found:
```
dh build --with python3,sphinxdoc --buildsystem=pybuild
dh_update_autotools_config -O--buildsystem=pybuild
dh_autoreconf -O--buildsystem=pybuild
dh_auto_configure -O--buildsystem=pybuild
pybuild --configure -i python{version} -p 3.11
dh_auto_build -O--buildsystem=pybuild
pybuild --build -i python{version} -p 3.11
I: pybuild plugin_pyproject:107: Building wheel for python3.11 with "build"
module
I: pybuild base:240: python3.11 -m build --skip-dependency-check --no-isolation
--wheel --outdir
/home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.pybuild/cpython3_3.11_pyfai
* Building wheel...
+ meson setup --prefix=/usr
/home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0
/home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-0rgp966m/build
--native-file=/home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-native-file.ini
-Ddebug=false -Doptimization=2
The Meson build system
Version: 1.0.1
Source dir: /home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0
Build dir:
/home/kieffer/workspace/pyFAI/build/debian12/pyFAI-2023.5.1a0/.mesonpy-0rgp966m/build
Build type: native build
Project name: pyFAI
Project version: 2023.5.1a0
C compiler for the host machine: ccache cc (gcc 12.2.0 "cc (Debian 12.2.0-14)
12.2.0")
C linker for the host machine: cc ld.bfd 2.40
Cython compiler for the host machine: cython3 (cython 0.29.32)
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program cython found: NO
../../meson.build:17:0: ERROR: Program 'cython' not found or not executable
```
I had a similar problem for python3 and it was solved by installing
`python-is-python3`. But there is no such trick for cython ...
Nota: `sudo ln -s /usr/bin/cython3 /usr/bin/cython` addresses the bug but I am
looking for a solution which is compliant with debian packaging !
Thanks for your help.
Jerome
Re: Helping hands needed to upgrade scipy
Hi Andreas, I am an upstream author of software packages (fabio & pyFAI) which took the same build system as scipy (meson-python) and I can confirm you there are issues with the debian packaging: * finding cython (apparently it should be fixed upstream in meson) * choose the proper python: sid has currently py3.10 and 3.11 co-installed and I found out the wrong one was used when building: https://github.com/mesonbuild/meson/issues/11248 I hope this helps, Jerome On Tue, 17 Jan 2023 18:12:42 +0100 Andreas Tille wrote: > Hi, > > I upgraded the experimental branch of scipy to contain the submodules > upstream includes. I'm *not* convinced that we need them all - > hopefully we can get rid of boost (but upstream has some patches here) > and scipy-mathjax. > > For the moment I've tried to build the experimental branch and ended > up with some issue where cython can't be found (should be cython3 for > sure)[1]. > > Unfortunately my spare time is occupied now by other more urgent tasks. > I'd be happy if someone else could review / pick up / continue from > here. > > Kind regards > Andreas. > > > [1] https://salsa.debian.org/python-team/packages/scipy/-/jobs/3809903 > > -- > http://fam-tille.de >
Re: build package xrayutilities - wheel and pip with setuptools
Hi Fred, Hi all other DD, I am the upstream author of pyFAI and fabio, I wish to thank you for taking care of the packaging of those software. I am aware there is an issue with the usage of those software since they use numpy.distutils (deprecated, to be removed in python3.12 and requires setuptools<60). I am working on a different packaging which uses meson-python (same as scipy>=1.9). The migration of fabio is more-or-less ready (see this branch: https://github.com/kif/fabio/tree/meson) and I am just waiting for a new release of meson-python (0.11) to make a new release (without setup.py). Unlike fabio which is ready for release, pyFAI will be released either in December or early next year. On the debian side, I don't know how far is the integration of meson-python ? https://salsa.debian.org/python-team/packages/meson-python To me, it is of crucial importance if one wants an updated version of scipy to be available in debian12. Fabio and pyFAI will follow without foreseeable issue. I am interested in the way the man-page should be produced since this will have to be redone in both those packages. Any advice on this is welcome. Best regards, -- Jérôme Kieffer tel +33 476 882 445
Re: Bug#1017959: RFP: meson-python -- Meson PEP 517 Python build backend
On Sat, 3 Sep 2022 19:08:42 +0100 Simon McVittie wrote: > https://salsa.debian.org/python-team/packages/meson-python > (not really tested yet, I don't yet have an upstream project that > needs it). Hi Simon, Scipy 1.9 is out and needs it before packaging. We are also waiting for it to migrate some other scientific packages to meson (fabio and pyFAI), since the numpy.distutils is deprecated in python 3.10. Cheers Jerome
Re: Need a Python 3.8 virtual environment
On Tue, 02 Mar 2021 21:17:36 -0600 Steven Robbins wrote: > Hi, > > I'm trying to use a (non-Debian) python system built on python 3.8. Debian's > default is currently 3.9 so I am advised to use a virtual environment. > Being > a newbie, I searched around and found a writeup covering several different > virtualization tools [1]. Note I am using Debian 'sid'. [...] > Is there something I've missed? Hi Steven, I don't think there is any simple solution for you within the debian ecosystem. As a developper, I faced the same issue and ended in installing `conda` to have a quick access to the various python versions (then I create some venv from the conda environments). I doubt our use-case is common enough to justify having multiple python (again) in Debian. Cheers, Jerome PS: venv are about 10x smaller than conda environments !
Re: Is PyPi down ?
Hi you all, I am trying to understand the python pachagin for debian, and discovered with this thread: https://pypi.debian.net/ Can you explain me (or better redirect me to the proper doc) on usage of this resource ? Last question, as the debian packaging is a moving target, is it something planned for the future, is it currently used or is it already deprecated ? Thanks a lot -- Jérôme Kieffer
Re: [SUSPECTED SPAM] Re: Python 3.7 or 3.6 in Buster
On Mon, 05 Nov 2018 12:26:41 -0500 Scott Kitterman wrote: > I only found out about it due to an upstream bug report on OS X. No one in > Debian (or Ubuntu) reported it. I got one also in fabio: https://github.com/silx-kit/fabio/pull/243 Actually there has been a depreciation warning for ages (back to python 3.4) but this has been silenced since then. There may be many such bugs around in many packages! Cheers, Jerome
Re: Advices on how to use salsa
On Sat, 20 Oct 2018 17:21:13 -0400 Sergio Durigan Junior wrote: > On Friday, October 19 2018, Jerome Kieffer wrote: > > > On Fri, 19 Oct 2018 15:48:13 -0400 > > Sergio Durigan Junior wrote: > > > >> Is it? I always thought it was OK to host Free Software projects there. > > > > No matter, I have github, gitlab for my code ... I intend to use salsa > > for packaging purposes. First and formost, I would like to thank Andrey Rahmatullin for answering my question, which was probably wrongly worded (explining why I did not find the resources I was looking for). > > If I may: please consider not using github. It uses proprietary > software for its backend infrastructure, and serves proprietary > JavaScript to its users, as well as promote centralization of a > distributed protocol. About 10 years ago, I heard "gitub made git useable for human being". At that time mercurial, bazar and git were equally used and it was a mess to chose one of them and then select a workflow to work with. The work github did (their tutorials !) deserve recognition to simplify all this. Please remind gitlab (which salsa is just an implementation) was originally just a re-implementation github hosted initially on github ! Maybe this company is private and has been acquired by another you don't like. Cheer, Jérôme pgpiIbxV5g0cz.pgp Description: Signature digitale OpenPGP
Re: Advices on how to use salsa
On Fri, 19 Oct 2018 15:48:13 -0400 Sergio Durigan Junior wrote: > Is it? I always thought it was OK to host Free Software projects there. No matter, I have github, gitlab for my code ... I intend to use salsa for packaging purposes. Cheers, Jerome pgpW6oGDUBoes.pgp Description: Signature digitale OpenPGP
Re: Advices on how to use salsa
On Sat, 20 Oct 2018 00:33:48 +0500 Andrey Rahmatullin wrote: > Salsa is only for official Debian packages. Are you going to update > python-rfoo in Debian? Well, I created the version which is in debian for a few years and yes, I would like to update it (with migration to python3, ...). Thanks for your advices. Jérôme PS: I am not a DD, just a guy who packages the code he needs and hope it is usfull for other. PS2: I am not the upstream dev of rfoo neither. pgpcnoN1j6HsF.pgp Description: Signature digitale OpenPGP
Advices on how to use salsa
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi all, I am a python developer and from time-to-time I also build some packages. I recently re-built python-rfoo for the latest version and I was suggest to make the packaging on salsa. Does the Debian-Python community have recommendation for building packages when using git and the gitlab-forge on salsa ? Thanks for your help, Jérôme -BEGIN PGP SIGNATURE- iQIzBAEBCAAdFiEEtt2uSu/8Nysc0g1Hnqh6TLjIznwFAlvKLNoACgkQnqh6TLjI znxDWg//X7MTsPvze9Cp63f11zAz7G+4r187AJpjIQge3OEI0kmNgrmJMYMdreUK 1Aa3XrRBckxFH1T234jhX9zMEW8tjkbc1B6Xi9Jdd84UOTpTaLYQMA0TGGToN+Ry pFVmspj1wUgmQRQacMxCTBARSzmTGjoobxdO514JZL/ZGptJRkZ1Du0Sq25pcjOR 42M/PDH01hloqENy9UR8ryDvFwKkf9P7g2aWMANQN8sFWMcFSWO+EGuXliLw57oQ jDu1dxX5kpoKTVX9+rvuUEGQaPMpwx3G809qMXwmRBgNVewX8BMrGKWuwBI3IZcX EUb8azmqvz6QSrh7SYEdGKzjQd9wpVQUIGDOqJghjcgsJbhvFo6SA/EPe8UkCxdU um70FYYA2zxf0RJsiqgJiCsPNVJDmiLt1prnzRZo4vSfI6KdF2cS8G6sVuqdadY0 Vh/XYQ+BKVBVVno25xiX3m3y0LEIFbb8OM2mtTW1OgZidT2yGYD3fWJGb60hBkMu h3QL5G62HhKU8Ftq1j6whT+BL56JRKvmIGLHrjsniK7v2fLhJVQxqftyWYRHhVq5 10/75tLrwk3+9U4rCbcZAC5FyyKs+ffLWnBgPUGwE2hnGNx2813byq8cqotp13mq JkJxgRj5e58dkzl3ekQsAYlHf/VMyo3IrfsVp63OUcWABIyXqV4= =zQ7g -END PGP SIGNATURE-
