Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
Le lundi, 30 janvier 2023, 18.59:04 h CET Andreas Beckmann a écrit :
> On 28/01/2023 18.09, Didier 'OdyX' Raboud wrote:
> > So. I have a much shorter (and better) patch, that I'll upload to
> > DELAYED/5 in few minutes.
>
> Thanks alot.
No problem.
Should I push it to unstable?
> There is now a new lintian error (not caused by your patch,
> but by some changed B-D). Since you were looking into
> the python building bits, perhaps you have an idea where this
> is originating from? Notice the difference lib vs. lib64
The code in setup.py does non-Debian stuff around guessing library paths.
A possible patch is attached, I've also pushed it on a branch:
https://salsa.debian.org/science-team/pyhst2/-/merge_requests/2
I have no idea if the resulting package works though.
Best,
OdyXDrop all CUDA-specific library path search
Fixes the superfluous /usr/lib or /usr/lib64 paths in RUNPATH
Author: Didier Raboud
Origin: vendor
--- a/setup.py
+++ b/setup.py
@@ -261,15 +261,6 @@ def do_pyhst():
cudaconfig = {'home':home, 'nvcc':nvcc,
'include': pjoin(home, 'include')}
-lib = pjoin(home, 'lib')
-if os.path.exists(lib+"64") andmath.log(sys.maxsize)/math.log(2) > 60: ##sys.maxint == 2**63-1:
-cudaconfig["lib"] = lib+"64"
-elif os.path.exists(lib+"32") and math.log(sys.maxsize)/math.log(2) < 60:
-cudaconfig["lib"] = lib+"32"
-elif os.path.exists(lib):
-cudaconfig["lib"] = lib
-else:
-print(("No cuda library found "))
for key, val in cudaconfig.items():
if not os.path.exists(val):
raise EnvironmentError('The CUDA %s path could not be located in %s' % (key, val))
@@ -469,9 +460,7 @@ def do_pyhst():
module = Extension('PyHST2_'+version+'/libgputomo',
sources=sources,
-library_dirs=[CUDA['lib']],
libraries=['cudart', "cublas", "cuda", "cufft", hdf5_lib],
-runtime_library_dirs=[CUDA['lib']],
# this syntax is specific to this build system
# we're only going to use certain compiler args with nvcc and not with gcc
# the implementation of this trick is in customize_compiler() below
@@ -495,11 +484,8 @@ def do_pyhst():
module = Extension(name='PyHST2_'+version+'.unsharp3D',
sources=sources ,
depends=[],
- library_dirs=[CUDA['lib']],
libraries=['cudart','QtGui','QtCore','cudart','tiff'],
- runtime_library_dirs=[CUDA['lib']],
-
extra_compile_args={'gcc': ["-std=c99","-g","-fPIC", "-O3","-Wall"],
'nvcc': CUDA["arch"] + [ "--compiler-options", "-fPIC", "-O3", "-g","-D_FORCE_INLINES" ]},
include_dirs=[numpy.get_include(), CUDA['include'], 'PyHST/Cspace',"/usr/include/qt4"] )
@@ -518,9 +504,7 @@ def do_pyhst():
global version
module = Extension('PyHST2_'+version+'/libprojection',
sources=["PyHST/Cspace/c_hst_project_1over.cu"],
- library_dirs=[CUDA['lib']],
libraries=['cudart', "cuda", "cufft"],
- runtime_library_dirs=[CUDA['lib']],
extra_compile_args={'gcc': ["-fPIC", "-O3"],
'nvcc': CUDA["arch"] + ["--compiler-options", "-fPIC,-O3" ,"-D_FORCE_INLINES"]},
include_dirs=[numpy.get_include(), CUDA['include'], 'PyHST/Cspace']+ hdf5_dirs)
@@ -542,9 +526,7 @@ def do_pyhst():
module = Extension('PyHST2_'+version+'/libwavelets',
sources=sources,
- library_dirs=[CUDA['lib']],
libraries=["cudart", "cuda", "cublas"],
- runtime_library_dirs=[CUDA['lib']],
extra_compile_args={'gcc': ["-fPIC", "-O3"],
'nvcc': CUDA["arch"] + ["--compiler-options", "-fPIC,-O3" ,"-D_FORCE_INLINES"]},
include_dirs=[numpy.get_include(), CUDA['include'], 'PyHST/Cspace'])
signature.asc
Description: This is a digitally signed message part.
Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
On 28/01/2023 18.09, Didier 'OdyX' Raboud wrote: So. I have a much shorter (and better) patch, that I'll upload to DELAYED/5 in few minutes. Thanks alot. There is now a new lintian error (not caused by your patch, but by some changed B-D). Since you were looking into the python building bits, perhaps you have an idea where this is originating from? Notice the difference lib vs. lib64 in sid: E: python3-pyhst2-cuda: custom-library-search-path RUNPATH /usr/lib64 [usr/lib/python3/dist-packages/PyHST2_2020c/libgputomo.cpython-311-x86_64-linux-gnu.so] E: python3-pyhst2-cuda: custom-library-search-path RUNPATH /usr/lib64 [usr/lib/python3/dist-packages/PyHST2_2020c/libprojection.cpython-311-x86_64-linux-gnu.so] in bookworm: E: python3-pyhst2-cuda: custom-library-search-path RUNPATH /usr/lib [usr/lib/python3/dist-packages/PyHST2_2020c/libgputomo.cpython-311-x86_64-linux-gnu.so] E: python3-pyhst2-cuda: custom-library-search-path RUNPATH /usr/lib [usr/lib/python3/dist-packages/PyHST2_2020c/libprojection.cpython-311-x86_64-linux-gnu.so] I get this also for rebuilding -6 in bookworm, while it did not happen in my November upload. The buildlog has INFO: x86_64-linux-gnu-gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -g -fwrapv -O2 -Wl,-z,relro -g -O2 -ffile-prefix-map=/build/pyhst2-2020c=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-cpython-311/PyHST/Cspace/c_hst_project_1over.o -L/usr/lib64 -L/usr/lib/x86_64-linux-gnu -Wl,--enable-new-dtags,-R/usr/lib64 -lcudart -lcuda -lcufft -o /build/pyhst2-2020c/.pybuild/cpython3_3.11_pyhst2/build/PyHST2_2020c/libprojection.cpython-311-x86_64-linux-gnu.so Andreas
Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
Control: tags -1 +pending
So. I have a much shorter (and better) patch, that I'll upload to DELAYED/5 in
few minutes.
See the attached patch, which consists of setting the setup.py version to a
PEP-440-compliant Debian-specific version (which I have confirmed to sort
bigger than "2020ca"). This fixes the FTBFS.
I've also pushed this as an MR on your VCS repository there:
https://salsa.debian.org/science-team/pyhst2/-/merge_requests/1
Best,
OdyX
Le samedi, 28 janvier 2023, 17.21:35 h CET Didier 'OdyX' Raboud a écrit :
> Hello there,
>
> while at St-Cergue's BSP, I took a look at this issue, so here are my
> findings.
>
> * I can confirm it fails to build from source in a main+contrib+non-free
> cowbuilder.
>
> * This FTBFS is the consequence of a new requirement from setuptools; see
> this issue to get it better: https://github.com/pypa/setuptools/issues/2497
> . setuptools now imposes PEP-440-style versioning, that is, the version has
> to match this format:
>
> [N!]N(.N)*[{a|b|rc}N][.postN][.devN]
>
> But pyHST2 uses the version number in some paths (such as /usr/lib/
python3/
> dist-packages/PyHST2_2020c(...) ), so it can't simply be patched to use
> "2020.3" (c ~= 3 ?). I can't login to upstream's issues tracker at
> https://gitlab.esrf.fr/mirone/pyhst2 but it's clearly something that needs
> to be fixed by a new upstream release.
>
> The attached patch is basically a `sed 's/2020c/2021/g' -i` (also check
> aumento_versione="a" which can work, but should also be amended) on
> concerned files. It allows to finish building, but the paths have become
> /usr/lib/python3/dist-packages/PyHST2_2021(...), which is clearly not
> upstream's intent.
>
> I'll check if it's possible to only patch the version and not all files.
>
> See you in a bit.
>
> Cheers,
>
> OdyX
>
> Le vendredi, 27 janvier 2023, 15.37:13 h CET Andreas Beckmann a écrit :
> > Source: pyhst2
> > Version: 2020c-6
> > Severity: serious
> > Tags: ftbfs
> > Justification: fails to build from source
> >
> > Hi,
> >
> > pyhst2 FTBFS with setuptools 66 that was uploaded today:
> > /usr/lib/python3/dist-packages/setuptools/dist.py:548: UserWarning: The
> > version specified ('2020ca') is an invalid version, this may not work as
> > expected with newer versions of setuptools, pip, and PyPI. Please see PEP
> > 440 for more details. warnings.warn(
> > running clean
> > removing '/build/pyhst2-2020c/.pybuild/cpython3_3.10_pyhst2/build' (and
> > everything under it) 'build/bdist.linux-x86_64' does not exist -- can't
> > clean it
> > 'build/scripts-3.10' does not exist -- can't clean it
Set a Debian-specific PEP-440-compliant version: 2020.3.20230128
This is needed since setuptools 66.0
Bug-Debian: https://bugs.debian.org/1029775
Author: Didier Raboud
Origin: vendor
--- a/setup.py
+++ b/setup.py
@@ -88,6 +88,9 @@ global version
global aumento_versione
aumento_versione="a"
+# Debian-specific version to match PEP-440 with setuptools v66.0.0
+pep440_version = "2020.3.20230128" # 3 for c, today's date
+
global version
if DO_LINK==0:
@@ -121,7 +124,8 @@ def do_link():
distrib = setup(name="pyhst2_links",
license="GPL",
-version=version+aumento_versione,
+# Debian-specific version to fit PEP-440
+version=pep440_version,
description=description,
author="Alessandro Mirone",
author_email="mir...@esrf.fr",
@@ -156,7 +160,8 @@ def do_link_unstable():
distrib = setup(name="pyhst2unstable_links",
license="GPL",
-version=version+aumento_versione,
+# Debian-specific version to fit PEP-440
+version=pep440_version,
description=description,
author="Alessandro Mirone",
author_email="mir...@esrf.fr",
@@ -697,7 +702,8 @@ def do_pyhst():
distrib = setup(name="PyHST2_"+version+post_fix,
license="GPL",
-version=version+aumento_versione, ## aumenta qua versione
+# Debian-specific version to fit PEP-440
+version=pep440_version,
description=description,
author="Alessandro Mirone",
author_email="mir...@esrf.fr",
signature.asc
Description: This is a digitally signed message part.
Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
Control: tags -1 confirmed +patch
Hello there,
while at St-Cergue's BSP, I took a look at this issue, so here are my
findings.
* I can confirm it fails to build from source in a main+contrib+non-free
cowbuilder.
* This FTBFS is the consequence of a new requirement from setuptools; see this
issue to get it better: https://github.com/pypa/setuptools/issues/2497 .
setuptools now imposes PEP-440-style versioning, that is, the version has to
match this format:
[N!]N(.N)*[{a|b|rc}N][.postN][.devN]
But pyHST2 uses the version number in some paths (such as /usr/lib/python3/
dist-packages/PyHST2_2020c(...) ), so it can't simply be patched to use
"2020.3" (c ~= 3 ?). I can't login to upstream's issues tracker at
https://gitlab.esrf.fr/mirone/pyhst2 but it's clearly something that needs to
be fixed by a new upstream release.
The attached patch is basically a `sed 's/2020c/2021/g' -i` (also check
aumento_versione="a" which can work, but should also be amended) on concerned
files. It allows to finish building, but the paths have become
/usr/lib/python3/dist-packages/PyHST2_2021(...), which is clearly not
upstream's intent.
I'll check if it's possible to only patch the version and not all files.
See you in a bit.
Cheers,
OdyX
Le vendredi, 27 janvier 2023, 15.37:13 h CET Andreas Beckmann a écrit :
> Source: pyhst2
> Version: 2020c-6
> Severity: serious
> Tags: ftbfs
> Justification: fails to build from source
>
> Hi,
>
> pyhst2 FTBFS with setuptools 66 that was uploaded today:
> /usr/lib/python3/dist-packages/setuptools/dist.py:548: UserWarning: The
> version specified ('2020ca') is an invalid version, this may not work as
> expected with newer versions of setuptools, pip, and PyPI. Please see PEP
> 440 for more details. warnings.warn(
> running clean
> removing '/build/pyhst2-2020c/.pybuild/cpython3_3.10_pyhst2/build' (and
> everything under it) 'build/bdist.linux-x86_64' does not exist -- can't
> clean it
> 'build/scripts-3.10' does not exist -- can't clean it
--
OdyXIndex: pyhst2/PyHST/__init__.py
===
--- pyhst2.orig/PyHST/__init__.py
+++ pyhst2/PyHST/__init__.py
@@ -30,4 +30,4 @@
# is a problem for you.
#*/
-version = "2020c"
+version = "2021"
Index: pyhst2/TEST_PYHST/nonregression.py
===
--- pyhst2.orig/TEST_PYHST/nonregression.py
+++ pyhst2/TEST_PYHST/nonregression.py
@@ -67,7 +67,7 @@ casi=[ "CRAYON", "HEIKKI", "HELICOID
"PATCHES_VECTORIAL", "SINO_THRESHOLD" ]
casi=["ID11_SNOW", "BIG", "LENA", "LENA_MULTIRINGS", "MOUSSE", "MULTIPAGANIN","NANOPOINTS", "PATCHES_VECTORIAL", "SINO_THRESHOLD"]
-LAUNCHING_INSTRUCTION = "PyHST2_2020c input.par"
+LAUNCHING_INSTRUCTION = "PyHST2_2021 input.par"
outputprefix="/home/esrf/mirone/nobackup/TEST_PYHST/RESULTS/p9/monogpu"
# outputprefix="/tmp/TEST_PYHST/RESULTS/p9-04/"
###
Index: pyhst2/scripts_link/pyhst2
===
--- pyhst2.orig/scripts_link/pyhst2
+++ pyhst2/scripts_link/pyhst2
@@ -1 +1 @@
-PyHST2_2020c $*
+PyHST2_2021 $*
Index: pyhst2/scripts_link/pyhst2_info
===
--- pyhst2.orig/scripts_link/pyhst2_info
+++ pyhst2/scripts_link/pyhst2_info
@@ -1,6 +1,6 @@
#!python
msg = """
-The pyhst2 script launches PyHST2_2020c
+The pyhst2 script launches PyHST2_2021
*** PyHST2_2020a
Index: pyhst2/setup.py
===
--- pyhst2.orig/setup.py
+++ pyhst2/setup.py
@@ -86,7 +86,7 @@ DO_LINK = 0
global version
global aumento_versione
-aumento_versione="a"
+aumento_versione=".1"
global version
@@ -95,7 +95,7 @@ if DO_LINK==0:
os.path.abspath(__file__)), "PyHST", "__init__.py"))
if l.strip().startswith("version")][0]
else:
-version = "2020c"
+version = "2021"
def do_link():
@@ -104,7 +104,7 @@ def do_link():
# aumento_versione=""
-version = "2020c"
+version = "2021"
packages = []
Index: pyhst2/stdeb_link.cfg
===
--- pyhst2.orig/stdeb_link.cfg
+++ pyhst2/stdeb_link.cfg
@@ -1,3 +1,3 @@
[DEFAULT]
-Depends: python-pyhst2-2020c | python-pyhst2-2020c-base
-Depends3: python3-pyhst2-2020c | python3-pyhst2-2020c-base
+Depends: python-pyhst2-2021 | python-pyhst2-2021-base
+Depends3: python3-pyhst2-2021 | python3-pyhst2-2021-base
signature.asc
Description: This is a digitally signed message part.
Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
Source: pyhst2
Version: 2020c-6
Severity: serious
Tags: ftbfs
Justification: fails to build from source
Hi,
pyhst2 FTBFS with setuptools 66 that was uploaded today:
...
dh_auto_clean -O--buildsystem=pybuild
I: pybuild base:240: python3.10 setup.py clean
/build/pyhst2-2020c/setup.py:59: DeprecationWarning:
`numpy.distutils` is deprecated since NumPy 1.23.0, as a result
of the deprecation of `distutils` itself. It will be removed for
Python >= 3.12. For older Python versions it will remain present.
It is recommended to use `setuptools < 60.0` for those Python versions.
For more details, see:
https://numpy.org/devdocs/reference/distutils_status_migration.html
from numpy.distutils.misc_util import get_numpy_include_dirs
/usr/lib/python3/dist-packages/setuptools/dist.py:548: UserWarning: The version
specified ('2020ca') is an invalid version, this may not work as expected with
newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more
details.
warnings.warn(
running clean
removing '/build/pyhst2-2020c/.pybuild/cpython3_3.10_pyhst2/build' (and
everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.10' does not exist -- can't clean it
...
byte-compiling
/build/pyhst2-2020c/debian/python3-pyhst2-cuda/usr/lib/python3.10/dist-packages/PyHST2_2020c/Parameters_module.py
to Parameters_module.cpython-310.pyc
running install_egg_info
Traceback (most recent call last):
File "/build/pyhst2-2020c/setup.py", line 724, in
do_pyhst()
File "/build/pyhst2-2020c/setup.py", line 698, in do_pyhst
distrib = setup(name="PyHST2_"+version+post_fix,
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line
185, in setup
return run_commands(dist)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line
201, in run_commands
dist.run_commands()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line
969, in run_commands
self.run_command(cmd)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1213, in
run_command
super().run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line
988, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 68,
in run
return orig.install.run(self)
File
"/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line
709, in run
self.run_command(cmd_name)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 318,
in run_command
self.distribution.run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1213, in
run_command
super().run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line
987, in run_command
cmd_obj.ensure_finalized()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 111,
in ensure_finalized
self.finalize_options()
File "/usr/lib/python3/dist-packages/setuptools/command/install_egg_info.py",
line 31, in finalize_options
ei_cmd = self.get_finalized_command("egg_info")
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 305,
in get_finalized_command
cmd_obj.ensure_finalized()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 111,
in ensure_finalized
self.finalize_options()
File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line
219, in finalize_options
parsed_version = parse_version(self.egg_version)
File
"/usr/lib/python3/dist-packages/pkg_resources/_vendor/packaging/version.py",
line 266, in __init__
raise InvalidVersion(f"Invalid version: '{version}'")
pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '2020ca'
E: pybuild pybuild:388: install: plugin distutils failed with: exit code=1:
/usr/bin/python3.10 setup.py install --root
/build/pyhst2-2020c/debian/python3-pyhst2-cuda
dh_auto_install: error: pybuild --install -i python{version} -p "3.10 3.11"
--dest-dir /build/pyhst2-2020c/debian/python3-pyhst2-cuda returned exit code 13
make: *** [debian/rules:16: binary] Error 25
...
Andreas
pyhst2_2020c-6.sid.build.gz
Description: application/gzip
