Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
Le lundi, 30 janvier 2023, 18.59:04 h CET Andreas Beckmann a écrit : > On 28/01/2023 18.09, Didier 'OdyX' Raboud wrote: > > So. I have a much shorter (and better) patch, that I'll upload to > > DELAYED/5 in few minutes. > > Thanks alot. No problem. Should I push it to unstable? > There is now a new lintian error (not caused by your patch, > but by some changed B-D). Since you were looking into > the python building bits, perhaps you have an idea where this > is originating from? Notice the difference lib vs. lib64 The code in setup.py does non-Debian stuff around guessing library paths. A possible patch is attached, I've also pushed it on a branch: https://salsa.debian.org/science-team/pyhst2/-/merge_requests/2 I have no idea if the resulting package works though. Best, OdyXDrop all CUDA-specific library path search Fixes the superfluous /usr/lib or /usr/lib64 paths in RUNPATH Author: Didier Raboud Origin: vendor --- a/setup.py +++ b/setup.py @@ -261,15 +261,6 @@ def do_pyhst(): cudaconfig = {'home':home, 'nvcc':nvcc, 'include': pjoin(home, 'include')} -lib = pjoin(home, 'lib') -if os.path.exists(lib+"64") andmath.log(sys.maxsize)/math.log(2) > 60: ##sys.maxint == 2**63-1: -cudaconfig["lib"] = lib+"64" -elif os.path.exists(lib+"32") and math.log(sys.maxsize)/math.log(2) < 60: -cudaconfig["lib"] = lib+"32" -elif os.path.exists(lib): -cudaconfig["lib"] = lib -else: -print(("No cuda library found ")) for key, val in cudaconfig.items(): if not os.path.exists(val): raise EnvironmentError('The CUDA %s path could not be located in %s' % (key, val)) @@ -469,9 +460,7 @@ def do_pyhst(): module = Extension('PyHST2_'+version+'/libgputomo', sources=sources, -library_dirs=[CUDA['lib']], libraries=['cudart', "cublas", "cuda", "cufft", hdf5_lib], -runtime_library_dirs=[CUDA['lib']], # this syntax is specific to this build system # we're only going to use certain compiler args with nvcc and not with gcc # the implementation of this trick is in customize_compiler() below @@ -495,11 +484,8 @@ def do_pyhst(): module = Extension(name='PyHST2_'+version+'.unsharp3D', sources=sources , depends=[], - library_dirs=[CUDA['lib']], libraries=['cudart','QtGui','QtCore','cudart','tiff'], - runtime_library_dirs=[CUDA['lib']], - extra_compile_args={'gcc': ["-std=c99","-g","-fPIC", "-O3","-Wall"], 'nvcc': CUDA["arch"] + [ "--compiler-options", "-fPIC", "-O3", "-g","-D_FORCE_INLINES" ]}, include_dirs=[numpy.get_include(), CUDA['include'], 'PyHST/Cspace',"/usr/include/qt4"] ) @@ -518,9 +504,7 @@ def do_pyhst(): global version module = Extension('PyHST2_'+version+'/libprojection', sources=["PyHST/Cspace/c_hst_project_1over.cu"], - library_dirs=[CUDA['lib']], libraries=['cudart', "cuda", "cufft"], - runtime_library_dirs=[CUDA['lib']], extra_compile_args={'gcc': ["-fPIC", "-O3"], 'nvcc': CUDA["arch"] + ["--compiler-options", "-fPIC,-O3" ,"-D_FORCE_INLINES"]}, include_dirs=[numpy.get_include(), CUDA['include'], 'PyHST/Cspace']+ hdf5_dirs) @@ -542,9 +526,7 @@ def do_pyhst(): module = Extension('PyHST2_'+version+'/libwavelets', sources=sources, - library_dirs=[CUDA['lib']], libraries=["cudart", "cuda", "cublas"], - runtime_library_dirs=[CUDA['lib']], extra_compile_args={'gcc': ["-fPIC", "-O3"], 'nvcc': CUDA["arch"] + ["--compiler-options", "-fPIC,-O3" ,"-D_FORCE_INLINES"]}, include_dirs=[numpy.get_include(), CUDA['include'], 'PyHST/Cspace']) signature.asc Description: This is a digitally signed message part.
Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
On 28/01/2023 18.09, Didier 'OdyX' Raboud wrote: So. I have a much shorter (and better) patch, that I'll upload to DELAYED/5 in few minutes. Thanks alot. There is now a new lintian error (not caused by your patch, but by some changed B-D). Since you were looking into the python building bits, perhaps you have an idea where this is originating from? Notice the difference lib vs. lib64 in sid: E: python3-pyhst2-cuda: custom-library-search-path RUNPATH /usr/lib64 [usr/lib/python3/dist-packages/PyHST2_2020c/libgputomo.cpython-311-x86_64-linux-gnu.so] E: python3-pyhst2-cuda: custom-library-search-path RUNPATH /usr/lib64 [usr/lib/python3/dist-packages/PyHST2_2020c/libprojection.cpython-311-x86_64-linux-gnu.so] in bookworm: E: python3-pyhst2-cuda: custom-library-search-path RUNPATH /usr/lib [usr/lib/python3/dist-packages/PyHST2_2020c/libgputomo.cpython-311-x86_64-linux-gnu.so] E: python3-pyhst2-cuda: custom-library-search-path RUNPATH /usr/lib [usr/lib/python3/dist-packages/PyHST2_2020c/libprojection.cpython-311-x86_64-linux-gnu.so] I get this also for rebuilding -6 in bookworm, while it did not happen in my November upload. The buildlog has INFO: x86_64-linux-gnu-gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -g -fwrapv -O2 -Wl,-z,relro -g -O2 -ffile-prefix-map=/build/pyhst2-2020c=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-cpython-311/PyHST/Cspace/c_hst_project_1over.o -L/usr/lib64 -L/usr/lib/x86_64-linux-gnu -Wl,--enable-new-dtags,-R/usr/lib64 -lcudart -lcuda -lcufft -o /build/pyhst2-2020c/.pybuild/cpython3_3.11_pyhst2/build/PyHST2_2020c/libprojection.cpython-311-x86_64-linux-gnu.so Andreas
Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
Control: tags -1 +pending So. I have a much shorter (and better) patch, that I'll upload to DELAYED/5 in few minutes. See the attached patch, which consists of setting the setup.py version to a PEP-440-compliant Debian-specific version (which I have confirmed to sort bigger than "2020ca"). This fixes the FTBFS. I've also pushed this as an MR on your VCS repository there: https://salsa.debian.org/science-team/pyhst2/-/merge_requests/1 Best, OdyX Le samedi, 28 janvier 2023, 17.21:35 h CET Didier 'OdyX' Raboud a écrit : > Hello there, > > while at St-Cergue's BSP, I took a look at this issue, so here are my > findings. > > * I can confirm it fails to build from source in a main+contrib+non-free > cowbuilder. > > * This FTBFS is the consequence of a new requirement from setuptools; see > this issue to get it better: https://github.com/pypa/setuptools/issues/2497 > . setuptools now imposes PEP-440-style versioning, that is, the version has > to match this format: > > [N!]N(.N)*[{a|b|rc}N][.postN][.devN] > > But pyHST2 uses the version number in some paths (such as /usr/lib/ python3/ > dist-packages/PyHST2_2020c(...) ), so it can't simply be patched to use > "2020.3" (c ~= 3 ?). I can't login to upstream's issues tracker at > https://gitlab.esrf.fr/mirone/pyhst2 but it's clearly something that needs > to be fixed by a new upstream release. > > The attached patch is basically a `sed 's/2020c/2021/g' -i` (also check > aumento_versione="a" which can work, but should also be amended) on > concerned files. It allows to finish building, but the paths have become > /usr/lib/python3/dist-packages/PyHST2_2021(...), which is clearly not > upstream's intent. > > I'll check if it's possible to only patch the version and not all files. > > See you in a bit. > > Cheers, > > OdyX > > Le vendredi, 27 janvier 2023, 15.37:13 h CET Andreas Beckmann a écrit : > > Source: pyhst2 > > Version: 2020c-6 > > Severity: serious > > Tags: ftbfs > > Justification: fails to build from source > > > > Hi, > > > > pyhst2 FTBFS with setuptools 66 that was uploaded today: > > /usr/lib/python3/dist-packages/setuptools/dist.py:548: UserWarning: The > > version specified ('2020ca') is an invalid version, this may not work as > > expected with newer versions of setuptools, pip, and PyPI. Please see PEP > > 440 for more details. warnings.warn( > > running clean > > removing '/build/pyhst2-2020c/.pybuild/cpython3_3.10_pyhst2/build' (and > > everything under it) 'build/bdist.linux-x86_64' does not exist -- can't > > clean it > > 'build/scripts-3.10' does not exist -- can't clean it Set a Debian-specific PEP-440-compliant version: 2020.3.20230128 This is needed since setuptools 66.0 Bug-Debian: https://bugs.debian.org/1029775 Author: Didier Raboud Origin: vendor --- a/setup.py +++ b/setup.py @@ -88,6 +88,9 @@ global version global aumento_versione aumento_versione="a" +# Debian-specific version to match PEP-440 with setuptools v66.0.0 +pep440_version = "2020.3.20230128" # 3 for c, today's date + global version if DO_LINK==0: @@ -121,7 +124,8 @@ def do_link(): distrib = setup(name="pyhst2_links", license="GPL", -version=version+aumento_versione, +# Debian-specific version to fit PEP-440 +version=pep440_version, description=description, author="Alessandro Mirone", author_email="mir...@esrf.fr", @@ -156,7 +160,8 @@ def do_link_unstable(): distrib = setup(name="pyhst2unstable_links", license="GPL", -version=version+aumento_versione, +# Debian-specific version to fit PEP-440 +version=pep440_version, description=description, author="Alessandro Mirone", author_email="mir...@esrf.fr", @@ -697,7 +702,8 @@ def do_pyhst(): distrib = setup(name="PyHST2_"+version+post_fix, license="GPL", -version=version+aumento_versione, ## aumenta qua versione +# Debian-specific version to fit PEP-440 +version=pep440_version, description=description, author="Alessandro Mirone", author_email="mir...@esrf.fr", signature.asc Description: This is a digitally signed message part.
Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
Control: tags -1 confirmed +patch Hello there, while at St-Cergue's BSP, I took a look at this issue, so here are my findings. * I can confirm it fails to build from source in a main+contrib+non-free cowbuilder. * This FTBFS is the consequence of a new requirement from setuptools; see this issue to get it better: https://github.com/pypa/setuptools/issues/2497 . setuptools now imposes PEP-440-style versioning, that is, the version has to match this format: [N!]N(.N)*[{a|b|rc}N][.postN][.devN] But pyHST2 uses the version number in some paths (such as /usr/lib/python3/ dist-packages/PyHST2_2020c(...) ), so it can't simply be patched to use "2020.3" (c ~= 3 ?). I can't login to upstream's issues tracker at https://gitlab.esrf.fr/mirone/pyhst2 but it's clearly something that needs to be fixed by a new upstream release. The attached patch is basically a `sed 's/2020c/2021/g' -i` (also check aumento_versione="a" which can work, but should also be amended) on concerned files. It allows to finish building, but the paths have become /usr/lib/python3/dist-packages/PyHST2_2021(...), which is clearly not upstream's intent. I'll check if it's possible to only patch the version and not all files. See you in a bit. Cheers, OdyX Le vendredi, 27 janvier 2023, 15.37:13 h CET Andreas Beckmann a écrit : > Source: pyhst2 > Version: 2020c-6 > Severity: serious > Tags: ftbfs > Justification: fails to build from source > > Hi, > > pyhst2 FTBFS with setuptools 66 that was uploaded today: > /usr/lib/python3/dist-packages/setuptools/dist.py:548: UserWarning: The > version specified ('2020ca') is an invalid version, this may not work as > expected with newer versions of setuptools, pip, and PyPI. Please see PEP > 440 for more details. warnings.warn( > running clean > removing '/build/pyhst2-2020c/.pybuild/cpython3_3.10_pyhst2/build' (and > everything under it) 'build/bdist.linux-x86_64' does not exist -- can't > clean it > 'build/scripts-3.10' does not exist -- can't clean it -- OdyXIndex: pyhst2/PyHST/__init__.py === --- pyhst2.orig/PyHST/__init__.py +++ pyhst2/PyHST/__init__.py @@ -30,4 +30,4 @@ # is a problem for you. #*/ -version = "2020c" +version = "2021" Index: pyhst2/TEST_PYHST/nonregression.py === --- pyhst2.orig/TEST_PYHST/nonregression.py +++ pyhst2/TEST_PYHST/nonregression.py @@ -67,7 +67,7 @@ casi=[ "CRAYON", "HEIKKI", "HELICOID "PATCHES_VECTORIAL", "SINO_THRESHOLD" ] casi=["ID11_SNOW", "BIG", "LENA", "LENA_MULTIRINGS", "MOUSSE", "MULTIPAGANIN","NANOPOINTS", "PATCHES_VECTORIAL", "SINO_THRESHOLD"] -LAUNCHING_INSTRUCTION = "PyHST2_2020c input.par" +LAUNCHING_INSTRUCTION = "PyHST2_2021 input.par" outputprefix="/home/esrf/mirone/nobackup/TEST_PYHST/RESULTS/p9/monogpu" # outputprefix="/tmp/TEST_PYHST/RESULTS/p9-04/" ### Index: pyhst2/scripts_link/pyhst2 === --- pyhst2.orig/scripts_link/pyhst2 +++ pyhst2/scripts_link/pyhst2 @@ -1 +1 @@ -PyHST2_2020c $* +PyHST2_2021 $* Index: pyhst2/scripts_link/pyhst2_info === --- pyhst2.orig/scripts_link/pyhst2_info +++ pyhst2/scripts_link/pyhst2_info @@ -1,6 +1,6 @@ #!python msg = """ -The pyhst2 script launches PyHST2_2020c +The pyhst2 script launches PyHST2_2021 *** PyHST2_2020a Index: pyhst2/setup.py === --- pyhst2.orig/setup.py +++ pyhst2/setup.py @@ -86,7 +86,7 @@ DO_LINK = 0 global version global aumento_versione -aumento_versione="a" +aumento_versione=".1" global version @@ -95,7 +95,7 @@ if DO_LINK==0: os.path.abspath(__file__)), "PyHST", "__init__.py")) if l.strip().startswith("version")][0] else: -version = "2020c" +version = "2021" def do_link(): @@ -104,7 +104,7 @@ def do_link(): # aumento_versione="" -version = "2020c" +version = "2021" packages = [] Index: pyhst2/stdeb_link.cfg === --- pyhst2.orig/stdeb_link.cfg +++ pyhst2/stdeb_link.cfg @@ -1,3 +1,3 @@ [DEFAULT] -Depends: python-pyhst2-2020c | python-pyhst2-2020c-base -Depends3: python3-pyhst2-2020c | python3-pyhst2-2020c-base +Depends: python-pyhst2-2021 | python-pyhst2-2021-base +Depends3: python3-pyhst2-2021 | python3-pyhst2-2021-base signature.asc Description: This is a digitally signed message part.
Bug#1029775: pyhst2: FTBFS with setuptools 66: Invalid version: '2020ca'
Source: pyhst2 Version: 2020c-6 Severity: serious Tags: ftbfs Justification: fails to build from source Hi, pyhst2 FTBFS with setuptools 66 that was uploaded today: ... dh_auto_clean -O--buildsystem=pybuild I: pybuild base:240: python3.10 setup.py clean /build/pyhst2-2020c/setup.py:59: DeprecationWarning: `numpy.distutils` is deprecated since NumPy 1.23.0, as a result of the deprecation of `distutils` itself. It will be removed for Python >= 3.12. For older Python versions it will remain present. It is recommended to use `setuptools < 60.0` for those Python versions. For more details, see: https://numpy.org/devdocs/reference/distutils_status_migration.html from numpy.distutils.misc_util import get_numpy_include_dirs /usr/lib/python3/dist-packages/setuptools/dist.py:548: UserWarning: The version specified ('2020ca') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details. warnings.warn( running clean removing '/build/pyhst2-2020c/.pybuild/cpython3_3.10_pyhst2/build' (and everything under it) 'build/bdist.linux-x86_64' does not exist -- can't clean it 'build/scripts-3.10' does not exist -- can't clean it ... byte-compiling /build/pyhst2-2020c/debian/python3-pyhst2-cuda/usr/lib/python3.10/dist-packages/PyHST2_2020c/Parameters_module.py to Parameters_module.cpython-310.pyc running install_egg_info Traceback (most recent call last): File "/build/pyhst2-2020c/setup.py", line 724, in do_pyhst() File "/build/pyhst2-2020c/setup.py", line 698, in do_pyhst distrib = setup(name="PyHST2_"+version+post_fix, File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1213, in run_command super().run_command(command) File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 68, in run return orig.install.run(self) File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 709, in run self.run_command(cmd_name) File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1213, in run_command super().run_command(command) File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command cmd_obj.ensure_finalized() File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized self.finalize_options() File "/usr/lib/python3/dist-packages/setuptools/command/install_egg_info.py", line 31, in finalize_options ei_cmd = self.get_finalized_command("egg_info") File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 305, in get_finalized_command cmd_obj.ensure_finalized() File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized self.finalize_options() File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line 219, in finalize_options parsed_version = parse_version(self.egg_version) File "/usr/lib/python3/dist-packages/pkg_resources/_vendor/packaging/version.py", line 266, in __init__ raise InvalidVersion(f"Invalid version: '{version}'") pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '2020ca' E: pybuild pybuild:388: install: plugin distutils failed with: exit code=1: /usr/bin/python3.10 setup.py install --root /build/pyhst2-2020c/debian/python3-pyhst2-cuda dh_auto_install: error: pybuild --install -i python{version} -p "3.10 3.11" --dest-dir /build/pyhst2-2020c/debian/python3-pyhst2-cuda returned exit code 13 make: *** [debian/rules:16: binary] Error 25 ... Andreas pyhst2_2020c-6.sid.build.gz Description: application/gzip