Re: paraview_5.7.0-5_source.changes ACCEPTED into unstable
Hello Alastair, had to add yet another bug fix for ParaView: https://git.inf.h-brs.de/jmeng2m/debian-package-paraview/-/commit/75154c321d64fe2087f972683a90752a2e55 CI builds currently fail because libopenmpi-dev is not installable on Sid. Jakob Am 04.09.20 um 08:55 schrieb Alastair McKinstry: > Hi Jakob > > Thanks, I'll add these patches over the weekend hopefully. > > Alastair > > On 04/09/2020 07:53, Jakob Meng wrote: >> Hello Alastair, >> thanks for uploading the fixed ParaView package to unstable! >> >> Perhaps you might want to consider these patches as well (details in >> commit messages) [1][2]. I do not know a better place where to discuss >> these changes (do not have write access to Salsa) hence I just wrote >> down my thoughts in the initial bug report [3]. >> >> [1] >> https://git.inf.h-brs.de/jmeng2m/debian-package-paraview/-/commit/7d9c5481fd96b5a9965a6b2ce9395e146f5b70c4 >> >> [2] >> https://git.inf.h-brs.de/jmeng2m/debian-package-paraview/-/commit/46a905e0a25a4e3d7de2dedca31fb746c6f69288 >> >> [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=959387 >> >> Could you push your changes to the ParaView git repo on Salsa as well, >> please? >> >> Thank you! >> >> Kind regards >> Jakob >> PS: My last mails from "jakobm...@web.de" were all rejected by host >> "mail.sceal.ie [83.70.207.66]" with "Message rejected due to: SPF fail - >> not authorized.". Hence I resent it again from a different mail address, >> hopefully this time without being rejected. Please excuse me, if you get >> my mails twice due to this. >> >> signature.asc Description: OpenPGP digital signature -- debian-science-maintainers mailing list debian-science-maintainers@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-science-maintainers
Re: paraview_5.7.0-5_source.changes ACCEPTED into unstable
Hello Alastair, thanks for uploading the fixed ParaView package to unstable! Perhaps you might want to consider these patches as well (details in commit messages) [1][2]. I do not know a better place where to discuss these changes (do not have write access to Salsa) hence I just wrote down my thoughts in the initial bug report [3]. [1] https://git.inf.h-brs.de/jmeng2m/debian-package-paraview/-/commit/7d9c5481fd96b5a9965a6b2ce9395e146f5b70c4 [2] https://git.inf.h-brs.de/jmeng2m/debian-package-paraview/-/commit/46a905e0a25a4e3d7de2dedca31fb746c6f69288 [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=959387 Could you push your changes to the ParaView git repo on Salsa as well, please? Thank you! Kind regards Jakob signature.asc Description: OpenPGP digital signature -- debian-science-maintainers mailing list debian-science-maintainers@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-science-maintainers
Bug#959387: [Re] paraview-dev: missing cmake vtk modules in the dev package, cannot use
Hello Anton, that is great news! Keeping VTK and ParaView in sync' is not going to be easy: Sync'ing VTK sources and ParaView's VTK sources and allowing both to be co-installable, increases the maintenance burden for both package maintainers. Especially for Alastair McKinstry, because he would have to copy and adapt all patches from (Debian's) VTK package sources to his ParaView VTK sources. Latest ParaView release is 5.8.1 [1] and that is using a pre-VTK9 development version of VTK [2][3] ?!? The master branch of next ParaView release [4] has VTK 9 [5]. ParaView's release tarballs are published without the VTK subfolder [6], but I am not sure whether that is due to a "bug" [7] of the GitHub release process or if that means it is valid to "mix" different ParaView and VTK releases. ParaView CMakeLists.txt uses VTK's CMake files [8] before considering PARAVIEW_USE_EXTERNAL_VTK [9], hence ParaView requires the VTK subdirectory to be present anyway. For Debian users, it would be beneficial to have VTK and ParaView packages in sync and possibly installed at the same time, because that enables them e.g. 1. run (i)python3 or pvpython and enter 'import vtk' without crashs 2. use find_package(VTK) and find_package(ParaView) in a single CMakeLists.txt without tedious if-else constructs [1] https://gitlab.kitware.com/paraview/paraview/-/tree/v5.8.1 [2] https://gitlab.kitware.com/paraview/paraview/-/commit/ce73a508883c599927967be854ecfbf1562f032f [3] https://gitlab.kitware.com/vtk/vtk/-/blob/da522f73bc0639f8c995b8c89e21d604656f99e2/CMake/vtkVersion.cmake [4] https://gitlab.kitware.com/paraview/paraview [5] https://gitlab.kitware.com/vtk/vtk/-/blob/ae3f586abc4032895ce05e664d4a6ad84f990ef2/CMake/vtkVersion.cmake [6] https://github.com/Kitware/ParaView/releases [7] https://stackoverflow.com/a/34721233/6490710 [8] https://github.com/Kitware/ParaView/blob/v5.8.1/CMakeLists.txt#L98 [9] https://github.com/Kitware/ParaView/blob/v5.8.1/CMakeLists.txt#L218 Best regards Jakob signature.asc Description: OpenPGP digital signature -- debian-science-maintainers mailing list debian-science-maintainers@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-science-maintainers
Bug#959387: [Re] paraview-dev: missing cmake vtk modules in the dev package, cannot use
PPPS: ParaView installs its Python modules as well as all the VTK Python modules as part of its CMake install process. If python3-vtk8 [1] ever enters Debian this might cause file collisions, as both python3-paraview and python3-vtk8 install files to the same locations and hence cannot be co-installed. So either we skip installing VTK Python modules during ParaView install or we install those ParaView's VTK Python modules to a different location by setting e.g. VTK_PYTHON_SITE_PACKAGES_SUFFIX. When skipping VTK Python module install, then we have to make sure that all VTK-related files in ParaView and VTK packages do match exactly and have the same patches applied. Else Python, ParaView or its Python modules might crash or even worse cause silent data and memory corruption bugs. Just calling 'rm -fr debian/tmp/usr/lib/python3*/dist-packages/vtk*' in ParaView's debian/rules is not a proper workaround. ParaView 5.7 from Debian Sid uses a development version of VTK8 (not far from VTK9 release?) [4], while Debian's dedicated VTK8 package [1] is at 8.2, so both VTK versions have many differences. ParaView has CMake flags to build with an external VTK install [6], but this is not supported at least in ParaView 5.7.0 [7]. BTW, these incompatibilities affect older Debian releases as well. For example, install both packages 'paraview-python' [2] and 'python3-vtk7' [3] on Debian 10 (Buster), run pvpython and enter 'import vtk'. ParaView Python will crash. So always remember to enter 'import paraview.vtk' on Debian and Ubuntu [5]. To avoid these bugs, my reworked ParaView package does not allow parallel installs of python3-vtk7 or python3-vtk8 [8]. Both paraview-dev and python3-paraview install a lot (all?) of VTK's headers, libraries and Python modules anyway. Ref.: [1] https://salsa.debian.org/science-team/vtk8/ [2] https://packages.debian.org/buster/paraview-python [3] https://packages.debian.org/buster/python3-vtk7 [4] https://salsa.debian.org/science-team/paraview/-/blob/master/VTK/CMake/vtkVersion.cmake [5] https://blog.kitware.com/import-vtk-or-import-paraview-vtk/ [6] https://github.com/Kitware/ParaView/blob/v5.8.1/CMakeLists.txt#L218 [7] https://github.com/Kitware/ParaView/blob/v5.7.0/CMakeLists.txt#L290 [8] https://git.inf.h-brs.de/jmeng2m/debian-package-paraview/-/blob/debian/5.7.0+ds.1-3/debian/control#L172 signature.asc Description: OpenPGP digital signature -- debian-science-maintainers mailing list debian-science-maintainers@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-science-maintainers
Bug#957660: [Re] paraview: ftbfs with GCC-10
A fix for compiling paraview 5.7.0-4 with gcc10 can be found in bug #959387 [1], esp. [2]. Ref.: [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=959387 [2] https://git.inf.h-brs.de/jmeng2m/debian-package-paraview/-/commit/35d8cae30d4e9d1c6da980f2b359b428a5beea89 signature.asc Description: OpenPGP digital signature -- debian-science-maintainers mailing list debian-science-maintainers@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-science-maintainers
Bug#959387: [Re] paraview-dev: missing cmake vtk modules in the dev package, cannot use
PPS: Binary packages can be downloaded here: http://home.infcs.de/~jmeng2m/debian-package-paraview/ They were build using my university's GitLab CI instance and the official debian/buildd:sid docker image. CI file: https://git.inf.h-brs.de/jmeng2m/debian-package-paraview/-/blob/gitlab-ci/.gitlab-ci.yml Build log: https://git.inf.h-brs.de/jmeng2m/debian-package-paraview/-/jobs/421 signature.asc Description: OpenPGP digital signature -- debian-science-maintainers mailing list debian-science-maintainers@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-science-maintainers
Bug#959387: [Re] paraview-dev: missing cmake vtk modules in the dev package, cannot use
PS: I've uploaded a patched package based on ParaView 5.7.0-4 from Debian Sid to https://git.inf.h-brs.de/jmeng2m/debian-package-paraview signature.asc Description: OpenPGP digital signature -- debian-science-maintainers mailing list debian-science-maintainers@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-science-maintainers
Bug#959387: [Re] paraview-dev: missing cmake vtk modules in the dev package, cannot use
Found another problem with the ParaView source package in Debian Sid. The Salsa GitLab Repo of ParaView has Git LFS Pointers, e.g. [1]. The source tarball paraview_5.7.0.orig.tar.xz [2] that has been uploaded to Debian Sid does not have the binary files but the raw Git LFS pointers instead! For example, file VTK/ThirdParty/vtkm/vtkvtkm/vtk-m/data/tornado.vec in the source tarball is: version https://git-lfs.github.com/spec/v1 oid sha256:dca8105bf888e67a9fe476a563d69ac708925aec519814fb520c6057e5ca0a1f size 1327116 [1] https://salsa.debian.org/science-team/paraview/-/tree/master/VTK/ThirdParty/vtkm/vtkvtkm/vtk-m/data [2] http://deb.debian.org/debian/pool/main/p/paraview/paraview_5.7.0.orig.tar.xz signature.asc Description: OpenPGP digital signature -- debian-science-maintainers mailing list debian-science-maintainers@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-science-maintainers
Bug#959387: [Re] paraview-dev: missing cmake vtk modules in the dev package, cannot use
Hello, paraview-dev from Debian Testing/Sid is broken. I tried to fix it, but new bugs were popping up as soon as i fixed one. Here are my findings. First, the VCS repository [1][2] is not up to date, it's missing commits: Salsa has 5.7.0-2 [1], but Debian Sid has 5.7.0-4 [3]. I worked on the newer version from Sid. I applied all patches from merge request #1 [4] on top of ParaView 5.7.0-4 to fix bug #959387 [5]. One set of bugs is caused by debian/rules, because it alters the filesystem after it has build ParaView. For example, it moves files from /usr/lib/x86_64-linux-gnu/python3* to /usr/lib/python3* [6], renames e.g. /usr/bin/pvbatch3.8 to /usr/bin/pvbatch and then dh_python3 renames libraries e.g. from vtkClientServerPython.so to vtkClientServerPython.cpython-38-x86_64-linux-gnu.so. Hence ParaView's exported CMake files [7] cannot find those files at the original locations. When you try to import ParaView into your own code using CMake (and its find_package command), then CMake fails due to missing files. For some ideas how to fix those bugs, please see file debian.patch [8]. ParaView's CMake flag VTK_PYTHON_OPTIONAL_LINK must be OFF to fix vtk issue #17692 [9]. Another problem is, that FFMPEG_VERSION in libavutil/ffversion.h [10] from Debian package libavutil-dev [11] is "4.3-3+b2". ParaView picks up this version in VTK/CMake/FindFFMPEG.cmake [12] during build, stores it in CMake variable FFMPEG_VERSION and then exports it to file VTK-vtk-module-find-packages.cmake [13]. The latter then contains "find_package(FFMPEG 4.3-3+b2 ...", which breaks CMake's find_package [14], because find_package's version format is major[.minor[.patch[.tweak]]], but no plus or minus. So once again, when you try to "find_package(ParaView...)" in your own CMakeLists.txt, then CMake fails again. One workaround is to remove the additional version postfix from the FFMPEG_VERSION CMake variable, please see file vtk-cmake-findffmpeg-cmake.patch [15]. By the way, in ParaView's sources [1], there is a subdirectory "VTK" which is a copy of the upstream vtk.git [16]. (This "VTK" folder is actually a git submodule in the upstream repo [17]). The vtk code is already packaged in Debian [18][19] and has a lot of patches [20], but the VTK subdirectory in ParaView lacks most of these patches. Perhaps it may be beneficial to either copy those patches [20] or copy the whole already-patched vtk source tree from [19] to ParaView's VTK subdirectory. Another quirk of Debian's ParaView package, that got me thinking, is mpi4py. ParaView builds and links against its own variant of mpi4py, but the resulting files are not integrated into the debian package. At runtime the system provided mpi4py libraries are used. One has to be very careful with keeping both versions, the ParaView-build-in-but-not-installed and the system-provided mpi4py libraries, in sync to prevent probably-hard-to-detect bugs. Unfortunately i could not figure out yet how to force ParaView to use mpi4py that is provided by the system. After all patches [8][15] have been applied, my little toy project [21] successfully compiles and links against ParaView. I never expected that creating debian packages for ParaView is that hard! Hopefully, someday someone with more debian skills fixes the "dh_python3 --no-ext-rename" hack in debian/rules and uploads a fixed debian package. Thank you and keep up the great work :-) Regards Jakob [1] https://salsa.debian.org/science-team/paraview [2] https://tracker.debian.org/pkg/paraview [3] https://packages.debian.org/sid/paraview [4] https://salsa.debian.org/science-team/paraview/-/merge_requests/1 [5] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=959387 [6] mv debian/tmp/usr/lib/*/python3* debian/tmp/usr/lib/ [7] /usr/lib/x86_64-linux-gnu/cmake/paraview-5.7/ParaViewPython-targets-release.cmake [8] debian.patch [attached to this mail] [9] https://gitlab.kitware.com/vtk/vtk/-/issues/17692 [10] /usr/include/x86_64-linux-gnu/libavutil/ffversion.h [11] https://packages.debian.org/buster/libavutil-dev [12] https://salsa.debian.org/science-team/paraview/-/blob/4c909f1888037e64a655f630910c9f90a5ae3bda/VTK/CMake/FindFFMPEG.cmake#L152 [13] /usr/lib/x86_64-linux-gnu/cmake/paraview-5.7/vtk/VTK-vtk-module-find-packages.cmake [14] https://cmake.org/cmake/help/latest/command/find_package.html [15] vtk-cmake-findffmpeg-cmake.patch [attached to this mail] [16] https://gitlab.kitware.com/vtk/vtk.git [17] https://github.com/Kitware/ParaView [18] https://salsa.debian.org/science-team/vtk7 [19] https://tracker.debian.org/pkg/vtk7 [20] https://salsa.debian.org/science-team/vtk7/-/tree/master/debian/patches [21] paraview-example.zip [attached to this mail] --- paraview-5.7.0.orig/debian/rules 2020-08-06 09:08:41.717132011 +0200 +++ paraview-5.7.0/debian/rules 2020-08-06 11:47:47.878542070 +0200 @@ -53,6 +53,8 @@ -DVTK_INSTALL_DATA_DIR="share/paraview" \ -DVTK_INSTALL_DOC_DIR="share/doc/paraview" \ -DVTK_INSTALL_PACKAGE_DIR="lib/cmake/paraview" \ +
