On Thu, Aug 27, 2020 at 2:50 PM Blaise A Bourdin <[email protected]> wrote:
> Hi, > > I am trying to build firedrake using intel compilers, mpi, and python. I > compiled petsc using the following options, and petsc4py is installed by > firedrake-install (see the compilation options below) > > ./configure --with-mpi-dir=$MPI_HOME --CFLAGS='-std=c11 > -D_GNU_SOURCE' --CXXFLAGS='' --FFLAGS='' --COPTFLAGS='-O3 > -xCASCADELAKE -g' --FOPTFLAGS='-O3 -xCASCADELAKE -g' > --CXXOPTFLAGS='-O3 -xCASCADELAKE -g' --with-c++-support > --with-c-support --with-fortran --with-fortran-bindings=0 > --with-cxx-dialect=C++11 --download-ptscotch --download-hdf5= > https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.4/src/hdf5-1.10.4.tar.bz2 > --download-hypre --download-superlu_dist --download-metis > --download-parmetis --download-scalapack --download-mumps > --download-chaco --download-ml --download-eigen= > https://github.com/eigenteam/eigen-git-mirror/archive/3.3.3.tar.gz > --download-sowing=0 > > > > Here is what I get when trying to import petsc4py: > > (firedrake) mef90:firedrake$ python3 > Python 3.7.7 (default, Jun 26 2020, 05:10:03) > [GCC 7.3.0] :: Intel(R) Corporation on linux > Type "help", "copyright", "credits" or "license" for more information. > Intel(R) Distribution for Python is brought to you by Intel Corporation. > Please check out: https://software.intel.com/en-us/python-distribution > >>> import petsc4py > >>> petsc4py.init() > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File > "/home/blaise/Development/firedrake/lib/python3.7/site-packages/petsc4py/__init__.py", > line 42, in init > PETSc = petsc4py.lib.ImportPETSc(arch) > File > "/home/blaise/Development/firedrake/lib/python3.7/site-packages/petsc4py/lib/__init__.py", > line 29, in ImportPETSc > return Import('petsc4py', 'PETSc', path, arch) > File > "/home/blaise/Development/firedrake/lib/python3.7/site-packages/petsc4py/lib/__init__.py", > line 73, in Import > module = import_module(pkg, name, path, arch) > File > "/home/blaise/Development/firedrake/lib/python3.7/site-packages/petsc4py/lib/__init__.py", > line 58, in import_module > with f: return imp.load_module(fullname, f, fn, info) > File "/share/apps/intel-2020.2/intelpython3/lib/python3.7/imp.py", line > 242, in load_module > return load_dynamic(name, filename, file) > File "/share/apps/intel-2020.2/intelpython3/lib/python3.7/imp.py", line > 342, in load_dynamic > return _load(spec) > File "<frozen importlib._bootstrap>", line 696, in _load > File "<frozen importlib._bootstrap>", line 670, in _load_unlocked > File "<frozen importlib._bootstrap>", line 583, in module_from_spec > File "<frozen importlib._bootstrap_external>", line 1043, in > create_module > File "<frozen importlib._bootstrap>", line 219, in > _call_with_frames_removed > ImportError: > /home/blaise/Development/firedrake/lib/python3.7/site-packages/petsc4py/lib/RHEL7-intel2020.2-impi-firedrake/ > PETSc.cpython-37m-x86_64-linux-gnu.so: undefined symbol: > PetscPartitionerInitializePackage > > I have this symbol: master $:/PETSc3/petsc/petsc-dev$ nm -o arch-master-debug/lib/libpetsc.dylib | grep PetscPartitionerInitializePackage arch-master-debug/lib/libpetsc.dylib: 0000000000e07f90 T _PetscPartitionerInitializePackage Can you run nm -o /home/blaise/Development/firedrake/lib/python3.7/site-packages/petsc4py/lib/RHEL7-intel2020.2-impi-firedrake/ PETSc.cpython-37m-x86_64-linux-gnu.so | grep PetscPartitionerInitializePackage Thanks, Matt > The relevant part of the firedrake-install.log are below. > > > 2020-08-27 12:48:23,861 INFO Installing petsc4py/ > 2020-08-27 12:48:23,882 DEBUG Running command > '/home/blaise/Development/firedrake/bin/python -m pip install > --no-binary mpi4py,randomgen,islpy --no-deps -vvv --ignore-installed > petsc4py/' > 2020-08-27 12:50:32,017 DEBUG Using pip 20.2.2 from > /home/blaise/Development/firedrake/lib/python3.7/site-packages/pip (python > 3.7) > Non-user install because user site-packages disabled > Created temporary directory: /tmp/pip-ephem-wheel-cache-5540399x > Created temporary directory: /tmp/pip-req-tracker-jo7ho6yz > Initialized build tracking at /tmp/pip-req-tracker-jo7ho6yz > Created build tracker: /tmp/pip-req-tracker-jo7ho6yz > Entered build tracker: /tmp/pip-req-tracker-jo7ho6yz > Created temporary directory: /tmp/pip-install-e3u5z76o > Processing ./petsc4py > Created temporary directory: /tmp/pip-req-build-r3agmgty > Added file:///home/blaise/Development/firedrake/src/petsc4py to build > tracker '/tmp/pip-req-tracker-jo7ho6yz' > Running setup.py (path:/tmp/pip-req-build-r3agmgty/setup.py) egg_info > for package from file:///home/blaise/Development/firedrake/src/petsc4py > Created temporary directory: /tmp/pip-pip-egg-info-dctmtwx0 > Running command python setup.py egg_info > running egg_info > creating /tmp/pip-pip-egg-info-dctmtwx0/petsc4py.egg-info > writing /tmp/pip-pip-egg-info-dctmtwx0/petsc4py.egg-info/PKG-INFO > writing dependency_links to > /tmp/pip-pip-egg-info-dctmtwx0/petsc4py.egg-info/dependency_links.txt > writing requirements to > /tmp/pip-pip-egg-info-dctmtwx0/petsc4py.egg-info/requires.txt > writing top-level names to > /tmp/pip-pip-egg-info-dctmtwx0/petsc4py.egg-info/top_level.txt > writing manifest file > '/tmp/pip-pip-egg-info-dctmtwx0/petsc4py.egg-info/SOURCES.txt' > reading manifest file > '/tmp/pip-pip-egg-info-dctmtwx0/petsc4py.egg-info/SOURCES.txt' > reading manifest template 'MANIFEST.in' > writing manifest file > '/tmp/pip-pip-egg-info-dctmtwx0/petsc4py.egg-info/SOURCES.txt' > Source in /tmp/pip-req-build-r3agmgty has version 3.13.0, which > satisfies requirement petsc4py==3.13.0 from > file:///home/blaise/Development/firedrake/src/petsc4py > Removed petsc4py==3.13.0 from > file:///home/blaise/Development/firedrake/src/petsc4py from build tracker > '/tmp/pip-req-tracker-jo7ho6yz' > Building wheels for collected packages: petsc4py > Created temporary directory: /tmp/pip-wheel-0o0xldo2 > Building wheel for petsc4py (setup.py): started > Destination directory: /tmp/pip-wheel-0o0xldo2 > Running command /home/blaise/Development/firedrake/bin/python -u -c > 'import sys, setuptools, tokenize; sys.argv[0] > = '"'"'/tmp/pip-req-build-r3agmgty/setup.py'"'"'; > __file__='"'"'/tmp/pip-req-build-r3agmgty/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', > open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > bdist_wheel -d /tmp/pip-wheel-0o0xldo2 > running bdist_wheel > running build > running build_src > cythonizing 'petsc4py.PETSc.pyx' -> 'petsc4py.PETSc.c' > > /home/blaise/Development/firedrake/lib/python3.7/site-packages/Cython/Compiler/Main.py:369: > FutureWarning: Cython directive 'language_level' not set, using 2 for now > (Py2). This will change in a later release! File: include/petsc4py/PETSc.pxd > tree = Parsing.p_module(s, pxd, full_module_name) > cythonizing 'libpetsc4py/libpetsc4py.pyx' -> 'libpetsc4py/libpetsc4py.c' > > /home/blaise/Development/firedrake/lib/python3.7/site-packages/Cython/Compiler/Main.py:369: > FutureWarning: Cython directive 'language_level' not set, using 2 for now > (Py2). This will change in a later release! File: > /tmp/pip-req-build-r3agmgty/src/libpetsc4py/libpetsc4py.pyx > tree = Parsing.p_module(s, pxd, full_module_name) > running build_py > creating build > creating build/lib.linux-x86_64-3.7 > creating build/lib.linux-x86_64-3.7/petsc4py > copying src/PETSc.py -> build/lib.linux-x86_64-3.7/petsc4py > copying src/__init__.py -> build/lib.linux-x86_64-3.7/petsc4py > copying src/__main__.py -> build/lib.linux-x86_64-3.7/petsc4py > creating build/lib.linux-x86_64-3.7/petsc4py/lib > copying src/lib/__init__.py -> build/lib.linux-x86_64-3.7/petsc4py/lib > creating build/lib.linux-x86_64-3.7/petsc4py/include > creating build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py > copying src/include/petsc4py/numpy.h -> > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py > copying src/include/petsc4py/petsc4py.h -> > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py > copying src/include/petsc4py/petsc4py.PETSc.h -> > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py > copying src/include/petsc4py/petsc4py.PETSc_api.h -> > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py > copying src/include/petsc4py/petsc4py.i -> > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py > copying src/include/petsc4py/PETSc.pxd -> > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py > copying src/include/petsc4py/__init__.pxd -> > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py > copying src/include/petsc4py/__init__.pyx -> > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py > copying src/PETSc.pxd -> build/lib.linux-x86_64-3.7/petsc4py > copying src/lib/petsc.cfg -> build/lib.linux-x86_64-3.7/petsc4py/lib > running build_ext > PETSC_DIR: /opt/HPC/petsc-maint > PETSC_ARCH: RHEL7-intel2020.2-impi-firedrake > version: 3.13.4 release > integer-size: 32-bit > scalar-type: real > precision: double > language: CONLY > compiler: > /share/apps/intel-2020.2/compilers_and_libraries/linux/mpi/intel64/bin/mpiicc > linker: > /share/apps/intel-2020.2/compilers_and_libraries/linux/mpi/intel64/bin/mpiicc > building 'PETSc' extension > creating build/temp.linux-x86_64-3.7 > creating build/temp.linux-x86_64-3.7/RHEL7-intel2020.2-impi-firedrake > creating build/temp.linux-x86_64-3.7/RHEL7-intel2020.2-impi-firedrake/src > > /share/apps/intel-2020.2/compilers_and_libraries/linux/mpi/intel64/bin/mpiicc > -pthread -B /share/apps/intel-2020.2/intelpython3/compiler_compat > -Wl,--sysroot=/ -std=c11 -D_GNU_SOURCE -fPIC -O3 -xCASCADELAKE -g -fPIC > -L/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/lib > -I/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/include > -I/share/apps/intel-2020.2/compilers_and_libraries/linux/mpi/intel64/include > -I/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/include/eigen3 > -I/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/include > -I/opt/HPC/petsc-maint/include -Isrc/include > -I/home/blaise/Development/firedrake/lib/python3.7/site-packages/numpy/core/include > -I/home/blaise/Development/firedrake/include > -I/share/apps/intel-2020.2/intelpython3/include/python3.7m -c src/PETSc.c > -o build/temp.linux-x86_64-3.7/RHEL7-intel2020.2-impi-firedrake/src/PETSc.o > In file included from > /home/blaise/Development/firedrake/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h(12), > from > /home/blaise/Development/firedrake/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h(4), > from src/include/petsc4py/numpy.h(11), > from src/petsc4py.PETSc.c(612), > from src/PETSc.c(4): > > /home/blaise/Development/firedrake/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h(84): > warning #2650: attributes ignored here > NPY_CHAR NPY_ATTR_DEPRECATE("Use NPY_STRING"), > ^ > > In file included from src/petsc4py.PETSc.c(619), > from src/PETSc.c(4): > src/include/initpkg.h(23): warning #266: function > "PetscPartitionerInitializePackage" declared implicitly > ierr = PetscPartitionerInitializePackage();CHKERRQ(ierr); > ^ > > In file included from src/PETSc.c(4): > src/petsc4py.PETSc.c(294008): warning #266: function > "PetscPartitionerReset" declared implicitly > __pyx_t_1 = > __pyx_f_8petsc4py_5PETSc_CHKERR(PetscPartitionerReset(__pyx_v_self->part)); > if (unlikely(__pyx_t_1 == ((int)-1))) __PYX_ERR(54, 55, __pyx_L1_error) > ^ > > > /share/apps/intel-2020.2/compilers_and_libraries/linux/mpi/intel64/bin/mpiicc > -pthread -B /share/apps/intel-2020.2/intelpython3/compiler_compat > -Wl,--sysroot=/ -std=c11 -D_GNU_SOURCE -fPIC -O3 -xCASCADELAKE -g -fPIC > -L/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/lib > -I/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/include > -I/share/apps/intel-2020.2/compilers_and_libraries/linux/mpi/intel64/include > -I/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/include/eigen3 > -I/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/include > -I/opt/HPC/petsc-maint/include -Isrc/include > -I/home/blaise/Development/firedrake/lib/python3.7/site-packages/numpy/core/include > -I/home/blaise/Development/firedrake/include > -I/share/apps/intel-2020.2/intelpython3/include/python3.7m -c > src/libpetsc4py.c > -o > build/temp.linux-x86_64-3.7/RHEL7-intel2020.2-impi-firedrake/src/libpetsc4py.o > In file included from > /home/blaise/Development/firedrake/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h(12), > from > /home/blaise/Development/firedrake/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h(4), > from src/include/petsc4py/numpy.h(11), > from src/libpetsc4py/libpetsc4py.c(612), > from src/libpetsc4py.c(6): > > /home/blaise/Development/firedrake/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h(84): > warning #2650: attributes ignored here > NPY_CHAR NPY_ATTR_DEPRECATE("Use NPY_STRING"), > ^ > > creating > build/lib.linux-x86_64-3.7/petsc4py/lib/RHEL7-intel2020.2-impi-firedrake > > /share/apps/intel-2020.2/compilers_and_libraries/linux/mpi/intel64/bin/mpiicc > -pthread -B /share/apps/intel-2020.2/intelpython3/compiler_compat > -Wl,--sysroot=/ -std=c11 -D_GNU_SOURCE -fPIC -O3 -xCASCADELAKE -g -shared > -L/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/lib > -I/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/include > -L/share/apps/intel-2020.2/intelpython3/lib > -Wl,-rpath=/share/apps/intel-2020.2/intelpython3/lib -Wl,--no-as-needed > -Wl,--sysroot=/ > build/temp.linux-x86_64-3.7/RHEL7-intel2020.2-impi-firedrake/src/PETSc.o > build/temp.linux-x86_64-3.7/RHEL7-intel2020.2-impi-firedrake/src/libpetsc4py.o > -L/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/lib > -Wl,-rpath,/opt/HPC/petsc-maint/RHEL7-intel2020.2-impi-firedrake/lib > -lpetsc -o > build/lib.linux-x86_64-3.7/petsc4py/lib/RHEL7-intel2020.2-impi-firedrake/ > PETSc.cpython-37m-x86_64-linux-gnu.so > writing build/lib.linux-x86_64-3.7/petsc4py/lib/petsc.cfg > installing to build/bdist.linux-x86_64/wheel > running install > running install_lib > creating build/bdist.linux-x86_64 > creating build/bdist.linux-x86_64/wheel > creating build/bdist.linux-x86_64/wheel/petsc4py > copying build/lib.linux-x86_64-3.7/petsc4py/PETSc.py -> > build/bdist.linux-x86_64/wheel/petsc4py > copying build/lib.linux-x86_64-3.7/petsc4py/__init__.py -> > build/bdist.linux-x86_64/wheel/petsc4py > copying build/lib.linux-x86_64-3.7/petsc4py/__main__.py -> > build/bdist.linux-x86_64/wheel/petsc4py > creating build/bdist.linux-x86_64/wheel/petsc4py/lib > copying build/lib.linux-x86_64-3.7/petsc4py/lib/__init__.py -> > build/bdist.linux-x86_64/wheel/petsc4py/lib > copying build/lib.linux-x86_64-3.7/petsc4py/lib/petsc.cfg -> > build/bdist.linux-x86_64/wheel/petsc4py/lib > creating > build/bdist.linux-x86_64/wheel/petsc4py/lib/RHEL7-intel2020.2-impi-firedrake > copying > build/lib.linux-x86_64-3.7/petsc4py/lib/RHEL7-intel2020.2-impi-firedrake/ > PETSc.cpython-37m-x86_64-linux-gnu.so > -> > build/bdist.linux-x86_64/wheel/petsc4py/lib/RHEL7-intel2020.2-impi-firedrake > creating build/bdist.linux-x86_64/wheel/petsc4py/include > creating build/bdist.linux-x86_64/wheel/petsc4py/include/petsc4py > copying build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py/numpy.h -> > build/bdist.linux-x86_64/wheel/petsc4py/include/petsc4py > copying build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py/petsc4py.h > -> build/bdist.linux-x86_64/wheel/petsc4py/include/petsc4py > copying > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py/petsc4py.PETSc.h -> > build/bdist.linux-x86_64/wheel/petsc4py/include/petsc4py > copying > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py/petsc4py.PETSc_api.h > -> build/bdist.linux-x86_64/wheel/petsc4py/include/petsc4py > copying build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py/petsc4py.i > -> build/bdist.linux-x86_64/wheel/petsc4py/include/petsc4py > copying build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py/PETSc.pxd > -> build/bdist.linux-x86_64/wheel/petsc4py/include/petsc4py > copying > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py/__init__.pxd -> > build/bdist.linux-x86_64/wheel/petsc4py/include/petsc4py > copying > build/lib.linux-x86_64-3.7/petsc4py/include/petsc4py/__init__.pyx -> > build/bdist.linux-x86_64/wheel/petsc4py/include/petsc4py > copying build/lib.linux-x86_64-3.7/petsc4py/PETSc.pxd -> > build/bdist.linux-x86_64/wheel/petsc4py > running install_egg_info > running egg_info > creating petsc4py.egg-info > writing petsc4py.egg-info/PKG-INFO > writing dependency_links to petsc4py.egg-info/dependency_links.txt > writing requirements to petsc4py.egg-info/requires.txt > writing top-level names to petsc4py.egg-info/top_level.txt > writing manifest file 'petsc4py.egg-info/SOURCES.txt' > reading manifest file 'petsc4py.egg-info/SOURCES.txt' > reading manifest template 'MANIFEST.in' > writing manifest file 'petsc4py.egg-info/SOURCES.txt' > Copying petsc4py.egg-info to > build/bdist.linux-x86_64/wheel/petsc4py-3.13.0-py3.7.egg-info > running install_scripts > adding license file "LICENSE.rst" (matched pattern "LICEN[CS]E*") > creating build/bdist.linux-x86_64/wheel/petsc4py-3.13.0.dist-info/WHEEL > creating > '/tmp/pip-wheel-0o0xldo2/petsc4py-3.13.0-cp37-cp37m-linux_x86_64.whl' and > adding 'build/bdist.linux-x86_64/wheel' to it > adding 'petsc4py/PETSc.pxd' > adding 'petsc4py/PETSc.py' > adding 'petsc4py/__init__.py' > adding 'petsc4py/__main__.py' > adding 'petsc4py/include/petsc4py/PETSc.pxd' > adding 'petsc4py/include/petsc4py/__init__.pxd' > adding 'petsc4py/include/petsc4py/__init__.pyx' > adding 'petsc4py/include/petsc4py/numpy.h' > adding 'petsc4py/include/petsc4py/petsc4py.PETSc.h' > adding 'petsc4py/include/petsc4py/petsc4py.PETSc_api.h' > adding 'petsc4py/include/petsc4py/petsc4py.h' > adding 'petsc4py/include/petsc4py/petsc4py.i' > adding 'petsc4py/lib/__init__.py' > adding 'petsc4py/lib/petsc.cfg' > adding 'petsc4py/lib/RHEL7-intel2020.2-impi-firedrake/ > PETSc.cpython-37m-x86_64-linux-gnu.so' > adding 'petsc4py-3.13.0.dist-info/LICENSE.rst' > adding 'petsc4py-3.13.0.dist-info/METADATA' > adding 'petsc4py-3.13.0.dist-info/WHEEL' > adding 'petsc4py-3.13.0.dist-info/top_level.txt' > adding 'petsc4py-3.13.0.dist-info/RECORD' > removing build/bdist.linux-x86_64/wheel > Building wheel for petsc4py (setup.py): finished with status 'done' > Created wheel for petsc4py: > filename=petsc4py-3.13.0-cp37-cp37m-linux_x86_64.whl > size=4287046 > sha256=95604a089946004038f4268d38948c35a5372a2cedaa9df21e354d68231cb99a > Stored in directory: > /tmp/pip-ephem-wheel-cache-5540399x/wheels/88/a6/a9/78aff17157d1fedc3a047afc3f1c84462e68ea89775758cd2e > Successfully built petsc4py > Installing collected packages: petsc4py > > Successfully installed petsc4py-3.13.0 > Removed build tracker: '/tmp/pip-req-tracker-jo7ho6yz' > > > > Any idea? > > Blaise > > > -- > A.K. & Shirley Barton Professor of Mathematics > Adjunct Professor of Mechanical Engineering > Adjunct of the Center for Computation & Technology > Louisiana State University, Lockett Hall Room 344, Baton Rouge, LA 70803, > USA > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 Web > http://www.math.lsu.edu/~bourdin > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
