How do you specify how to build PETSc in your HashDist profile? Are you using 'download' to allow PETSc to download and build any packages or are you only using 'build_with'?
I do not think the warning about MPI_DIR has anything to do with this. Johannes On Mon, Oct 5, 2015 at 3:19 PM Johannes Neumann <[email protected]> wrote: > Hello, > > I have an update here as I ran into the problem again, where the wrong > parmetis is used by petsc. I tried a couple of different things (static > build, tell petsc to download parmetis on its own etc pp.) but this did not > help. I think the problem with this is connected to the following warning > that I get during the petsc buid in the log: > > 2015/10/05 15:09:41 - INFO: [package:run_job] > =============================================================================== > 2015/10/05 15:09:41 - INFO: [package:run_job] ***** > WARNING: MPI_DIR (set to /Home/flow/neumann/.hashdist/bld/mpi/gmkx3ftk3tz2) > found in environment variables - ignoring > 2015/10/05 15:09:41 - INFO: [package:run_job] use > ./configure MPI_DIR=$MPI_DIR if you really want to use that value ****** > 2015/10/05 15:09:41 - INFO: [package:run_job] > =============================================================================== > > I couldn't find anything related to that variable in the petsc.yaml or the > accompanying .py script. So I gues that is why Petsc uses the system > parmetis and not the locally build one. As I understand, I will have to add > this in the py. script. I will try to do it myself locally but help would > be appreciated. > > Best, > Jo > > > Am 25.08.2015 um 18:49 schrieb Johannes Ring: > > Yes, mumps will not be available, but we can try to add it back later. > > Johannes > > On Tue, Aug 25, 2015 at 6:41 PM Johannes Neumann <[email protected]> > wrote: > > ok i will try it now. thanks for your help. this means that mumps for >> example will not be available? I don't need it at the moment so just, that >> I know. >> >> >> Am 25.08.2015 um 18:35 schrieb Johannes Ring: >> >> Try to remove this part from the petsc build: >> >> download: | >> mumps, scalapack, blacs, ml >> >> I think this could be the problem. >> >> Johannes >> >> On Tue, Aug 25, 2015 at 6:26 PM Johannes Neumann <[email protected]> >> wrote: >> >> oh, all right ;) here it is. >>> >> >>> >>> Am 25.08.2015 um 18:24 schrieb Johannes Ring: >>> >> [Please keep fenics-support in Cc] >>> >>> Thanks Jo, but I was thinking about the yaml file, where you specify the >>> packages you want to build. >>> >>> Johannes >>> >>> On Tue, Aug 25, 2015 at 6:13 PM Johannes Neumann <[email protected]> >>> wrote: >>> >>> >>>> >>>> >>>> >>>> >>>> I compressed the profile without most of the libs and shared >>>> data/header files. If you need any of these, please let me know. You >>>> can download it here: >>>> >>>> >>>> >>>> >>>> https://drive.google.com/file/d/0BybZyFhAQBNMa3pYdGk5SURKWUk/view?usp=sharing >>>> >>>> >>>> >>>> The custom profile is the default linux profile plus scipy and >>>> joblib. >>>> >>>> >>>> >>>> Best, >>>> >>>> Jo >>>> >>> >>>> >>>> >>>> >>>> Am 25.08.2015 um 14:06 schrieb Johannes >>>> Ring: >>>> >>>> >>>> >>>> >>>> Can you send us your hashdist profile? >>>> >>>> >>>> >>>> >>>> Johannes >>>> >>>> >>>> >>>> >>>> >>>> On Tue, Aug 25, 2015 at 1:27 PM Johannes Neumann >>>> <[email protected]> >>>> >>>> wrote: >>>> >>>> >>>> >>>>> >>>>> Hello, >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> we use a compute cluster with shared home dir but different >>>>> software >>>>> >>>>> running on some of the nodes. so I am using the >>>>> hashdist/hashstack >>>>> >>>>> version of fenics with works great except for when I use >>>>> other nodes >>>>> >>>>> with different library versions. The output it generates is >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> from dolfin import * >>>>> >>>>> >>>>> >>>>> File >>>>> >>>>> >>>>> "$HOME/.hashdist/bld/profile/7tifyrwdwqtd/lib/python2.7/site-packages/dolfin/__init__.py", >>>>> >>>>> line 16, in <module> >>>>> >>>>> >>>>> >>>>> from . import cpp >>>>> >>>>> >>>>> >>>>> File >>>>> >>>>> >>>>> "$HOME/.hashdist/bld/profile/7tifyrwdwqtd/lib/python2.7/site-packages/dolfin/cpp/__init__.py", >>>>> >>>>> line 42, in <module> >>>>> >>>>> >>>>> >>>>> exec("from .. import %s" % module_name) >>>>> >>>> >>>>> >>>>> >>>>> >>>>> File "<string>", line 1, in <module> >>>>> >>>>> >>>>> >>>>> File >>>>> >>>>> >>>>> "$HOME/.hashdist/bld/profile/7tifyrwdwqtd/lib/python2.7/site-packages/dolfin/cpp/common.py", >>>>> >>>>> line 32, in <module> >>>>> >>>>> >>>>> >>>>> _common = swig_import_helper() >>>>> >>>>> >>>>> >>>>> File >>>>> >>>>> >>>>> "$HOME/.hashdist/bld/profile/7tifyrwdwqtd/lib/python2.7/site-packages/dolfin/cpp/common.py", >>>>> >>>>> line 28, in swig_import_helper >>>>> >>>>> >>>>> >>>>> _mod = imp.load_module('_common', fp, pathname, >>>>> description) >>>>> >>>>> >>>>> >>>>> ImportError: >>>>> >>>>> $HOME/.hashdist/bld/petsc/cgj5xhkahatd/lib/libpetsc.so.3.5: >>>>> >>>>> undefined symbol: ParMETIS_V3_Mesh2Dual >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> And indeed, when I ldd >>>>> >>>>> the petsc library I get >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> neumann@erhard-17:~/fenics/tests> ldd >>>>> >>>>> $HOME/.hashdist/bld/petsc/cgj5xhkahatd/lib/libpetsc.so.3.5 >>>>> >>>>> >>>>> >>>>> linux-vdso.so.1 (0x00007ffc359b3000) >>>>> >>>>> >>>>> >>>>> libHYPRE-2.9.1a.so >>>>> => >>>>> >>>>> $HOME/.hashdist/bld/hypre/kh4vd5dptalu/lib/libHYPRE-2.9.1a.so >>>>> >>>>> (0x00002acfabca6000) >>>>> >>>>> >>>>> >>>>> libmpichcxx.so.10 => >>>>> >>>>> $HOME/.hashdist/bld/mpi/gmkx3ftk3tz2/lib/libmpichcxx.so.10 >>>>> >>>>> (0x00002acfac13c000) >>>>> >>>>> >>>>> >>>>> libstdc++.so.6 => /usr/lib64/libstdc++.so.6 >>>>> >>>>> (0x00002acfac35f000) >>>>> >>>>> >>>>> >>>>> libopenblas.so.0 => >>>>> >>>>> $HOME/.hashdist/bld/lapack/vwdn72phzwm3/lib/libopenblas.so.0 >>>>> >>>>> (0x00002acfac667000) >>>>> >>>>> >>>>> >>>>> libparmetis.so => >>>>> >>>>> /usr/lib64/mpi/gcc/openmpi/lib64/libparmetis.so >>>>> >>>>> (0x00002acfad417000) >>>>> >>>>> >>>>> >>>>> libmetis..so => >>>>> >>>> >>>>> >>>>> $HOME/.hashdist/bld/parmetis/hbifmb4gf2ps/lib/libmetis.so >>>>> >>>>> (0x00002acfad61b000) >>>>> >>>>> >>>>> >>>>> libX11.so.6 => /usr/lib64/libX11.so.6 >>>>> >>>>> (0x00002acfad887000) >>>>> >>>>> >>>>> >>>>> libpthread.so.0 => /lib64/libpthread.so.0 >>>>> >>>>> (0x00002acfadbc5000) >>>>> >>>>> >>>>> >>>>> libm.so.6 => /lib64/libm.so.6 >>>>> (0x00002acfadde2000) >>>>> >>>>> >>>>> >>>>> libmpichf90.so.10 => >>>>> >>>>> $HOME/.hashdist/bld/mpi/gmkx3ftk3tz2/lib/libmpichf90.so.10 >>>>> >>>>> (0x00002acfae0e3000) >>>>> >>>>> >>>>> >>>>> libgfortran.so.3 => /usr/lib64/libgfortran.so.3 >>>>> >>>>> (0x00002acfae2e6000) >>>>> >>>>> >>>>> >>>>> libquadmath.so.0 => /usr/lib64/libquadmath.so.0 >>>>> >>>>> (0x00002acfae600000) >>>>> >>>>> >>>>> >>>>> libdl.so..2 => /lib64/libdl.so.2 >>>>> (0x00002acfae83c000) >>>>> >>>>> >>>>> >>>>> libmpich..so.10 => >>>>> >>>> >>>>> >>>>> $HOME/.hashdist/bld/mpi/gmkx3ftk3tz2/lib/libmpich.so.10 >>>>> >>>>> (0x00002acfaea40000) >>>>> >>>>> >>>>> >>>>> libopa.so.1 => >>>>> >>>>> $HOME/.hashdist/bld/mpi/gmkx3ftk3tz2/lib/libopa.so.1 >>>>> >>>>> (0x00002acfaeeaf000) >>>>> >>>>> >>>>> >>>>> libmpl.so.1 => >>>>> >>>>> $HOME/.hashdist/bld/mpi/gmkx3ftk3tz2/lib/libmpl.so.1 >>>>> >>>>> (0x00002acfaf0b1000) >>>>> >>>>> >>>>> >>>>> librt.so..1 => /lib64/librt.so.1 >>>>> (0x00002acfaf2b7000) >>>>> >>>>> >>>>> >>>>> libgcc_s..so.1 => /lib64/libgcc_s.so.1 >>>>> >>>> >>>>> >>>>> (0x00002acfaf4bf000) >>>>> >>>>> >>>>> >>>>> libc.so.6 => /lib64/libc.so.6 >>>>> (0x00002acfaf6d6000) >>>>> >>>>> >>>>> >>>>> /lib64/ld-linux-x86-64.so.2 (0x00002acfaa8ee000) >>>>> >>>>> >>>>> >>>>> libptscotch.so.0 => >>>>> >>>>> /usr/lib64/mpi/gcc/openmpi/lib64/libptscotch.so.0 >>>>> >>>>> (0x00002acfafa7d000) >>>>> >>>>> >>>>> >>>>> libptscotcherr.so.0 => >>>>> >>>>> /usr/lib64/mpi/gcc/openmpi/lib64/libptscotcherr.so.0 >>>>> >>>>> (0x00002acfafd5b000) >>>>> >>>>> >>>>> >>>>> libz.so.1 => /lib64/libz.so.1 >>>>> (0x00002acfaff5d000) >>>>> >>>>> >>>>> >>>>> libmpi.so.1 => >>>>> >>>>> /usr/lib64/mpi/gcc/openmpi/lib64/libmpi.so.1 >>>>> (0x00002acfb0173000) >>>>> >>>>> >>>>> >>>>> libxcb.so.1 => /usr/lib64/libxcb.so.1 >>>>> >>>>> (0x00002acfb0443000) >>>>> >>>>> >>>>> >>>>> libopen-rte.so.5 => >>>>> >>>>> /usr/lib64/mpi/gcc/openmpi/lib64/libopen-rte.so.5 >>>>> >>>>> (0x00002acfb0663000) >>>>> >>>>> >>>>> >>>>> libopen-pal.so.5 => >>>>> >>>>> /usr/lib64/mpi/gcc/openmpi/lib64/libopen-pal.so.5 >>>>> >>>>> (0x00002acfb08d2000) >>>>> >>>>> >>>>> >>>>> libXau.so.6 => /usr/lib64/libXau.so.6 >>>>> >>>>> (0x00002acfb0b92000) >>>>> >>>>> >>>>> >>>>> libutil.so.1 => /lib64/libutil.so.1 >>>>> >>>>> (0x00002acfb0d96000) >>>>> >>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Checking in hashdist shows, that they are build but not used >>>>> for >>>>> >>>>> some reason. I am now rebuilding with the dev version >>>>> (stable gives >>>>> >>>>> me the urllib error) and I will see, if this results in >>>>> correct >>>>> >>>>> linking. but at the moment I have a hard time getting a >>>>> stable >>>>> >>>>> connection to github. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Best, >>>>> >>>>> >>>>> >>>>> Jo >>>>> >>>>> >>>>> >>>>> -- >>>>> Dipl.-Math. Johannes Neumann >>>>> Weierstraß-Institut für Angewandte >>>>> Analysis und Stochastik >>>>> Mohrenstraße 39 >>>>> 10117 Berlin >>>>> Tel. +49 (0)30 20372 567 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> >>>>> >>>>> >>>>> fenics-support mailing list >>>>> >>>>> >>>>> >>>>> [email protected] >>>>> >>>>> >>>>> >>>>> http://fenicsproject.org/mailman/listinfo/fenics-support >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Dipl.-Math. Johannes Neumann >>>> Weierstraß-Institut für Angewandte >>>> Analysis und Stochastik >>>> Mohrenstraße 39 >>>> 10117 Berlin >>>> Tel. +49 (0)30 20372 567 >>>> >>>> >>>> >>>> >>>> >>> >>> -- >>> Dipl.-Math. Johannes Neumann >>> Weierstraß-Institut für Angewandte >>> Analysis und Stochastik >>> Mohrenstraße 39 >>> 10117 Berlin >>> Tel. +49 (0)30 20372 567 >>> >>> >>> >> -- >> Dipl.-Math. Johannes Neumann >> Weierstraß-Institut für Angewandte >> Analysis und Stochastik >> Mohrenstraße 39 >> 10117 Berlin >> Tel. +49 (0)30 20372 567 >> >> > -- > Dipl.-Math. Johannes Neumann > Weierstraß-Institut für Angewandte > Analysis und Stochastik > Mohrenstraße 39 > 10117 Berlin > Tel. +49 (0)30 20372 567 > >
_______________________________________________ fenics-support mailing list [email protected] http://fenicsproject.org/mailman/listinfo/fenics-support
