On May 8, 2010, at 7:36 PM, jtg wrote:
> 1) At the moment, it seems that the lack of MPI symbols in > libpytrilinos is the most fundamental issue I don't *know* that the lack of symbols is the problem. It was the first difference I noted between *my* builds of serial-Trilinos-10.2.0 and parallel-Trilinos-10.2.0. It may not be a significant thing to look for in Trilinos 9. > 2) Bear with me just a moment. We have an mpi4py that > "works" -- that is, this very simple-minded program behaves > as expected (and others, not so simple-minded) Just to confirm, by "works", you mean you've run this with `mpirun -np <something-greater-than-one>` and you get Aloha, World. This proc= 1 (of <something-greater-than-one>) is running on machine= eeny. Aloha, World. This proc= 2 (of <something-greater-than-one>) is running on machine= meeny. etc. correct? > 3) HOW DOES PyTrilinos KNOW ABOUT mpi4py? It doesn't. FiPy needs mpi4py. PyTrilinos links to MPI directly. > All the trilinos configuration flags have to do with "mpi"; there > are no python stubs of any kind in my release of OpenMPI. > there are cryptic/generic import statements in the PyTrilinos > files, but nowhere is anything like mpi4py explicitly called out, > that I have been able to find. PyTrilinos uses swig to get the access it needs into OpenMPI. It does not rely on any (non-existent) Python stubs in OpenMPI or on mpi4py. FiPy requires mpi4py because there are a couple of operations we need (allgather and bcast) that are not exposed by PyTrilinos, at least as far was we could figure. > I use only 2 mpi flags when configuring trilinos > --with-mpi=[...]/trilinos/OPENMPI \ > --with-mpi-compilers \ You don't pass '--enable-mpi'? That could be the issue. > > I believe that the code implementing the bindings between > mpi4py and (standard) MPI is located in a lib called MPI.so; > which, at our facility, after an standard install of mpi4py, > wound up here > [our_python_area]/site-packages/mpi4py/MPI.so > > (As mentioned, this is the same "site-packages" containing > things like numpy and fipy). Certainly one conceivable issue might be if mpi4py and PyTrilinos are built against different MPI libraries. However, I would expect this to generate an error (or nonsense), not two independent FiPy processes. Likewise, if mpi4py is not present, FiPy just fails with an ImportError.
