I'm glad you got such a benefit. I'm sorry to hear about the incompatible linking between mpi4py and pytrilinos. pytrilinos unfortunately doesn't provide sufficient access to its communicator for us to do everything we need.
On Oct 22, 2015, at 7:48 PM, Michael Waters <[email protected]> wrote: > Jon and Trevor, > > Thank you for your answers. I've spent the afternoon here optimizing > based your suggestions and their associated documentation trails. This > gave me ~100x speed up! > > I also discovered that mpi4py and pytrilinos are not openmpi version > compatible on the cluster here. Hopefully a new version of mpi4py will > be installed soon so I may continue to parallelize. > > Thank you, > -mike waters > > On 10/22/2015 11:50 AM, Guyer, Jonathan E. Dr. wrote: >> Mike - >> >> The most expedient way to get this normalization is >> >> rho / (rho.cellVolumeAverage * mesh.cellVolumes.sum()) >> >> `cellVolumeAverage` and `cellVolumes.sum()` automatically take care of the >> MPI communication and account for not double-counting the overlaps between >> partitions. >> >> Also, if you use CylindricalGrid2D, then mesh.cellVolumes will automatically >> account for the radial variation. >> >> You shouldn't ever find yourself looping over cells. Let FiPy (which lets >> numpy) do that. >> >> - Jon >> >> On Oct 20, 2015, at 3:22 PM, Michael Waters <[email protected]> wrote: >> >>> Hello, I am trying to implement a parallel normalization function. Right >>> now I have something like this: >>> >>> rc, zc = mesh.cellCenters >>> dv = 2.0*pi*rc*dr*dz # cylindrical mesh >>> >>> >>> def normalize(rho): >>> factor = 1.0 / ((rho*dv).globalValue).sum() >>> >>> for i in range(rho.shape[0]): >>> rho[i] = rho[i]*factor >>> >>> Is there an efficient way I can do this where I don't need to reference the >>> individual cells (and all the associated MPI communication for each node)? >>> Can the summations for the normalizing factor be on each node then >>> collected? I am very new to parallel FiPy and would love any feedback. >>> >>> Thanks, >>> -Mike Waters >>> >>> >>> >>> >>> >>> _______________________________________________ >>> fipy mailing list >>> [email protected] >>> http://www.ctcms.nist.gov/fipy >>> [ NIST internal ONLY: https://email.nist.gov/mailman/listinfo/fipy ] >> >> _______________________________________________ >> fipy mailing list >> [email protected] >> http://www.ctcms.nist.gov/fipy >> [ NIST internal ONLY: https://email.nist.gov/mailman/listinfo/fipy ] > > _______________________________________________ > fipy mailing list > [email protected] > http://www.ctcms.nist.gov/fipy > [ NIST internal ONLY: https://email.nist.gov/mailman/listinfo/fipy ] _______________________________________________ fipy mailing list [email protected] http://www.ctcms.nist.gov/fipy [ NIST internal ONLY: https://email.nist.gov/mailman/listinfo/fipy ]
