On Tue, May 20, 2014 at 12:57 AM, Andrew Cramer <[email protected]>wrote:
> On 20 May 2014 12:27, Matthew Knepley <[email protected]> wrote: > >> On Mon, May 19, 2014 at 8:50 PM, Andrew Cramer < >> [email protected]> wrote: >> >>> Hi All, >>> >>> I'm new to PETSc and would like to use it as my linear elasticity solver >>> within a structural optimisation program. Originally I was using GP-GPUs >>> and CUDA for my solver but I would like to shift to using PETSc to leverage >>> it's breadth of trustworthy solvers. We have some SMP servers and a couple >>> compute clusters (one with GPUs, one without). I've been digging through >>> the docs and I'd like some feedback on my plan and perhaps some pointers if >>> at all possible. >>> >>> The plan is to keep the 6000 lines or so of current code and try as much >>> as possible to use PETSc as a 'drop-in'. This would require giving one >>> field (array) of densities and receiving a 3d field (array) of >>> displacements back. Providing the density field would be easy with the >>> usual array construction functions on one node/process but pulling the >>> displacements back to the 'controlling' node would be difficult. >>> >>> I understand that this goes against the ethos of PETSc which is >>> distributed all the way. My code is highly modular with differing objective >>> functions and optimisers (some of which are written by other research >>> groups) that I drop in and pull out. I don't want to throw all that away. I >>> would need to relearn object oriented programming within PETSc (currently I >>> use c++) and rewrite my entire code base. In terms of performance the >>> optimisers typically rely heavily on tight loops of reductions once the >>> solve is completed so I'm not sure that the speed-up would be too great >>> rewriting them as distributed anyway. >>> >>> Sorry for the long winded post but I'm just not sure how to move >>> forward, I'm sick of implementing every solver I want to try in CUDA >>> especially knowing that people have done it better than I can in PETSc. But >>> it's a framework that I don't know how to interface with, all the examples >>> seem to have the solve as the main thing rather than one part of a broader >>> program. >>> >> >> 1) PETSc can do a good job on linear elasticity. GAMG is particularly >> effective, and we have an example: >> >> >> http://www.mcs.anl.gov/petsc/petsc-dev/src/ksp/ksp/examples/tutorials/ex56.c.html >> >> 2) You can use this function to go back and forth from 1 process >> >> >> http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Vec/VecScatterCreateToZero.html >> >> 3) The expense of pushing all that data to nodes can large. You might be >> better off just using GAMG on 1 process, which is how I would start. >> >> Matt >> >> >>> Andrew Cramer >>> University of Queensland, Australia >>> PhD Candidate >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > Thanks for your help, I was eyeing off ksp/ex29 as it uses DMDA which I > thought would simplify things. I'll take a look at ex56 instead and see > what I can do. > If you have a completely structured grid, DMDA is definitely simpler, although it is a little awkward for cell-centered discretizations. We have some really new support for arbitrary discretizations on top of DMDA, but it is alpha. Thanks, Matt > Andrew > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
