"Eray Ozkural (exa)" wrote: > Hi Adam, > > On Thu, Apr 12, 2001 at 02:31:07PM -0400, Adam C Powell IV wrote: > > Greetings, > > > > I think most of us do distributed computation, but what about > > distributed visualization? > > > > About unstructured grids: I think you can't do that with PetSC because > PetSC has no idea of sparsity.
Hmm, I'm not quite sure what you mean by "sparsity". PETSc has some very nice distributed sparse matrix solvers, though it really doesn't have an object which is dynamically resizable on each node, which would be helpful for an unstructured grid. This is one reason I haven't done the communications part yet, that is, sending all of the triangle data to node 0. On the other hand, one can create the triangle data on each node, then create a vector with the different sizes on each node to handle the data, send everything to node 0, and destroy the vector. It's inelegant, but should work. I think I'll try that. I'll try to put up 0.1 this weekend before implementing this (just got some showstopper automake/PETSc issues settled), then do this for 0.2. > A friend of mine did parallel volume > visualization on irregular grids but that requires a communication > volume model which is a hypergraph and a hypergraph partitioner to > reach the desired effect; anything else would be wasting the hardware. You're absolutely right. PETSc has an interface with ParMETIS for this purpose. I don't know much about it or its data structures at this point, but plan to learn soon, and maybe package ParMETIS for Debian, and make at least petsc2.0.29-dev depend on it... (Not that PETSc is the only thing out there, but I don't know of other Newton-Krylov solvers which scale so well, and with it I don't have to learn MPI or PVM. :-) > Making that real-time for a timestepping simulation would be a great > challenge. What's interesting is that the time required to do a single (semi-)implicit timestep can be much greater than the time required to repartition the mesh! See for example a writeup of a Lagrangian-Eulerian model of blood cell deformation in a flow field at: http://www.cs.cmu.edu/~oghattas/papers/sc2000/sc2000.pdf They actually repartition during each timestep! (Using a much more primitive algorithm than ParMETIS.) But visualizing this should be pretty trivial, just loop through the elements to generate triangles, just as I currently loop through the finite difference grid as if it were made of linear hexahedral elements. Unless I'm misunderstanding what your friend did... By the way, Drake, thanks for the CUMULVS tip, I found the web page at: http://www.epm.ornl.gov/cs/cumulvs.html The fault tolerance and automatic recovery part sounds very interesting. But I don't think the visualization part is quite what I'm looking for. It sounds like their approach to visualization is to slurp *all* the data from the cluster onto one node (or even to multiple viewer front-ends!), then have some visualization package digest it all and display it. I'd like to push as much visualization processing as possible out to the large array of computation nodes where the data live, then sip as little data as possible back to one node to display it. Otherwise, your massive supercluster has moved forward several timesteps in the time your at-most-dual-CPU visualization station has taken to display one! HARNESS (http://www.epm.ornl.gov/harness/) is also interesting, but the only visualization reference on their homepage is a link to CUMULVS... Thanks for the rapid feedback! -Adam P. GPG fingerprint: D54D 1AEE B11C CE9B A02B C5DD 526F 01E8 564E E4B6 Welcome to the best software in the world today cafe!
