> The concept of ‘local_size’ is a bit vague (the PETSc doc cautions the use >>>>> of VecGetLocalSize), e.g. should it contain ghost values or not? >>>>> >>>> >>>> To me it seems like VecGetLocalSize only returns the local dofs. >>>> >>> >>> It returns PETSc's concept of the local size. It breaks abstractions >>> because it makes an assumption on the underlying storage which is not >>> necessarily valid for all implementations. >>> >> >> What do you mean with different implementations? Different usage? >> > > Implementations. The 'feature' of PETSc, Tpetra, etc is that they abstract > away details of the internal representation/storage. For example, the > concept of 'local size' is different for Tpetra and PETSc.
Ok, so your concept of implementation is between PETSc and Trilinos, not within present DOLFIN, as Tpetra is not in DOLFIN, yet. You had me pretty confused for a while there. But the local_range is always fixed (as long as we do not repartition), >> right? >> > > We don't have a consistent concept of local_range. We could go with 'all > values present on a process'. This would include ghosts/shared entries. I thought we had and that GenericTensor::local_range, and GenericDofMap::ownership_range gave these values. I am talking about present implementation of local_values in dolfin and not any future definitions. I am aiming of implementing a feature for present pre 1.5 release ;) If so we can assume that only dofs within the local range can be set, at >> least from the numpy interface. These dofs are set using local numbering by >> the method set_local. Aren't, at least for now, ghost values stored on top >> of the local dof range, so if we keep our indices below >> local_range[1]-local_range[0] we should be fine? >> >> You need to first get the ghosted vector by VecGhostGetLocalForm and >>>> then call VecGetSize on that to also get the size of the local + ghosted >>>> vector. It also seems like local_size is the same as >>>> local_range[1]-local_range[0] regardless of the size of the local dofs and >>>> the present of any ghosted dofs. >>>> >>>> Can you give an example where the ghost dofs are set using set_local? >>>> >>>> >>> During assembly. set_local(const double*, std::size_t, const >>> dolfin::la_index*) takes local indices - the actual data can reside >>> elsewhere. >>> >> >> So local size will vary during assemble? >> >> When a vector is created, a local-to-global map for the vector is set. >>> >> easy way for providing the local size, >> And then fixing the local_size? >> > > > No, because the local-to-global map can contain off-process entries. The > local indices are in [0, n), but the target entry may reside on a different > process, e.g. [0, m) entries on the process, and [m, n) on another process. Sure. My understanding of Tepetra is that it doesn’t have a concept of >>>>> ‘ownership’, i.e. vectors entries can be stored on more than one process >>>>> and are updated via a function call. No one process is designated as the >>>>> ‘owner’. >>>>> >>>> >>>> So you have shared dofs instead of dedicated owned and ghosted dofs? >>>> >>> >>> Yes. >>> >>> That will of course make things more complicated... >>>> >>> >>> In terms of interfacing to NumPy, yes. At a lower level in DOLFIN I >>> think it will makes things simpler. >>> >> >> Ok, are we going to change dolfin's concept of owing a dofs (think what >> we do in the dofmap) when tepetra will be added? >> > > This is a different topic because it's not pure linear algebra - we can > decide how dof ownership is defined. > > I'm leaning towards 'local' meaning all available entries on a process, > which in cases will mean duplication for shared/ghost values. Yeah, I figured that. I do not care. As long as it is transparent for the user and we provides means to figure out the local indices of the local ghost value. Then a user can set vector values using v[indices], which eventually just calls GenericVector::set_local. We also need to provide a robust way of figuring out the local_size, (what ever that means) via for example GenericVector::local_size. Johan
_______________________________________________ fenics mailing list [email protected] http://fenicsproject.org/mailman/listinfo/fenics
