It looks like the semantics of this call are different from localize with
the send_list:
the former gets the whole vector redundantly on every processor, while the
latter only
updates the ghosts, resulting in a local vec of a very different size, as
far as I understand.
I'm not sure when in the libmesh context it is appropriate to get full
redundant localization,
but whenever that is it should be done with a different call to PETSc:
VecScatterCreateg(xvec, PETSC_NULL, yvec, PETSC_NULL, &scatter)
with yvec having PETSC_COMM_SELF.
That way a special case VecScatter_MPI_to_ToAll is built, which might be
quite a bit
faster than doing the same with point-to-point communication.
Dmitry.
On Tue, Feb 7, 2012 at 8:34 PM, Roy Stogner <royst...@ices.utexas.edu>wrote:
>
> On Tue, 7 Feb 2012, Derek Gaston wrote:
>
> In System::project_solution() it looks like we're doing a full
>> localization of the solution vector! Here is the offending line
>> of code:
>>
>> solution->localize(*current_**local_solution);
>>
>> Shouldn't this either be a call to the version of localize that takes the
>> send_list? Or more likely just call update()...
>>
>
> At first glance I'd say yes - this looks like a huge bug in the
> original version of this function. I'll wait for Ben to weigh in,
> though.
>
> Shouldn't the (later) introduction of ghosted PetscVectors have
> ameliorated the problem? They've got all the send_list information
> implicitly built-in to the vector. It seems like it should be
> possible to avoid the excess communication even when users don't
> provide a send_list. Or is it that they have the send_list
> information on sending processors, only on receiving processors, so
> sending processors can't pre-filter what they send?
>
>
> Some of my huge runs are dying here (trying to do a full localization of
>> 400 Million DoFs is not working out!).
>>
>
> We're within 1 OOM of hitting 32-bit int based bugs, huh? Nice.
> ---
> Roy
>
> ------------------------------------------------------------------------------
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-d2d
> _______________________________________________
> Libmesh-devel mailing list
> Libmesh-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>
>
------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
_______________________________________________
Libmesh-devel mailing list
Libmesh-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libmesh-devel
