Resending from an email address that is accepted by libmesh-devel
On Mon, Feb 13, 2012 at 14:41, Jed Brown <jedbr...@mcs.anl.gov> wrote:
> On Mon, Feb 13, 2012 at 12:11, Cody Permann <codyperm...@gmail.com> wrote:
>
>> This is to be expected - if you think about what happens in assembly we
>> are looping over elements and building up just a "few" values at a time for
>> our local element matrices. We then scatter those into the PETSc vector.
>> The startup cost and speed of the GPU processors means that you need to be
>> doing hundreds or thousands of computations to overcome the memory transfer
>> costs. I've done preliminary work in libMesh showing that this "can"
>> be efficiently but it is non-trivial and will require significant changes
>> in the way these algorithms are designed.
>>
>
> Code, this user is still assembling on the CPU, just into a matrix format
> that will do linear algebra on the GPU. The problem, as I explained in the
> thread on the PETSc list (that was then forwarded) is that preallocation
> information is being lost because the matrix type is changed after
> preallocation information is given. The solution is to change the calls to
> MatCreateMPIAIJ()/MatCreateSeqAIJ()/etc to the following. This one piece of
> code can be used for all formats (no need to duplicate the calls, you can
> call all the preallocation routines and only the one that applies to that
> matrix format will be used). I can prepare a patch eventually, but I would
> have to build libmesh on a machine with CUDA to test and I don't have time
> to work on that before SIAM PP.
>
> ierr = MatCreate(comm,A);CHKERRQ(ierr);
> ierr = MatSetSizes(*A,m,n,M,N);CHKERRQ(ierr);
> ierr = MPI_Comm_size(comm,&size);CHKERRQ(ierr);
> ierr = MatSetType(*A,MATAIJ);CHKERRQ(ierr);
> ierr = MatSetOptionsPrefix(*A,optional_prefix);CHKERRQ(ierr);
> ierr = MatSetFromOptions(*A);CHKERRQ(ierr);
> ierr = MatMPIAIJSetPreallocation(*A,d_nz,d_nnz,o_nz,o_nnz);CHKERRQ(ierr);
> ierr = MatSeqAIJSetPreallocation(*A,d_nz,d_nnz);CHKERRQ(ierr);
>
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Libmesh-devel mailing list
Libmesh-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libmesh-devel
