Both _p_Mat and _p_Vec have void pointers that can be used for the gpu
memory as well as flags that could potentially be used. See below
struct _p_Mat {
PETSCHEADER(struct _MatOps);
….
PetscBool nooffprocentries,nooffproczerorows;
#if defined(PETSC_HAVE_CUSP)
PetscCUSPFlag valid_GPU_matrix; /* flag pointing to the matrix on
the gpu*/
#endif
#if defined(PETSC_HAVE_VIENNACL)
PetscViennaCLFlag valid_GPU_matrix; /* flag pointing to the matrix
on the gpu*/
#endif
void *spptr; /* pointer for special library like
SuperLU */
On Aug 27, 2013, at 12:20 AM, Harshad Sahasrabudhe <[email protected]> wrote:
> Hi Barry,
>
> Thanks, MatGetFactor() now much more clear.
>
> However, I have run into trouble with the device memory allocation for
> Mat_SeqDense. How do I figure out whether the allocated memory is on host or
> device in MatSeqDenseSetPreallocation_SeqDense() ?
>
> For some functions such as magma_xgetrf_gpu and magma_xgetrs_gpu, I need the
> memory allocated on the device. As the preallocation is done in
> MatSeqDenseSetPreallocation_SeqDense for dense Matrices, I want to write a
> similar function. But checking for if(!data) is not enough as I need to make
> sure that the allocated memory is on the device.
>
> Is it a good idea to make another Mat class for magma which has 2 pointers,
> one for the host memory and one for device memory (only one of which should
> point to data at any given time) ?
>
> Harshad
>
> ----- Original Message -----
> From: "Barry Smith" <[email protected]>
> To: "Harshad Sahasrabudhe" <[email protected]>
> Cc: "For users of the development version of PETSc" <[email protected]>
> Sent: Monday, August 26, 2013 6:19:21 PM
> Subject: Re: [petsc-dev] Adding Seq Dense MAGMA Mat
>
>
> On Aug 26, 2013, at 4:24 PM, Harshad Sahasrabudhe <[email protected]> wrote:
>
>> Hi Barry,
>>
>> Here is my analysis, please correct if I'm wrong:
>>
>> For using the MatGetFactor() mechanism, I need to define MATSOLVERMAGMA in
>> petscmat.h. I also need to define the functions
>> MatFactorGetSolverPackage_SeqDense_magma(), MatGetFactor_SeqDense_magma() in
>> dense.c
>>
>> I'm not creating a new matrix class, so I think I don't need to add anything
>> to MatRegisterAll()
>>
>> I don't understand the following:
>> How does PETSc figure out that there is an additional solver present? Do I
>> need to define any other variables or add anything for this?
>
> In MatCreate_SeqDense() note the line
> ierr =
> PetscObjectComposeFunction((PetscObject)B,"MatGetFactor_petsc_C",MatGetFactor_seqdense_petsc);CHKERRQ(ierr);
>
> you will add an additional line (protected by ifdef for magma being available
> that registers your routine
>
> ierr =
> PetscObjectComposeFunction((PetscObject)B,"MatGetFactor_magma_C",MatGetFactor_seqdense_magma);CHKERRQ(ierr);
>
> Barrt
>
>>
>
>> What else do I need to change if I have to add the functions
>> MatSolve_SeqDense_magma() and MatMatMult_magma() (for magma_xgemm dense
>> matrix multiplication).
>>
>> Harshad
>>
>>
>> ----- Original Message -----
>> From: "Barry Smith" <[email protected]>
>> To: "Harshad Sahasrabudhe" <[email protected]>
>> Cc: "For users of the development version of PETSc" <[email protected]>
>> Sent: Tuesday, August 20, 2013 7:10:40 PM
>> Subject: Re: [petsc-dev] Adding Seq Dense MAGMA Mat
>>
>>
>> How general do you plan for this "magma" matrix class to be? If it all it
>> is for is to do LU/Cholesky factorizations then you do NOT need to introduce
>> src/mat/impls/dense/seq/magma at all. Simply use the MatGetFactor()
>> mechanism to "How will the user be able to access this function?". This is
>> definitely the easy way to go.
>>
>> If you want to use the "magma" matrix class for all kinds of
>> non-factorization matrix operations then you need to write a full magma
>> matrix class. I don't recommend this.
>>
>> Barry
>>
>> On Aug 20, 2013, at 1:35 PM, Harshad Sahasrabudhe <[email protected]>
>> wrote:
>>
>>> Hi,
>>>
>>> I am working on adding the Seq Dense MAGMA matrix type. For this, I have
>>> created the following files and directories:
>>>
>>> src/mat/impls/dense/seq/magma
>>> src/mat/impls/dense/seq/magma/magma.c
>>> src/mat/impls/dense/seq/magma/makefile
>>>
>>> Right now I am just trying the make LU factorization work through MAGMA.
>>> Barry had suggested looking at dense.c, as the MAGMA function for LU has
>>> the same sequence as the LAPACK function getrf. MAGMA does the memory
>>> allocation on the GPU inside it's function magma_?getrf. So can I directly
>>> use the matrix type declared in dense.c, and just define a function
>>> MatLUFactor_SeqDense_MAGMA magma.c which uses the same matrix type?
>>>
>>> How will the user be able to access this function?
>>>
>>> Thanks,
>>> Harshad
>>
>
