Hi,
I'm trying to run the SLEPc Eigensolver on a machine with CPUs GPUs. I
found that a package called MAGMA implements LAPACK functions on
heterogeneous systems (CPU + GPU). MAGMA has functions very similar to
LAPACK so that we can go and replace a call to a LAPACK function with a
call to
,
-
Keita Teranishi
RD Principal Staff Member
Sandia National Laboratories
Livermore, CA 94551
From: Matthew Knepley knep...@gmail.com mailto:knep...@gmail.com
Date: Thursday, June 20, 2013 1:42 PM
To: Harshad Sahasrabudhe hsaha...@purdue.edu
mailto:hsaha...@purdue.edu
Cc: petsc-users
2013, Harshad Sahasrabudhe wrote:
Hi,
Yes, MAGMA's API is not exactly the same. There are a few additional arguments
in the functions related to number of GPU etc. So it cannot be directly linked
instead of LAPACK.
Hacking SLEPc/PETSc code seems to be the easier way to go. But I can't find
Hi Jed,
Thanks for your reply.
You're on your own for storage of factors. Alternatively, you could add
library support so that you could use PCLU and
'-pc_factor_mat_solver_package magma' (or PCFactorSetMatSolverPackage).
Doing this is not a priority for us, but we can provide guidance if you
lapack you could possible
steal chunks of code from the routines I pointed you to yesterday and modify
them as needed so you don't need to reinvent the wheel.
Barry
On Jul 16, 2013, at 1:37 PM, Matthew Knepley knep...@gmail.com wrote:
On Tue, Jul 16, 2013 at 1:13 PM, Harshad Sahasrabudhe
Yes, they has the same calling sequence for LU factorization. Thanks a
lot! Will look into this.
Harshad
On 07/16/2013 03:22 PM, Barry Smith wrote:
On Jul 16, 2013, at 2:18 PM, Harshad Sahasrabudhe hsaha...@purdue.edu wrote:
Hi Barry,
I'm confused, can you please explain what you meant
Hi Jed,
I am now working to add library support for LU decomposition using MAGMA. I
need your help with the following:
1) How do I add the options --download-magma, --with-magma, etc. to the
configure script for building with MAGMA?
2) I have a fair idea how the PETSc code is structured and
Sorry, I was talking about the Python build system in PETSc. I didn't mean the
build system used by Python.
And thanks, Chaco.py does clear my doubts.
Harshad
- Original Message -
From: Matthew Knepley knep...@gmail.com
To: Harshad Sahasrabudhe hsaha...@purdue.edu
Cc: Barry Smith bsm
Oh I get what you are trying to say. Please correct me if I'm wrong. I don't
need to use the make.inc prototypes defined in MAGMA, PETSc build system gets
all the required options.
- Original Message -
From: Harshad Sahasrabudhe hsaha...@purdue.edu
To: Matthew Knepley knep...@gmail.com
Hi,
I am trying to compile my code, which uses std:complexdouble with PETSc
(Complex) compiled with CUSP. At certain places, we use references to send
vectors to PETSc for VecDot. In such a case, I get following errors:
no known conversion for argument 3 from 'cusp::complexdouble' to
Hi Matt,
I have attached a small test case which reproduces the problem. Here is the
compilation command with the error:
gcc -c test.cpp -I/u/sciteam/sahasrab/NEMO5/libs/petsc/include
-I/u/sciteam/sahasrab/NEMO5/libs/petsc/arch-linux2-cxx-opt-cplx/include/
Sorry the last line of the error should read like this:
test.cpp:9:8: note: no known conversion for argument 3 from
'cusp::complexdouble' to 'std::complexdouble'
Didn't copy it properly the last time.
Harshad
- Original Message -
From: Harshad Sahasrabudhe hsaha...@purdue.edu
Hi,
In our code, PetscInitialize() is called initially. Then, depending on whether
we need libmesh or not, LibMeshInit() is called. This works fine when we are
not using CUDA in PETSc. But when we are using CUDA, cudaSetDevice() gets
called twice which throws an error:
[0]PETSC ERROR:
Basically, PetscInitialize is called 2 times, the second time from libmesh.
PetscInitialize calls cudaSetDevice(), which fails the second time.
- Original Message -
From: Matthew Knepley knep...@gmail.com
To: Harshad Sahasrabudhe hsaha...@purdue.edu
Cc: petsc-users@mcs.anl.gov
Sent
the PETSc libraries could be
initialized with CUDA on the same process.
Do you have any idea?
I can only think of turning CUDA off on one PETSc as a way to avoid this.
Harshad
- Original Message -
From: Jed Brown jedbr...@mcs.anl.gov
To: Harshad Sahasrabudhe hsaha...@purdue.edu, Matthew
I'll try doing that. We will have to take control of cudaSetDevice in our code,
instead of PETSc.
Thanks,
Harshad
- Original Message -
From: Paul Mullowney paulmullow...@gmail.com
To: Jed Brown jedbr...@mcs.anl.gov
Cc: Harshad Sahasrabudhe hsaha...@purdue.edu, petsc-users@mcs.anl.gov
Hi,
Is there any way to convert a sparse matrix to a dense matrix? I have an almost
full Aij matrix, which I would like to convert to MatDense and offload it to a
GPU.
MatConvert(*mat, MATDENSE, MAT_INITIAL_MATRIX, temp);
mat is of type MATAIJ.
I get the error
No support for this operation
Hi,
Can we use the original Mat in Matconvert to store the converted matrix?
E.g. MatConvert(A, MATDENSE, MAT_INITIAL_MATRIX, A);
Or do I need MAT_REUSE_MATRIX flag instead of MAT_INITIAL_MATRIX?
A is initially MATAIJ.
Thanks,
Harshad
Hi Jed,
Sometime back we talked about an interface which could handle other
libraries calling cudaSetDevice simultaneously with PETSc. For example, in
our case 2 different instances of PETSc calling cudaSetDevice.
Sure, but how will we actually share the device between libraries? What
if the
Hi Karli,
Lazy initialization mechanism looks good. But the problem is that the
initialize will still happen twice if we create Vec in both our Petsc
instances.
Can we have an option for Petsc in which we can specify whether
cudaSetDevice should be run by Petsc or not?
Thanks,
Harshad
On Jan
API on your end to avoid conflicts with cudaSetDevice
calls inside PETSc?
Cheers,
Dominic
On 01/14/2014 09:27 AM, Harshad Sahasrabudhe wrote:
Hi Jed,
Sometime back we talked about an interface which could handle other
libraries calling cudaSetDevice simultaneously with PETSc
10:10:26 AM MST, Harshad Sahasrabudhe wrote:
Hi Dominic,
We use external libraries such as MAGMA and cuSPARSE. It looks like
they use the runtime API as you mentioned above. At the moment,
conflict is between the two instances of PETSc that we run (one each
for real and complex). We
Hi,
I need to scatter (distribute) a sequential vector on process 0 to all the
processes according to a know set of global indices for each process. How
can this be done using VecScatter?
Thanks,
Harshad
Hi,
I get the following error when I call MatGetType even though I'm calling
the function with a Mat object.
[0]PETSC ERROR: Invalid argument!
[0]PETSC ERROR: Wrong type of object: Parameter # 1!
What could be going wrong?
Thanks,
Harshad
() line 100 in
/home/hsahasra/NEMO5/libs/petsc/build-real/src/mat/interface/matreg.c
On Mon, Oct 20, 2014 at 4:07 PM, Harshad Sahasrabudhe hsaha...@purdue.edu
wrote:
Hi,
I get the following error when I call MatGetType even though I'm calling
the function with a Mat object.
[0]PETSC ERROR
:
On Mon, Oct 20, 2014 at 3:07 PM, Harshad Sahasrabudhe hsaha...@purdue.edu
wrote:
Hi,
I get the following error when I call MatGetType even though I'm calling
the function with a Mat object.
[0]PETSC ERROR: Invalid argument!
[0]PETSC ERROR: Wrong type of object: Parameter # 1!
What
...@gmail.com wrote:
On Mon, Oct 20, 2014 at 3:17 PM, Harshad Sahasrabudhe hsaha...@purdue.edu
wrote:
I get the Mat object from libMesh::PetscMatrix::mat()
http://libmesh.github.io/libmesh/doxygen/classlibMesh_1_1PetscMatrix.html#aaca7a5afeb063716ee67fa11302c2aea,
and it's been created and assembled
Hi,
I want to implement a Predictor-Corrector algorithm for solving a coupled
Schrödinger-Poisson system using SNES. I can calculate the next step using
my predictor algorithm and don't want PETSc to calculate the next step. I
only need PETSc to check for convergence. What is the best way to
, a las 22:06, Harshad Sahasrabudhe escribió:
I thought ARPACK was faster when the system size is large and number of
eigenvalues required is small. I will be working with sparse matrices of
size ~60,000. Does SLEPC's Krylov-Schur have about the same performance as
ARPACK for calculating ~50
, a las 21:29, Harshad Sahasrabudhe escribió:
Hi,
I'm trying to use the ARPACK interface in SLEPc for calculating smallest
eigenvalues with eigenvectors of a generalized eigenproblem. The matrices
are symmetric.
What are the suggested linear solvers/preconditioners for this type
Krylov-Schur have about the same performance as
ARPACK for calculating ~50 eigenvalues for such matrices?
On Wed, Mar 25, 2015 at 5:00 PM, Jose E. Roman jro...@dsic.upv.es wrote:
El 25/03/2015, a las 21:47, Harshad Sahasrabudhe escribió:
With MUMPS you should not get spurious eigenvalues
Hi,
I'm trying to use the ARPACK interface in SLEPc for calculating smallest
eigenvalues with eigenvectors of a generalized eigenproblem. The matrices
are symmetric.
What are the suggested linear solvers/preconditioners for this type of a
system when using ARPACK? I am using shift and invert
Hi,
I'm solving a non-linear equation using NEWTONLS. The SNES is called from a
wrapper in the LibMesh library. I'm trying to use the default FD Jacobian
by not setting any Mat or callback function for the Jacobian.
When doing this I get the following error. I'm not able to figure out why I
get
which I can read up so that I can
generate the coloring for the Jacobian myself?
Thanks,
Harshad
On Fri, Jun 5, 2015 at 1:59 PM, Matthew Knepley knep...@gmail.com wrote:
On Fri, Jun 5, 2015 at 12:32 PM, Harshad Sahasrabudhe hsaha...@purdue.edu
wrote:
Hi,
I'm solving a non-linear equation
.
Barry
Thanks,
Harshad
On Fri, Jun 5, 2015 at 1:59 PM, Matthew Knepley knep...@gmail.com
wrote:
On Fri, Jun 5, 2015 at 12:32 PM, Harshad Sahasrabudhe
hsaha...@purdue.edu wrote:
Hi,
I'm solving a non-linear equation using NEWTONLS. The SNES is called
from a wrapper
FYI, this information is contained in the libMesh::CouplingMatrix which
you can get from the libMesh::DofMap.
Awesome! Thanks!
On Fri, Jun 5, 2015 at 2:49 PM, Paul T. Bauman ptbau...@gmail.com wrote:
On Fri, Jun 5, 2015 at 2:42 PM, Harshad Sahasrabudhe hsaha...@purdue.edu
wrote
Hi,
Does PETSc have automatic differentiation capability? I'm solving an
elliptic non-linear equation in which calculating the correct Jacobian is
extremely demanding. The system doesn't converge with an approximate
Jacobian. Is there any SNES solver I should try which doesn't use a
Jacobian?
I haven't tried the FD Jacobian. How do I enable that? I have been
providing the Jacobian.
On Tue, Jun 2, 2015 at 4:59 PM, Matthew Knepley knep...@gmail.com wrote:
On Tue, Jun 2, 2015 at 3:55 PM, Harshad Sahasrabudhe hsaha...@purdue.edu
wrote:
Hi,
Does PETSc have automatic differentiation
Thanks, if the results are just slightly different with the same options,
then it's not a problem. I'll run my program using both to check.
On Wed, May 27, 2015 at 1:25 PM, Matthew Knepley knep...@gmail.com wrote:
On Wed, May 27, 2015 at 12:22 PM, Harshad Sahasrabudhe
hsaha...@purdue.edu
:07 PM, Harshad Sahasrabudhe hsaha...@purdue.edu
wrote:
Hi Matt,
Actually I just checked, I don't need to re-init the preconditioner
LibMesh solver.
The wrapper is interfaced to the mesh and makes it easier to assemble the
system matrices. It also keeps track of constraints. The wrapper
Hi,
I need to offload the linear solves in a time stepper to GPUs. I'll be
using the MAGMA sparse iterative solvers. What is the simplest way to
achieve this? Is is better to write a new KSPType or is there a KSP wrapper
which I can use?
Many thanks,
Harshad
to 1
GPU from the root process. Then distribute the result later.
Harshad
On Sat, May 30, 2015 at 10:21 PM, Barry Smith bsm...@mcs.anl.gov wrote:
On May 30, 2015, at 8:44 PM, Harshad Sahasrabudhe hsaha...@purdue.edu
wrote:
Hi,
I need to offload the linear solves in a time stepper
and 180 seconds on 16 processes. So
the scaling isn't that good. We also run out of memory with more number of
processes.
On Sat, May 30, 2015 at 11:01 PM, Jed Brown j...@jedbrown.org wrote:
Harshad Sahasrabudhe hsaha...@purdue.edu writes:
For now, I want to serialize the matrices and vectors
for the guidance. I'll get back with the answers
tomorrow.
On Sat, May 30, 2015 at 11:50 PM, Jed Brown j...@jedbrown.org wrote:
Harshad Sahasrabudhe hsaha...@purdue.edu writes:
Surely you're familiar with this.
Yes, I'm familiar with this. We are running on Intel Xeon E5 processor
Brown j...@jedbrown.org wrote:
Harshad Sahasrabudhe hsaha...@purdue.edu writes:
Is your intent to solve a problem that matters in a way that makes sense
for a scientist or engineer
I want to see if we can speed up the time stepper for a large system
using
GPUs. For large systems
Hi,
I'm trying to copy the solution and functions vecs from a Newton SNES to a
Shell SNES. I'm doing it like this:
SNES newton_snes, shell_snes;
Vec newton_function, newton_solution;
Vec shell_function, shell_solution;
.
.
Newton SNES initialized and solved here. Shell SNES
and matrices, then calls SNESSolve. Thus, I need
to call PetscNonlinearSolver::solve instead of SNESSolve in SNESApplyNPC.
Do you mean wrap it in SNESSHELL?
Thanks,
Harshad
On Tue, May 26, 2015 at 3:25 PM, Matthew Knepley knep...@gmail.com wrote:
On Tue, May 26, 2015 at 12:42 PM, Harshad
Hi,
I'm using LibMesh for meshing a non-linear equation. I want to use
NRichardson non-linear solver to solve the equation. I have a Newton trust
region non-linear solver as a pre-conditioner. Both these SNES are created
via the LibMesh NonlinearImplicitSystem and PetscNonlinearSolver wrappers.
Hi,
I'm using GAMG + GMRES for my Poisson problem. The solver converges with
KSP_CONVERGED_STEP_LENGTH at a residual of 9.773346857844e-02, which is
much higher than what I need (I need a tolerance of at least 1E-8). I am
not able to figure out which tolerance I need to set to avoid convergence
ain debugging symbols (checked using objdump -g). How do I get
PETSc library to have debugging info?
Thanks,
Harshad
On Tue, Sep 13, 2016 at 2:47 PM, Barry Smith <bsm...@mcs.anl.gov> wrote:
>
> > On Sep 13, 2016, at 1:01 PM, Harshad Sahasrabudhe <hsaha...@purdue.edu>
&
I think I found the problem. I configured PETSc with COPTFLAGS=-O3. I'll
remove that option and try again.
Thanks!
Harshad
On Wed, Sep 14, 2016 at 10:06 AM, Harshad Sahasrabudhe <hsaha...@purdue.edu>
wrote:
> Hi Barry,
>
> Thanks for your inputs. I tried to set a watchpoint on
h as a PETSc ./configure option,
> this will eliminate all the MPICH valgrind errors. Then send as an
> attachment the resulting valgrind file.
>
> I do not 100 % trust any code that produces such valgrind errors.
>
>Barry
>
>
>
> > On Sep 8, 2016, at 10:12 PM, Harsh
(argc=6, argv=0x7ffdcf2d7908) at main.cpp:447
Thanks!
Harshad
On Wed, Sep 14, 2016 at 10:10 AM, Harshad Sahasrabudhe <hsaha...@purdue.edu>
wrote:
> I think I found the problem. I configured PETSc with COPTFLAGS=-O3. I'll
> remove that option and try again.
>
> Thanks!
> Harshad
&g
Actually, I was using TR with MUMPS before I tried GAMG and it was working
pretty well. The reason I want to switch to GAMG is that I have to increase
my system size, and the simulation just takes too long with MUMPS and
doesn't scale well.
On Wed, Sep 14, 2016 at 3:09 PM, Harshad Sahasrabudhe
ze you had be using TR
> or we could have resolved this much sooner.
>
>Barry
>
>
>
> > On Sep 14, 2016, at 1:31 PM, Harshad Sahasrabudhe <hsaha...@purdue.edu>
> wrote:
> >
> > Thanks. I now put a watchpoint on
> >
> > watch *( (PetscErrorC
y you can use the same technique to track down where the
> function pointer changes. Just watch ksp->converged to see when the
> function pointer gets changed and send back the new stack trace.
>
>Barry
>
>
>
>
> > On Sep 14, 2016, at 12:39
Thank you for the reply.
> It is better to set pass the options directly to the objects. For instance
> the following example calls EPSGetST(), STGetKSP(), KSPGetPC() to extract
> objects that are nested one inside the other:
>
> Jose
>
I was only able to set PCTYPE, KSPTYPE,
Hello,
I am using PCGAMG+KSPBCGS for shift and invert spectral transformation in
SLEPc. I optimized the options that I need for GAMG using command line:
-st_ksp_type bcgs
-st_pc_type gamg
-st_pc_gamg_agg_nsmooths 1
-st_mg_levels_ksp_type chebyshev
-st_mg_levels_pc_type jacobi
>
> ierr = PetscOptionsSetValue(NULL, "-st_mg_levels_ksp_type",
>> "chebyshev");
>> chkerr(ierr);
>> ierr = PetscOptionsSetValue(NULL, "-st_mg_levels_pc_type",
>> "jacobi");
>> chkerr(ierr);
>> ierr = PetscOptionsSetValue(NULL,
Hello,
I have a set of m n-dimensional vectors and I need to find orthogonal
vectors in this subspace. This can be done using the Gram-Schmidt process.
Is there a way to set up the KSP to do this directly in PETSc?
Thanks,
Harshad
Hello,
I am solving for the lowest eigenvalues and eigenvectors of symmetric
positive definite matrices in the generalized eigenvalue problem. I am
using the GD solver with the default settings of PCBJACOBI. When I run a
standalone executable on 16 processes which loads the matrices from a file
anl.gov> wrote:
>
>
> > On May 3, 2018, at 10:24 PM, Harshad Sahasrabudhe <
> harshad.sahasrabu...@gmail.com> wrote:
> >
> > Hi Barry,
> >
> > There's an overflow in the division:
> >
> > Program received signal SIGFPE, Arithmetic exceptio
zero. Perhaps the
> code needs above a check that a is not zero. Or rewrite the check as
>
> if (d->nR[i] < a*data->fix) {
>
>Barry
>
>
> > On May 3, 2018, at 7:58 PM, Harshad Sahasrabudhe <
> harshad.sahasrabu...@gmail.com> wrote:
> >
>
davidson/common/dvd_updatev.c
static PetscErrorCode dvd_updateV_start(dvdDashboard *d)
..
124 for (i=0;ieps->ncv;i++) d->nR[i] = PETSC_MAX_REAL;
On Thu, May 3, 2018 at 11:39 PM, Harshad Sahasrabudhe <
harshad.sahasrabu...@gmail.com> wrote:
> Jumps like this usually indicate
64 matches
Mail list logo
