[petsc-users] Question about petsc4py with cuda

[MIA via petsc-users]

Hi,

I am trying to solve a large linear equation, which needs a GPU solver as 
comparison. I install a CUDA-enabled PETSc and petsc4py from sources using the 
release tarball. According to the test results after installation, the PETSc 
can successfully work with cuda. 

All my programs are written in python, so I turn to petsc4py. But I do not find 
any commands that define variables on coda device or define where the direct 
solver is executed. I check `nvidia-smi` and find my cuda does not work at all 
when executing my python script:

from petsc4py import PETSc
import numpy as np

n = 1000

nnz = 3 * np.ones(n, dtype=np.int32)
nnz[0] = nnz[-1] = 2

A = PETSc.Mat()
A.createAIJ([n, n], nnz=nnz)

# First set the first row
A.setValue(0, 0, 2)
A.setValue(0, 1, -1)
# Now we fill the last row
A.setValue(999, 998, -1)
A.setValue(999, 999, 2)

# And now everything else
for index in range(1, n - 1):
    A.setValue(index, index - 1, -1)
    A.setValue(index, index, 2)
    A.setValue(index, index + 1, -1)   
    
A.assemble()

indexptr, indices, data = A.getValuesCSR()
b = A.createVecLeft()
b.array[:] = 1
for i in range(10):
    ksp = PETSc.KSP().create()
    ksp.setOperators(A)
    ksp.setType('preonly')
    ksp.setConvergenceHistory()
    ksp.getPC().setType('lu')
    x = A.createVecRight()
    ksp.solve(2*b, x)
    residual = A * x - 2*b
    if i % 10 == 0:
        print(f"The relative residual is: {residual.norm() / b.norm()}.”)

What should I do to utlize GPU to execute the KSP task? Are there some settings 
to be modified?

Looking forward to your early reply. Thanks a lot.