Re: [petsc-users] GPUs and the float-double dilemma

[Jed Brown]

Ces VLC  writes:

> El El vie, 10 feb 2023 a las 21:44, Barry Smith  escribió:
>
>>
>>What is the use case you are looking for that cannot be achieved by
>> just distributing a single precision application? If the user is happy when
>> they happen to have GPUs to use single precision everywhere, then why would
>> they need double precision if they happen not to have a GPU?
>>
>>Are you just using KSP or also SNES, TS etc?
>
>
> Thanks for your replies. The use case is structural analysis (so, sparse
> symmetrical matrix, and minimum degree reorder tends to work fine in CPU
> (for GPU I’ll need to check the best performing scenarios).

Sparse direct solvers are rather lacking on GPUs. You may be able to use an 
algebraic multigrid on GPUs. If you're using an iterative solver, you'll be 
limited by memory bandwidth, not flops, so double -> float is at best an 8/12 
improvement.

You may be interested in this work on structural mechanics for GPUs.

https://arxiv.org/abs/2204.01722

> Obviously, this use case requires double precision. But single precision
> might be fine enough for faster low quality runs if the user happens to
> have a GPU that accelerates float and not double (I have a 12GB Pascal
> Titan, it accelerates float, not double).
>
> Kind regards,
>
> César

Re: [petsc-users] GPUs and the float-double dilemma

[Ces VLC]

El El vie, 10 feb 2023 a las 21:44, Barry Smith  escribió:

>
>What is the use case you are looking for that cannot be achieved by
> just distributing a single precision application? If the user is happy when
> they happen to have GPUs to use single precision everywhere, then why would
> they need double precision if they happen not to have a GPU?
>
>Are you just using KSP or also SNES, TS etc?


Thanks for your replies. The use case is structural analysis (so, sparse
symmetrical matrix, and minimum degree reorder tends to work fine in CPU
(for GPU I’ll need to check the best performing scenarios).

Obviously, this use case requires double precision. But single precision
might be fine enough for faster low quality runs if the user happens to
have a GPU that accelerates float and not double (I have a 12GB Pascal
Titan, it accelerates float, not double).

Kind regards,

César

Re: [petsc-users] GPUs and the float-double dilemma

[Barry Smith]

   What is the use case you are looking for that cannot be achieved by just 
distributing a single precision application? If the user is happy when they 
happen to have GPUs to use single precision everywhere, then why would they 
need double precision if they happen not to have a GPU? 

   Are you just using KSP or also SNES, TS etc?


> On Feb 10, 2023, at 7:59 AM, Ces VLC  wrote:
> 
> Hi!
> 
> I searched if it’s supported to link an application to two different builds 
> of PETSc (one configured as float precision, and the other as double). The 
> last post I found on that topic was from 2016 and it said it’s not 
> recommended.
> 
> The point is that if you wish to prepare builds of your application for 
> end-users, and if your app offers the option of using GPUs, you have a 
> critical problem if you cannot link with two different PETSc builds in the 
> same executable: either you don’t provide support for most GPUs (as they are 
> float only), or you force float precision even when using the CPU. A third 
> option (shipping two executables for the app) is not practical, as the user 
> won’t be able to compare results without quitting the app and running the 
> other version.
> 
> Has the situation changed since 2016, now that GPU support is being added to 
> PETSc?
> 
> An obvious solution would be if PETSc could be built prepending a prefix to 
> all symbols (functions, structs, types, everything).
> 
> Any advances, plans, or thoughts on this?
> 
> Thanks!
> 
> César
> 

Re: [petsc-users] GPUs and the float-double dilemma

[Matthew Knepley]

On Fri, Feb 10, 2023 at 11:31 AM Junchao Zhang 
wrote:

>
> On Fri, Feb 10, 2023 at 8:16 AM Ces VLC  wrote:
>
>> Hi!
>>
>> I searched if it’s supported to link an application to two different
>> builds of PETSc (one configured as float precision, and the other as
>> double). The last post I found on that topic was from 2016 and it said it’s
>> not recommended.
>>
>> The point is that if you wish to prepare builds of your application for
>> end-users, and if your app offers the option of using GPUs, you have a
>> critical problem if you cannot link with two different PETSc builds in the
>> same executable: either you don’t provide support for most GPUs (as they
>> are float only), or you force float precision even when using the CPU. A
>> third option (shipping two executables for the app) is not practical, as
>> the user won’t be able to compare results without quitting the app and
>> running the other version.
>>
> Why do you say most GPUs are float only?  I do not have a survey but the
> NVIDIA, AMD, Intel GPUs I have access to all support double :)
>
>
>>
>> Has the situation changed since 2016, now that GPU support is being added
>> to PETSc?
>>
>> An obvious solution would be if PETSc could be built prepending a prefix
>> to all symbols (functions, structs, types, everything).
>>
> Sounds like a bomb
>
>>
>> Any advances, plans, or thoughts on this?
>>
> Interfacing petsc with libraries (e.g.,Gingko) that support
> mixed-precision could be an approach. But we have not tried that yet.
>

The datatype on device need not match the datatype on the CPU. This is how
I prefer to do things, running float on device and double on the CPU. This
is possible now I think.

   Matt


>
>> Thanks!
>>
>> César
>>
>>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ 

Re: [petsc-users] GPUs and the float-double dilemma

[Junchao Zhang]

On Fri, Feb 10, 2023 at 8:16 AM Ces VLC  wrote:

> Hi!
>
> I searched if it’s supported to link an application to two different
> builds of PETSc (one configured as float precision, and the other as
> double). The last post I found on that topic was from 2016 and it said it’s
> not recommended.
>
> The point is that if you wish to prepare builds of your application for
> end-users, and if your app offers the option of using GPUs, you have a
> critical problem if you cannot link with two different PETSc builds in the
> same executable: either you don’t provide support for most GPUs (as they
> are float only), or you force float precision even when using the CPU. A
> third option (shipping two executables for the app) is not practical, as
> the user won’t be able to compare results without quitting the app and
> running the other version.
>
Why do you say most GPUs are float only?  I do not have a survey but the
NVIDIA, AMD, Intel GPUs I have access to all support double :)


>
> Has the situation changed since 2016, now that GPU support is being added
> to PETSc?
>
> An obvious solution would be if PETSc could be built prepending a prefix
> to all symbols (functions, structs, types, everything).
>
Sounds like a bomb

>
> Any advances, plans, or thoughts on this?
>
Interfacing petsc with libraries (e.g.,Gingko) that support mixed-precision
could be an approach. But we have not tried that yet.

>
> Thanks!
>
> César
>
>

Re: [petsc-users] Question about applying algebraic multigrid (AMG) PC and multilevel Schwarz PC

[Mark Adams]

On Fri, Feb 10, 2023 at 9:17 AM ­권승리  wrote:

> Dear petsc developers.
>
> Hello.
> While I was applying the preconditioner in contact with the FEM code, I
> had some questions.
>
> How can I apply the AMG PC and the multilevel Schwarz PC in the FEM code?
> Purpose is to compare which PC is better in the FEM code with contact
> situation.
>
> In my case, the Jacobi PC could be easily applied as shown in the example
> below. However, AMG and multilevel have too many options.
>
> ex) ./app -ksp_type gmres -pc_type jacobi -ksp_view
>

You can use the built-in AMG PC with: -pc_type gamg

Technically AMG is a  multilevel Schwarz PC, but I think you may be
referring to: -pc_type bddc

All solvers are equipped with default parameters that tend to be
optimized for robustness rather than peak performance and they are a good
place to start.

Thanks,
Mark


> Best regards
>
> Seung Lee Kwon
> --
> Seung Lee Kwon, Ph.D.Candidate
> Aerospace Structures and Materials Laboratory
> Department of Mechanical and Aerospace Engineering
> Seoul National University
> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826
> E-mail : ksl7...@snu.ac.kr
> Office : +82-2-880-7389
> C. P : +82-10-4695-1062
>

[petsc-users] Question about applying algebraic multigrid (AMG) PC and multilevel Schwarz PC

[­권승리]

Dear petsc developers.

Hello.
While I was applying the preconditioner in contact with the FEM code, I had
some questions.

How can I apply the AMG PC and the multilevel Schwarz PC in the FEM code?
Purpose is to compare which PC is better in the FEM code with contact
situation.

In my case, the Jacobi PC could be easily applied as shown in the example
below. However, AMG and multilevel have too many options.

ex) ./app -ksp_type gmres -pc_type jacobi -ksp_view

Best regards

Seung Lee Kwon
-- 
Seung Lee Kwon, Ph.D.Candidate
Aerospace Structures and Materials Laboratory
Department of Mechanical and Aerospace Engineering
Seoul National University
Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826
E-mail : ksl7...@snu.ac.kr
Office : +82-2-880-7389
C. P : +82-10-4695-1062

[petsc-users] GPUs and the float-double dilemma

[Ces VLC]

Hi!

I searched if it’s supported to link an application to two different builds
of PETSc (one configured as float precision, and the other as double). The
last post I found on that topic was from 2016 and it said it’s not
recommended.

The point is that if you wish to prepare builds of your application for
end-users, and if your app offers the option of using GPUs, you have a
critical problem if you cannot link with two different PETSc builds in the
same executable: either you don’t provide support for most GPUs (as they
are float only), or you force float precision even when using the CPU. A
third option (shipping two executables for the app) is not practical, as
the user won’t be able to compare results without quitting the app and
running the other version.

Has the situation changed since 2016, now that GPU support is being added
to PETSc?

An obvious solution would be if PETSc could be built prepending a prefix to
all symbols (functions, structs, types, everything).

Any advances, plans, or thoughts on this?

Thanks!

César