Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

On Wed, Apr 7, 2021 at 5:47 PM Scott Kruger wrote: > On 2021-04-06 14:44, Matthew Knepley did write: > > > > Does spack have some magic for this we could use? > > > > > > > > > > spack developed the archspec repo to abstract all of these issues: > > > https://github.com/archspec/archspec > > > >

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

On 2021-04-06 14:44, Matthew Knepley did write: > > > Does spack have some magic for this we could use? > > > > > > > spack developed the archspec repo to abstract all of these issues: > > https://github.com/archspec/archspec > > > I do not love it. Besides the actual code (you can always

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

On Tue, Apr 6, 2021 at 2:08 PM Scott Kruger wrote: > > I wrote sent this yesterday but am having some strange mailing issues. > > On 2021-04-03 22:42, Barry Smith did write: > > > > It would be very nice to NOT require PETSc users to provide this flag, > how the heck will they know what it

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

I wrote sent this yesterday but am having some strange mailing issues. On 2021-04-03 22:42, Barry Smith did write: > > It would be very nice to NOT require PETSc users to provide this flag, how > the heck will they know what it should be when we cannot automate it > ourselves? > > Any

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

Jeff, Likely deviceQuery provides more than enough information; sometimes it is prebuilt but it seems now it is only provided as source code so the user needs to build it (and the Makefile is huge :-)). I think it would be enough if NVIDIA just always provided prebuilt deviceQuery in

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

> > > Generically, independent of Kokkos, ideally I would run a single > precompiled NVIDIA program that gave me all the information about the > current hardware I was running and that would provide in simple format > exactly the information I needed to configure PETSc, Kokkos etc for THAT >

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

Junchao, I hope my latest MRs manages that for the current generation of those values. If not, we need refinement. Barry > On Apr 5, 2021, at 9:30 PM, Junchao Zhang wrote: > > > > > On Mon, Apr 5, 2021 at 7:33 PM Jeff Hammond > wrote: > NVCC has

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

Thanks Jeff, The information is eventually there somewhere, the issue is more getting the information in a simple way, automatically, at PETSc configure time that is portable and will never crash. https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__DEVICE.html

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

On Mon, Apr 5, 2021 at 7:33 PM Jeff Hammond wrote: > NVCC has supported multi-versioned "fat" binaries since I worked for > Argonne. Libraries should figure out what the oldest hardware they are > about is and then compile for everything from that point forward. Kepler > (3.5) is oldest

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

NVCC has supported multi-versioned "fat" binaries since I worked for Argonne. Libraries should figure out what the oldest hardware they are about is and then compile for everything from that point forward. Kepler (3.5) is oldest version any reasonable person should be thinking about at this

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

This is nvidia mess-up. Why isn't there a command that give me these values [if they insist on this interface for nvcc] I see Barry want configure to do something here - but whatever we do - we would be shifting the problem around. [even if we detect stuff - build box might not have the GPU

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

Hmm, OK, I found a table at https://sparta.sandia.gov/doc/accelerate_kokkos.html and it tells me that "PASCAL60" refers to "NVIDIA Pascal generation CC 6.0 GPU" and "PASCAL61" refers to "NVIDIA Pascal generation CC 6.1 GPU". But I have no idea what those 6.0 vs 6.1 version numbers mean, and

Re: [petsc-dev] -with-kokkos-cuda-arch=AMPERE80 nonsense

You raise a good point, Barry. I've been completely mystified by what some of these names even mean. What does "PASCAL60" vs. "PASCAL61" even mean? Do you know of where this is even documented? I can't really find anything about it in the Kokkos documentation. The only thing I can really find