On Mon, 25 Mar 2019, Mills, Richard Tran via petsc-dev wrote:
> Folks,
>
> I've spent a while looking at the BuildSystem code, and I think this is going
> to take me more time than I have available right now to figure it out on my
> own. Someone more familiar with BuildSystem needs to give me some hints --
> soon, if possible, as I really think that building with non-GCC compilers and
> CUDA should be supported in the upcoming release.
>
> What I want to do is to add a test inside cuda.py that checks to see if
> something like
>
> nvcc --compiler-option=<compiler options PETSc has identified> <CUDAFLAGS>
> hello.c
>
> will return successfully.
>
> What I wasn't sure about was how to get at the values for a bunch of the
> above variables within the cuda.py code. After deciding I couldn't really
> follow everything that is happening in the code buy just looking at it, I
> used the 'pdb' python debugger to stick a breakpoint in the
> configureLibrary() method in cuda.py so I could poke around.
>
> **** Aside: Looking at contents of configure objects? ****
> I had hoped I could look at everything that is stashed in the different
> objects by doing things like
>
> (Pdb) p dir(self.compilers)
>
> But this doesn't actually list everything in there. There is no 'CUDAC'
> attribute listed, for instance, but it is there for me to print:
>
> (Pdb) p self.compilers.CUDAC
> 'nvcc'
>
> Is there a good way for me to actually see all the attributes in something
> like the self.compilers object? Sorry, my Python skills are very rusty --
> haven't written much Python in about a decade.
> **** End aside ****
>
> It appears that what I need to construct my command line is then available in
>
> self.compilers.CUDAC -- The invocation for the CUDA compiler
> self.compilers.CXXFLAGS -- The flags passed to the C++ compiler (our "host")
Hm for CXXFLAGS - I think you have to do:
self.pushLanguage('Cxx')
CXXFLAGS = self.getCompilerFlags()
self.popLanguage()
> self.compilers.CUDAFLAGS -- The flags like "-ccbin pgc++" being passed to
> nvcc or whatever CUDAC is
>
> I could use these to construct a command that I then pass to the command
> shell, and maybe I should just do this, but this doesn't seem to follow the
> BuildSystem paradigm. It seems like I should be able to run this test by
> doing something like
>
> self.pushLanguage('CUDA')
> self.checkCompile(cuda_test)
>
> which is, in fact, invoked in checkCUDAVersion(). But the command put
> together by checkCompile() does not include "--compiler-option=<compiler
> options PETSc has identified>". Should I be modifying the code the code
> somewhere so that this argument goes into the compiler invocation constructed
> in self.checkCompile? If so, where should I be doing this?
I think you look for oldFlags usage - something like MPI.py has:
oldFlags = self.compilers.CPPFLAGS
self.compilers.CPPFLAGS = "flags I'd like to test with"
<run test>
self.compilers.CPPFLAGS = oldFlags
Satish
>
> --Richard
>
>
>
> On 3/22/19 10:24 PM, Mills, Richard Tran wrote:
>
>
> On 3/22/19 3:28 PM, Mills, Richard Tran wrote:
> On 3/22/19 12:13 PM, Balay, Satish wrote:
>
> Is there currently an existing check like this somewhere? Or will things just
> fail when running 'make' right now?
>
>
>
> Most likely no. Its probably best to attempt the error case - and
> figure-out how to add a check.
>
> I gave things a try and verified that there is no check for this anywhere in
> configure -- things just fail at 'make' time. I think that all we need is a
> test that will try to compile any simple, valid C program using "nvcc
> --compiler-options=<compiler options PETSc has identified> <CUDAFLAGS>". If
> the test fails, it should report something like "Compiler flags do not work
> with CUDA compiler; perhaps you need to provide to use -ccbin in CUDAFLAGS to
> specify the intended host compiler".
>
> I'm not sure where this test should go. Does it make sense for this to go in
> cuda.py with the other checks like checkNVCCDoubleAlign()? If so, how do I
> get at the values of <compiler options PETSc has identified> and <CUDAFLAGS>?
> I'm not sure what modules I need to import from BuildSystem...
> OK, answering part of my own question here: Re-familiarizing myself with how
> the configure packages work, and then looking through the makefiles, I see
> that the argument to "--compiler-options" is filled in by the makefile
> variables
>
> ${PCC_FLAGS} ${CFLAGS} ${CCPPFLAGS}
>
> and it appears that this partly maps to self.compilers.CFLAGS in BuildSystem.
> But so far I've not managed to employ the right combination of find and grep
> to figure out where PCC_FLAGS and CCPPFLAGS come from.
>
> --Richard
>
> --Richard
>
> Satish
>
> On Fri, 22 Mar 2019, Mills, Richard Tran via petsc-dev wrote:
>
>
>
> On 3/18/19 7:29 PM, Balay, Satish wrote:
>
> On Tue, 19 Mar 2019, Mills, Richard Tran via petsc-dev wrote:
>
>
>
> Colleagues,
>
> It took me a while to get PETSc to build at all with anything on Summit other
> than the GNU compilers, but, once this was accomplished, editing out the
> isGNU() test and then passing something like
>
> '--with-cuda=1',
> '--with-cudac=nvcc -ccbin pgc++',
>
>
>
> Does the following also work?
>
> --with-cuda=1 --with-cudac=nvcc CUDAFLAGS='-ccbin pgc++'
>
> Yes, using CUDAFLAGS as above also works, and that does seem to be a better
> way to do things.
>
> After experimenting with a lot of different builds on Summit, and doing more
> reading about how CUDA compilation works on different platforms, I'm now
> thinking that perhaps configure.py should *avoid* doing anything clever to
> try figure out what the value of "-ccbin" should be. For one, this is not
> anything that NVIDIA's toolchain does for the user in the first place: If you
> want to use nvcc with a host compiler that isn't whatever NVIDIA considers
> the default (g++ on Linux, clang on Mac OS, MSVC on Windows), NVIDIA expects
> you to provide the appropriate '-ccbin' argument. Second, nvcc isn't the only
> CUDA compiler that a user might want to use: some people use Clang directly
> to compile CUDA code. Third, which host compilers are supported appears to be
> platform independent; for example, GCC is the default/preferred host compiler
> on Linux, but isn't even supported on Mac OS! Figuring out what is supported
> is very convoluted, and I think that trying to get configure to determine
> this may be more trouble than it is worth. I think we should instead let the
> user try whatever, and print out a helpful message how they "may need to
> specify host compiler to nvcc with -ccbin" if the CUDA compiler doesn't seem
> to work. Also, I'll put something about this in the CUDA configure examples.
> Any objections?
>
>
>
>
> Sometimes we have extra options in configure for specific features for
> ex: --with-pic --with-visibility etc.
>
> But that gets messy. On cuda side - we've have --with-cuda-arch and at
> some point elimiated it [so CUDAFLAGS is now the interface for this
> flag]. We could add --with-cuda-internal-compiler option to petsc
> configure - but it will again have similar drawbacks. I personally
> think most users will gravitate towards specifying such option via
> CUDAFLAGS
>
>
>
>
> to configure works fine. So, I should make a change to the BuildSystem
> cuda.py along these lines. I'm wondering exactly how I should make this work.
> I could just remove the check,
>
>
>
> sure
>
>
>
> but I think that maybe the better thing to do is to check isGNU(), then if
> the compiler is *not* GNU, configure should add the appropriate '-ccbin'
> argument to "--with-cudac", unless the user has specified '-ccbin' in their
> '--with-cudac' already. Or do we need to get this fancy?
>
>
>
> The check should be: do --compiler-options= constructed by PETSc configure
> work with CUDAC
>
> Is there currently an existing check like this somewhere? Or will things just
> fail when running 'make' right now?
>
>
>
> [or perhaps we should - just trim the --compiler-options to only -I flags?]
>
> I think we should avoid explict check for a compiler type [i.e isGNU() check]
> as much as possible.
>
>
>
>
> CUDA is only supposed to work with certain compilers, but there doesn't seem
> to be a correct official list (for instance, it supposedly won't work with
> the IBM XL compilers, but they certainly *are* actually supported on Summit).
> Heck, the latest GCC suite won't even work right now. Since what compilers
> are supported seems to be in flux, I suggest we just let the user try
> anything and then let things fail if it doesn't work.
>
>
>
> I suspec the list is dependent on the install [for ex: linux vs Windows vs
> somthing else?] and version of cuda [for ex: each version of cuda supports
> only specific versions of gcc]
>
> Yes, you are correct about this, as I detailed above.
>
>
>
> Satish
>
>
>
>
> --Richard
>
> On 3/12/19 8:45 PM, Smith, Barry F. wrote:
>
>
> Richard,
>
> You need to remove the isGNU() test and then experiment with getting the
> Nvidia tools to use the compiler you want it to use.
>
> No one has made a serious effort to use any other compilers but Gnu (at
> least not publicly).
>
> Barry
>
>
>
>
>
> On Mar 12, 2019, at 10:40 PM, Mills, Richard Tran via petsc-dev
> <[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]><mailto:[email protected]>
> wrote:
>
> Fellow PETSc developers,
>
> If I try to configure PETSc with CUDA support on the ORNL Summit system using
> non-GNU compilers, I run into an error due to the following code in
> packages/cuda.py:
>
> def configureTypes(self):
> import config.setCompilers
> if not config.setCompilers.Configure.isGNU(self.setCompilers.CC,
> self.log):
> raise RuntimeError('Must use GNU compilers with CUDA')
> ...
>
> Is this just because this code predates support for other host compilers with
> nvcc, or is there perhaps some more subtle reason that I, with my
> inexperience using CUDA, don't know about? I'm guessing that I just need to
> add support for using '-ccbin' appropriately to set the location of the
> non-GNU host compiler, but maybe there is something that I'm missing. I poked
> around in the petsc-dev mailing list archives and can find a few old threads
> on using non-GNU compilers, but I'm not sure what conclusions were reached.
>
> Best regards,
> Richard
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
