Re: [petsc-users] ex19: Segmentation Violation when run with MUMPS on MacOS (arm64)

[Zongze Yang]

 The branch with the patched openmpi works fine for me. Thanks for sharing! Best wishes, Zongze > On 4 Apr 2024, at 12: 44, Satish Balay  wrote: > > With xcode-15. 3 and branch "barry/2024-04-03/fix-chaco-modern-c/release"




ZjQcmQRYFpfptBannerStart




  

  
	This Message Is From an External Sender
  
  
This message came from outside your organization.
  



 
  


ZjQcmQRYFpfptBannerEnd




The branch with the patched openmpi works fine for me.

Thanks for sharing!

Best wishes,
Zongze

> On 4 Apr 2024, at 12:44, Satish Balay  wrote:
> 
> With xcode-15.3 and branch "barry/2024-04-03/fix-chaco-modern-c/release" from https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/merge_requests/7433__;!!G_uCfscf7eWS!b2jrTotLzInMflHGijS61VKFqgwsNfFEui4cpKyEgYLspq5b8HyH38UJWxEuke5cGC4RnZf7jdYuh8CtFfZlodjP$ [and a patched openmpi tarball to remove -Wl,-commons,use_dylibs] the following works for me.
> 
> Satish
> 
> 
> 
> petsc@mpro petsc.x % ./configure --download-bison --download-chaco --download-ctetgen --download-eigen --download-fftw --download-hdf5 --download-hpddm --download-hwloc --download-hypre --download-libpng --download-metis --download-mmg --download-mumps --download-netcdf --download-openblas --download-openblas-make-options="'USE_THREAD=0 USE_LOCKING=1 USE_OPENMP=0'" --download-p4est --download-parmmg --download-pnetcdf --download-pragmatic --download-ptscotch --download-scalapack --download-slepc --download-suitesparse --download-superlu_dist --download-tetgen --download-triangle --with-c2html=0 --with-debugging=1 --with-fortran-bindings=0 --with-shared-libraries=1 --with-x=0 --with-zlib --download-openmpi=https://urldefense.us/v3/__https://web.cels.anl.gov/projects/petsc/download/externalpackages/openmpi-5.0.2-xcode15.tar.gz__;!!G_uCfscf7eWS!b2jrTotLzInMflHGijS61VKFqgwsNfFEui4cpKyEgYLspq5b8HyH38UJWxEuke5cGC4RnZf7jdYuh8CtFUMGGKvW$ --download-pastix && make && make check
> 
>  CC arch-darwin-c-debug/obj/src/lme/interface/lmesolve.o
> CLINKER arch-darwin-c-debug/lib/libslepc.3.21.0.dylib
>DSYMUTIL arch-darwin-c-debug/lib/libslepc.3.21.0.dylib
> Now to install the library do:
> make SLEPC_DIR=/Users/petsc/petsc.x/arch-darwin-c-debug/externalpackages/git.slepc PETSC_DIR=/Users/petsc/petsc.x install
> =
> *** Installing SLEPc ***
> *** Installing SLEPc at prefix location: /Users/petsc/petsc.x/arch-darwin-c-debug  ***
> 
> Install complete.
> Now to check if the libraries are working do (in current directory):
> make SLEPC_DIR=/Users/petsc/petsc.x/arch-darwin-c-debug PETSC_DIR=/Users/petsc/petsc.x PETSC_ARCH=arch-darwin-c-debug check
> 
> /usr/bin/make --no-print-directory -f makefile PETSC_ARCH=arch-darwin-c-debug PETSC_DIR=/Users/petsc/petsc.x SLEPC_DIR=/Users/petsc/petsc.x/arch-darwin-c-debug/externalpackages/git.slepc install-builtafterslepc
> /usr/bin/make --no-print-directory -f makefile PETSC_ARCH=arch-darwin-c-debug PETSC_DIR=/Users/petsc/petsc.x SLEPC_DIR=/Users/petsc/petsc.x/arch-darwin-c-debug/externalpackages/git.slepc slepc4py-install
> make[6]: Nothing to be done for `slepc4py-install'.
> *** Building and installing HPDDM ***
> =
> Now to check if the libraries are working do:
> make PETSC_DIR=/Users/petsc/petsc.x PETSC_ARCH=arch-darwin-c-debug check
> =
> Running PETSc check examples to verify correct installation
> Using PETSC_DIR=/Users/petsc/petsc.x and PETSC_ARCH=arch-darwin-c-debug
> C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process
> C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes
> C/C++ example src/snes/tutorials/ex19 run successfully with HYPRE
> C/C++ example src/snes/tutorials/ex19 run successfully with MUMPS
> C/C++ example src/snes/tutorials/ex19 run successfully with SuiteSparse
> C/C++ example src/snes/tutorials/ex19 run successfully with SuperLU_DIST
> C/C++ example src/vec/vec/tests/ex47 run successfully with HDF5
> Running SLEPc check examples to verify correct installation
> Using SLEPC_DIR=/Users/petsc/petsc.x/arch-darwin-c-debug/externalpackages/git.slepc, PETSC_DIR=/Users/petsc/petsc.x, and PETSC_ARCH=arch-darwin-c-debug
> C/C++ example src/eps/tests/test10 run successfully with 1 MPI process
> C/C++ example src/eps/tests/test10 run successfully with 2 MPI processes
> Completed SLEPc check examples
> Completed PETSc check examples
> petsc@mpro petsc.x % clang --version
> Apple clang version 15.0.0 (clang-1500.3.9.4)
> Target: arm64-apple-darwin23.4.0
> Thread model: posix
> InstalledDir: /Library/Developer/CommandLineTools/usr/bin
> petsc@mpro petsc.x % 
> 
> 
> On Tue, 2 Apr 2024, Zongze Yang wrote:
> 
>> Thank you for the suggestion.
>> 
>> I'd like to share some test results using the current Xcode. When I added the flag `LDFLAGS=-Wl,-ld_classic` and configured PETSc 

Re: [petsc-users] Problem with NVIDIA compiler and OpenACC

[Frank Bramkamp]

 Ok, I will have a look. It is already evening here in Sweden, so it might take until tomorrow. Thanks Frank  ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍




ZjQcmQRYFpfptBannerStart




  

  
	This Message Is From an External Sender
  
  
This message came from outside your organization.
  



 
  


ZjQcmQRYFpfptBannerEnd




Ok, I will have a look.

It is already evening here in Sweden, so it might take until tomorrow.

Thanks Frank

Re: [petsc-users] Problem with NVIDIA compiler and OpenACC

[Barry Smith]

   Frank,

Please try the PETSc git branch barry/2024-04-04/rm-lnvc-link-line/release 

  This will hopefully resolve the -lnvc issue. Please let us know and we can 
add the fix to our current release.

 Barry

> On Apr 4, 2024, at 9:37 AM, Frank Bramkamp  wrote:
> 
> This Message Is From an External Sender 
> This message came from outside your organization.
> Dear PETSC Team,
> 
> I found the following problem:
> I compile petsc 3.20.5 with Nvidia compiler 23.7.
> 
> 
> I use a pretty standard configuration, including
> 
> --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpifort COPTFLAGS="-O2 -g" 
> CXXOPTFLAGS="-O2 -g" FOPTFLAGS="-O2 -g"  --with-debugging=0 --with-log=1 
> --download-fblaslapack --with-cuda=0
> 
> I exclude cuda, since I was not sure if the problem was cuda related. 
> 
> 
> The problem is now, if I have s simple fortran program where I link the petsc 
> library, but I actually do not use petsc in that program
> (Just for testing). I want to use OpenACC directives in my program, e.g. 
> !$acc parallel loop .
> The problem is now, as soon I link with the petsc library, the openacc 
> commands do not work anymore.
> It seems that openacc is not initialised and hence it cannot find a GPU.
> 
> The problem seems that you link with -lnvc.
> In “petscvariables” => PETSC_WITH_EXTERNAL_LIB you include “-lnvc”.
> If I take this out, then openacc works. With “-lnvc” something gets messed up.
> 
> The problem is also discussed here:
> https://urldefense.us/v3/__https://forums.developer.nvidia.com/t/failed-cuda-device-detection-when-explicitly-linking-libnvc/203225/1__;!!G_uCfscf7eWS!Z2uhPVP0GUrttP3rh6nLk6BQsoI2EIfKfoLVXcwQFksSvtvvRILt4Yq0y-FFYmi3ugybPdn-te0Pw5mfExHSw7Y$
>   
> 
> 
> My understanding is that libnvc is more a runtime library that does not need 
> to be included by the linker.
> Not sure if there is a specific reason to include libnvc (I am not so 
> familiar what this library does).
> 
> If I take out -lnvc from “petscvariables”, then my program with openacc works 
> as expected. I did not try any more realistic program that includes petsc.
> 
> 
> 
> 2)
> When compiling petsc with cuda support, I also found that in the petsc 
> library the library libnvJitLink.so 
>   >.12
> Is not found. On my system this library is in $CUDA_ROOT/lib64
> I am not sure where this library is on your system ?! 
> 
> 
> Thanks a lot, Frank Bramkamp

Re: [petsc-users] Problem with NVIDIA compiler and OpenACC

[Frank Bramkamp]

 Ok, I will look for the config. log file. Frank  ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍




ZjQcmQRYFpfptBannerStart




  

  
	This Message Is From an External Sender
  
  
This message came from outside your organization.
  



 
  


ZjQcmQRYFpfptBannerEnd





Ok, I will look for the config.log file.

Frank

Re: [petsc-users] Problem with NVIDIA compiler and OpenACC

[Barry Smith]

   Please send configure.log 

   We do not explicitly include libnvc but as Satish noted it may get listed 
when configure is generating link lines.

   With configure.log we'll know where it is being included (and we may be able 
to provide a fix that removes it  explicitly since it is apparently not needed 
according to the NVIDIA folks).

   Barry


> On Apr 4, 2024, at 10:33 AM, Frank Bramkamp  wrote:
> 
> This Message Is From an External Sender 
> This message came from outside your organization.
> Thanks for the reply,
> 
> Do you know if you actively include the libnvc library ?!
> Or is this somehow automatically included ?! 
> 
> Greetings, Frank
> 
> 
> 
> 
>> On 4 Apr 2024, at 15:56, Satish Balay > > wrote:
>> 
>> 
>> On Thu, 4 Apr 2024, Frank Bramkamp wrote:
>> 
>>> Dear PETSC Team,
>>> 
>>> I found the following problem:
>>> I compile petsc 3.20.5 with Nvidia compiler 23.7.
>>> 
>>> 
>>> I use a pretty standard configuration, including
>>> 
>>> --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpifort COPTFLAGS="-O2 -g" 
>>> CXXOPTFLAGS="-O2 -g" FOPTFLAGS="-O2 -g" --with-debugging=0 --with-log=1 
>>> --download-fblaslapack --with-cuda=0
>>> 
>>> I exclude cuda, since I was not sure if the problem was cuda related. 
>> 
>> Can you try using (to exclude cuda): --with-cudac=0
>> 
>>> 
>>> 
>>> The problem is now, if I have s simple fortran program where I link the 
>>> petsc library, but I actually do not use petsc in that program
>>> (Just for testing). I want to use OpenACC directives in my program, e.g. 
>>> !$acc parallel loop .
>>> The problem is now, as soon I link with the petsc library, the openacc 
>>> commands do not work anymore.
>>> It seems that openacc is not initialised and hence it cannot find a GPU.
>>> 
>>> The problem seems that you link with -lnvc.
>>> In “petscvariables” => PETSC_WITH_EXTERNAL_LIB you include “-lnvc”.
>>> If I take this out, then openacc works. With “-lnvc” something gets messed 
>>> up.
>>> 
>>> The problem is also discussed here:
>>> https://urldefense.us/v3/__https://forums.developer.nvidia.com/t/failed-cuda-device-detection-when-explicitly-linking-libnvc/203225/1__;!!G_uCfscf7eWS!dlXNyKBzSbximQ13OXxwO506OF71yRM_H5KEnarqXE75D6Vg-ePZr2u6SJ5V3YpRETatvb9pMOUVmpyN0-19SFlbug$>>  >
>>> 
>>> My understanding is that libnvc is more a runtime library that does not 
>>> need to be included by the linker.
>>> Not sure if there is a specific reason to include libnvc (I am not so 
>>> familiar what this library does).
>>> 
>>> If I take out -lnvc from “petscvariables”, then my program with openacc 
>>> works as expected. I did not try any more realistic program that includes 
>>> petsc.
>>> 
>>> 
>>> 
>>> 2)
>>> When compiling petsc with cuda support, I also found that in the petsc 
>>> library the library libnvJitLink.so.12
>>> Is not found. On my system this library is in $CUDA_ROOT/lib64
>>> I am not sure where this library is on your system ?! 
>> 
>> Hm - good if you can send configure.log for this. configure attempts '$CC 
>> -v' to determine the link libraries to get c/c++/fortran compatibility 
>> libraries. But it can grab other libraries that the compilers are using 
>> internally here.
>> 
>> To avoid this - you can explicitly list these libraries to configure. For 
>> ex: for gcc/g++/gfortran
>> 
>> ./configure CC=gcc CXX=g++ FC=gfortran LIBS="-lgfortran -lstdc++"
>> 
>> Satish
>> 
>>> 
>>> 
>>> Thanks a lot, Frank Bramkamp

Re: [petsc-users] Problem with NVIDIA compiler and OpenACC

[Frank Bramkamp]

Thanks for the reply,

Do you know if you actively include the libnvc library ?!
Or is this somehow automatically included ?! 

Greetings, Frank




> On 4 Apr 2024, at 15:56, Satish Balay  wrote:
> 
> 
> On Thu, 4 Apr 2024, Frank Bramkamp wrote:
> 
>> Dear PETSC Team,
>> 
>> I found the following problem:
>> I compile petsc 3.20.5 with Nvidia compiler 23.7.
>> 
>> 
>> I use a pretty standard configuration, including
>> 
>> --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpifort COPTFLAGS="-O2 -g" 
>> CXXOPTFLAGS="-O2 -g" FOPTFLAGS="-O2 -g" --with-debugging=0 --with-log=1 
>> --download-fblaslapack --with-cuda=0
>> 
>> I exclude cuda, since I was not sure if the problem was cuda related. 
> 
> Can you try using (to exclude cuda): --with-cudac=0
> 
>> 
>> 
>> The problem is now, if I have s simple fortran program where I link the 
>> petsc library, but I actually do not use petsc in that program
>> (Just for testing). I want to use OpenACC directives in my program, e.g. 
>> !$acc parallel loop .
>> The problem is now, as soon I link with the petsc library, the openacc 
>> commands do not work anymore.
>> It seems that openacc is not initialised and hence it cannot find a GPU.
>> 
>> The problem seems that you link with -lnvc.
>> In “petscvariables” => PETSC_WITH_EXTERNAL_LIB you include “-lnvc”.
>> If I take this out, then openacc works. With “-lnvc” something gets messed 
>> up.
>> 
>> The problem is also discussed here:
>> https://urldefense.us/v3/__https://forums.developer.nvidia.com/t/failed-cuda-device-detection-when-explicitly-linking-libnvc/203225/1__;!!G_uCfscf7eWS!dlXNyKBzSbximQ13OXxwO506OF71yRM_H5KEnarqXE75D6Vg-ePZr2u6SJ5V3YpRETatvb9pMOUVmpyN0-19SFlbug$
>>  
>> >  
>> 
>>  >
>> 
>> My understanding is that libnvc is more a runtime library that does not need 
>> to be included by the linker.
>> Not sure if there is a specific reason to include libnvc (I am not so 
>> familiar what this library does).
>> 
>> If I take out -lnvc from “petscvariables”, then my program with openacc 
>> works as expected. I did not try any more realistic program that includes 
>> petsc.
>> 
>> 
>> 
>> 2)
>> When compiling petsc with cuda support, I also found that in the petsc 
>> library the library libnvJitLink.so.12
>> Is not found. On my system this library is in $CUDA_ROOT/lib64
>> I am not sure where this library is on your system ?! 
> 
> Hm - good if you can send configure.log for this. configure attempts '$CC -v' 
> to determine the link libraries to get c/c++/fortran compatibility libraries. 
> But it can grab other libraries that the compilers are using internally here.
> 
> To avoid this - you can explicitly list these libraries to configure. For ex: 
> for gcc/g++/gfortran
> 
> ./configure CC=gcc CXX=g++ FC=gfortran LIBS="-lgfortran -lstdc++"
> 
> Satish
> 
>> 
>> 
>> Thanks a lot, Frank Bramkamp

Re: [petsc-users] Problem with NVIDIA compiler and OpenACC

[Satish Balay via petsc-users]

On Thu, 4 Apr 2024, Frank Bramkamp wrote:

> Dear PETSC Team,
> 
> I found the following problem:
> I compile petsc 3.20.5 with Nvidia compiler 23.7.
> 
> 
> I use a pretty standard configuration, including
> 
> --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpifort COPTFLAGS="-O2 -g" 
> CXXOPTFLAGS="-O2 -g" FOPTFLAGS="-O2 -g"  --with-debugging=0 --with-log=1 
> --download-fblaslapack --with-cuda=0
> 
> I exclude cuda, since I was not sure if the problem was cuda related. 

Can you try using (to exclude cuda): --with-cudac=0

> 
> 
> The problem is now, if I have s simple fortran program where I link the petsc 
> library, but I actually do not use petsc in that program
> (Just for testing). I want to use OpenACC directives in my program, e.g. 
> !$acc parallel loop .
> The problem is now, as soon I link with the petsc library, the openacc 
> commands do not work anymore.
> It seems that openacc is not initialised and hence it cannot find a GPU.
> 
> The problem seems that you link with -lnvc.
> In “petscvariables” => PETSC_WITH_EXTERNAL_LIB you include “-lnvc”.
> If I take this out, then openacc works. With “-lnvc” something gets messed up.
> 
> The problem is also discussed here:
> https://urldefense.us/v3/__https://forums.developer.nvidia.com/t/failed-cuda-device-detection-when-explicitly-linking-libnvc/203225/1__;!!G_uCfscf7eWS!dlXNyKBzSbximQ13OXxwO506OF71yRM_H5KEnarqXE75D6Vg-ePZr2u6SJ5V3YpRETatvb9pMOUVmpyN0-19SFlbug$
>   
>   >
> 
> My understanding is that libnvc is more a runtime library that does not need 
> to be included by the linker.
> Not sure if there is a specific reason to include libnvc (I am not so 
> familiar what this library does).
> 
> If I take out -lnvc from “petscvariables”, then my program with openacc works 
> as expected. I did not try any more realistic program that includes petsc.
> 
> 
> 
> 2)
> When compiling petsc with cuda support, I also found that in the petsc 
> library the library libnvJitLink.so.12
> Is not found. On my system this library is in $CUDA_ROOT/lib64
> I am not sure where this library is on your system ?! 

Hm - good if you can send configure.log for this. configure attempts '$CC -v' 
to determine the link libraries to get c/c++/fortran compatibility libraries. 
But it can grab other libraries that the compilers are using internally here.

To avoid this - you can explicitly list these libraries to configure. For ex: 
for gcc/g++/gfortran

./configure CC=gcc CXX=g++ FC=gfortran LIBS="-lgfortran -lstdc++"

Satish

> 
> 
> Thanks a lot, Frank Bramkamp
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

[petsc-users] Problem with NVIDIA compiler and OpenACC

[Frank Bramkamp]

Dear PETSC Team,

I found the following problem:
I compile petsc 3.20.5 with Nvidia compiler 23.7.


I use a pretty standard configuration, including

--with-cc=mpicc --with-cxx=mpicxx --with-fc=mpifort COPTFLAGS="-O2 -g" 
CXXOPTFLAGS="-O2 -g" FOPTFLAGS="-O2 -g"  --with-debugging=0 --with-log=1 
--download-fblaslapack --with-cuda=0

I exclude cuda, since I was not sure if the problem was cuda related. 


The problem is now, if I have s simple fortran program where I link the petsc 
library, but I actually do not use petsc in that program
(Just for testing). I want to use OpenACC directives in my program, e.g. !$acc 
parallel loop .
The problem is now, as soon I link with the petsc library, the openacc commands 
do not work anymore.
It seems that openacc is not initialised and hence it cannot find a GPU.

The problem seems that you link with -lnvc.
In “petscvariables” => PETSC_WITH_EXTERNAL_LIB you include “-lnvc”.
If I take this out, then openacc works. With “-lnvc” something gets messed up.

The problem is also discussed here:
https://urldefense.us/v3/__https://forums.developer.nvidia.com/t/failed-cuda-device-detection-when-explicitly-linking-libnvc/203225/1__;!!G_uCfscf7eWS!dlXNyKBzSbximQ13OXxwO506OF71yRM_H5KEnarqXE75D6Vg-ePZr2u6SJ5V3YpRETatvb9pMOUVmpyN0-19SFlbug$
  


My understanding is that libnvc is more a runtime library that does not need to 
be included by the linker.
Not sure if there is a specific reason to include libnvc (I am not so familiar 
what this library does).

If I take out -lnvc from “petscvariables”, then my program with openacc works 
as expected. I did not try any more realistic program that includes petsc.



2)
When compiling petsc with cuda support, I also found that in the petsc library 
the library libnvJitLink.so.12
Is not found. On my system this library is in $CUDA_ROOT/lib64
I am not sure where this library is on your system ?! 


Thanks a lot, Frank Bramkamp