Alright, this is a lot of progress!
1) We are using Makefiles. I agree with you about the dependency graph,
so I'll try and sort that out. I'll let you know what the result is.
2) I just checked and, indeed, the *_intermediate_link.o file is not
being passed -fPIC. Is this our problem? What is the correct fix?
Irwin
James Bigler wrote:
On Mon, Jan 5, 2015 at 1:57 PM, Irwin Zaid <irwin.z...@physics.ox.ac.uk
<mailto:irwin.z...@physics.ox.ac.uk>> wrote:
Hi James,
Thanks for the quick reply! As I mentioned, we've hit two issues.
The first is the project dependencies one, which I'll try and
describe more a bit below. I'm not a CMake expert, so please bear
with me.
The second is what I've put under 2).
The only CMake build dependency changes when doing separable
compilation
are found in CUDA_LINK_SEPARABLE___COMPILATION_OBJECTS.
Basically what
this does is create a new rule to build an intermediate link
file. For
everything but some versions of MSVC generators it adds a custom
command
to generate this intermediate link file. The other case adds a
custom
command that runs as a PRE_LINK command to generate the object
file (the
reasons for this is a bug in VS custom command dependency
scanning), but
this should happen during link phase and not compile phase.
Nothing in here should change what happens before the target is
built,
though.
1) Okay, I understand that, but I do think we saw a different
behaviour when we switched to separable compilation. Let me describe
what we are doing.
We generate part of our library from a simple program (call the
simple program 'gen', which generates a source file 'gen.hpp') that
we want to execute before compiling our library (call our library
'main'). We set this up with the following:
- add_executable(...) is called for 'gen'
- add_custom_command(...) sets up a command that executes 'gen'
- set_source_files_properties(..__.) is called to set 'gen.hpp' as
having the PROPERTY of GENERATED
- add_dependencies(main gen) is called to establish 'main' depends
on 'gen'
So far, this has only failed for CUDA with separable compilation.
(It has worked for all of our other configurations. including CUDA
without separable compilation.)
Have we done something wrong? Is there some additional information
we can look at to figure out what's going on?
What kind of generator are you using (e.g. makefile)?
Here's what I'm thinking might be the problem, though I'm not sure why
it would have worked without CUDA_SEPARABLE_COMPILATION.
There's a dependency between gen and gen.hpp (encoded in the call to
add_custom_command(OUTPUTS gen.hpp COMMAND gen)).
There's a dependency between main and gen (can't start building main
until gen is built).
There's a dependency between gen.hpp and main (gen.hpp is an input
source to main, so it needs to be built as part of main).
What I don't see is a dependency between gen.hpp and all the cuda
sources that might use it as input. So from a dependency graph
standpoint a makefile (if one is being used in this case) is entirely
free to start compiling the CUDA code once the gen target has been
satisfied.
What you need is another target that builds the gen.hpp file which can
be forced to run before main starts to build. There are more than one
way to do this, and I'm not sure what the best option is, but you might
try this:
add_custom_command(OUTPUTS gen.hpp ....)
add_custom_target(make_gen_hpp DEPENDS gen.hpp)
add_dependency(main make_gen_hpp)
2) A second problem we've run into is an error when trying to link a
CUDA shared library with separable compilation. This is specifically
a Linux problem, on Mac it is fine. A static library is also fine,
working for both Linux and Mac.
The particular error is "relocation R_X86_64_32S against
`__nv_module_id' can not be used when making a shared object;
recompile with -fPIC". However, we are already compiling with -fPIC.
I can confirm that -fPIC appears to be passed to both the host
compiler for non-CUDA source and via -Xcompiler -fPIC for CUDA source.
This error occurs when trying to link all the different object files
together of our library.
Do you have any idea of what this could be?
I'm not sure which object file wasn't compiled with -fPIC, but I would
suspect it might be the intermediate link object file. FindCUDA is
supposed to pass this flag along (see
function(_cuda_get_important_host_flags)), but you might want to verify
this with a 'make VERBOSE=1' and look for
<target_name>_intermediate_link.o (substitute your target name in or
leave it out of the search string).
James
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
