Re: [CMake] Fwd: module and cmake
On Saturday 19 May 2012, Johannes Zarl wrote: Hello Paul, I don't know about your specific find_package file for FFTW, but we do use modules together with CMake, so I'll add my thoughts: As Eric already said, the modules command alters your environment. CMake doesn't know about shell modules, but most find_package commands provide some way to tell them about the location of a package by specifying an environment variable (normally something like PACKAGE_ROOT or PACKAGE_DIR). You'll have to look in the find-package script if this is the case with your FindFFTW.cmake script. If it's not written in the documentation on top of that file, search for $ENV. If your FindFFTW3.cmake does not examine any environment variable, you should fix it in your project (and file a bug with the original project, if it's not your project). If that's not an option, it's probably best if you just set whatever variable your findFFTW3.cmake script expects as a clue as parameter to cmake (e.g. cmake .. -DFFTW3_INCLUDE_DIR=$FFTW3_ROOT/include). So in summary, you should fix your module file to provide a suitable variable pointing to your installation root, and you should also fix poorly written FindXXX.cmake scripts circulating in the wild. Yes, either such a env. variable as pointed out here, or a modified CMAKE_PREFIX_PATH as mentioned in the other email. Alex -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
[CMake] Fwd: module and cmake
Hi all. I am currently trying to create a more tidy CMakeLists.txt script for a simulation code I'm working on. The target platforms are mac and linux desktops (for development) and HPC [0] servers (all linux or, in some hypothetical future, unix systems). For the desktop/laptop case, it's mostly easy enough, as one installs libraries in a standard location - /opt, /usr/local, or something of that sort. On HPC servers, it is very common to install software in modules. I am not sure how well aquainted the CMake crowd is with the module system [1] so I'll post a brief explanation. HPC servers often have a large amount of users with varying requirements. Therefore, it is common to require, say, two different versions of a library, or the same library for two different compilers. As an example, a machine I'm currently using has several modules for the FFTW library. Example: paulanto@rocks:~/Rayleigh2D $ module avail fftw fftw/2.1.5 fftw/3.2.2(default) So, there's two different fftw versions available, and I'm currently using the default one as such: paulanto@rocks:~/Rayleigh2D $ module load fftw paulanto@rocks:~/Rayleigh2D $ module list Currently Loaded Modulefiles: 1) fftw/3.2.2 There we go. Now what does it mean to load a module? Well, basically, the module command works by adjusting environment variables: paulanto@rocks:~/Rayleigh2D $ module show fftw /share/apps/modulefiles/fftw/3.2.2: prepend-path CPATH /share/apps/modulessoftware/fftw/fftw-3.2.2/include prepend-path FPATH /share/apps/modulessoftware/fftw/fftw-3.2.2/include prepend-path LD_LIBRARY_PATH /share/apps/modulessoftware/fftw/fftw-3.2.2/lib prepend-path LIBRARY_PATH /share/apps/modulessoftware/fftw/fftw-3.2.2/lib We see that the fftw module sets various paths, and that these will all be different based on the exact version I'm using on the current machine. Hence, I can't tell beforehand, in my CMakeLists.txt, where to search for, say, libfftw3.a. So why am i ranting on about this? Well, basically find_package(FFTW3) does not find the fftw library, and other libraries I'm using (hdf5, mkl, ...) share the same fate. Previously I've just hand-added all sorts of include_directories and link_directories but I'm getting fed up and want to use the more elegant find_package approach, at least where possible. So, what am I asking? 1) Is there a standard way for CMake to interact with modules? 2) Is there a standard way to tell FindXXX.cmake (where XXX=fftw, hdf5, mkl, etc) where to search for libraries? 3) Can I convince CMake to include link and include directores from the environment variables, like LIBRARY_PATH, FPATH etc? 4) How do I handle LIBRARY_PATHs with multiple entries, such as this one? paulanto@rocks:~/Rayleigh2D $ echo $LIBRARY_PATH /share/apps/modulessoftware/fftw/fftw-3.2.2/lib:/share/apps/modulessoftware/hdf5/hdf-5.1.8.5-intel/lib:/share/apps/modulessoftware/openmpi/openmpi-1.4.3-intel/lib:/share/apps/modulessoftware/intel/compilers/11.1.059/lib/intel64:/share/apps/modulessoftware/intel/compilers/11.1.059/ipp/em64t/lib:/share/apps/modulessoftware/intel/compilers/11.1.059//mkl/lib/em64t:/share/apps/modulessoftware/intel/compilers/11.1.059/tbb/intel64/cc4.1.0_libc2.4_kernel2.6.16.21/lib Perhaps I'm not asking the right questions, but hopefully, some can tell me what I _should_ be asking. Or even better, the answer. Cheers Paul [0] High Performance Computing [1] http://modules.sourceforge.net/ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Fwd: module and cmake
Am Samstag, 19. Mai 2012, 14:42:49 schrieb Paul Anton Letnes: Hi all. I am currently trying to create a more tidy CMakeLists.txt script for a simulation code I'm working on. The target platforms are mac and linux desktops (for development) and HPC [0] servers (all linux or, in some hypothetical future, unix systems). For the desktop/laptop case, it's mostly easy enough, as one installs libraries in a standard location - /opt, /usr/local, or something of that sort. On HPC servers, it is very common to install software in modules. I am not sure how well aquainted the CMake crowd is with the module system [1] so I'll post a brief explanation. HPC servers often have a large amount of users with varying requirements. Therefore, it is common to require, say, two different versions of a library, or the same library for two different compilers. As an example, a machine I'm currently using has several modules for the FFTW library. Example: [...] So why am i ranting on about this? Well, basically find_package(FFTW3) does not find the fftw library, This module is not part of CMake, so you should complain somewhere else. and other libraries I'm using (hdf5, mkl, ...) share the same fate. Previously I've just hand-added all sorts of include_directories and link_directories but I'm getting fed link_directories() is surely not the solution you need. In fact, it usually creates only more problems. But to have some useful information: set CMAKE_PREFIX_PATH to the path where the libraries can be found without the lib/ or include/ suffix, CMake will add them itself. So if you have fftw2 in /opt/fftw2/lib/libfftw2.a and fftw3 in /opt/fftw-3/lib/libfftw3.so just call: cmake -D CMAKE_PREFIX_PATH=/opt/fftw2 ... Then CMake should pick up that library and all should be well. Eike signature.asc Description: This is a digitally signed message part. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Fwd: module and cmake
2012/5/19 Rolf Eike Beer e...@sf-mail.de:
Am Samstag, 19. Mai 2012, 14:42:49 schrieb Paul Anton Letnes:
Hi all.
I am currently trying to create a more tidy CMakeLists.txt script for
a simulation code I'm working on. The target platforms are mac and
linux desktops (for development) and HPC [0] servers (all linux or, in
some hypothetical future, unix systems). For the desktop/laptop case,
it's mostly easy enough, as one installs libraries in a standard
location - /opt, /usr/local, or something of that sort. On HPC
servers, it is very common to install software in modules. I am not
sure how well aquainted the CMake crowd is with the module system [1]
so I'll post a brief explanation.
HPC servers often have a large amount of users with varying
requirements. Therefore, it is common to require, say, two different
versions of a library, or the same library for two different
compilers. As an example, a machine I'm currently using has several
modules for the FFTW library. Example:
[...]
So why am i ranting on about this? Well, basically
find_package(FFTW3) does not find the fftw library,
This module is not part of CMake, so you should complain somewhere else.
Yes right but may be Paul could have a look at this thread
http://www.cmake.org/pipermail/cmake/2012-January/048742.html
where he can find some explaination about the interaction between
module system
and CMake. Since module plays with environment variable CMake may
or may not be seing the environment change depending on how it is called.
and other libraries I'm using (hdf5,
mkl, ...) share the same fate. Previously I've just hand-added all
sorts of include_directories and link_directories but I'm getting fed
link_directories() is surely not the solution you need. In fact, it usually
creates only more problems.
But to have some useful information: set CMAKE_PREFIX_PATH to the path where
the libraries can be found without the lib/ or include/ suffix, CMake will add
them itself.
Eike is right and I think you (Paul) should read the documentation of
find_package more carefully:
cmake --help-command find_package
the algorithm used to search library, path, files etc... is explained in there.
After that you can try:
cmake --help-variable CMAKE_PREFIX_PATH
cmake --help-variable CMAKE_INCLUDE_PATH
cmake --help-variable CMAKE_LIBRARY_PATH
cmake --help-variable CMAKE_PROGRAM_PATH
...
Concerning the questions:
1) Is there a standard way for CMake to interact with modules?
No, beside the fact that CMake do already use some PATH env var that
module is playing with.
2) Is there a standard way to tell FindXXX.cmake (where XXX=fftw,
hdf5, mkl, etc) where to search for libraries?
Yes read find_package documentation.
3) Can I convince CMake to include link and include directores from
the environment variables, like LIBRARY_PATH, FPATH etc?
Yes you can using $ENV{ENV_VAR_NAME} in order to set
some CMAKE_xxx_PATH variable.
4) How do I handle LIBRARY_PATHs with multiple entries, such as this one?
I'm not sure I understand the question?
CMake has builtin list support:
cmake --help-command list
if you replace the : by a ;
(see cmake --help-command string)
you'll get a CMake list and can use it in CMAKE__PATH var.
--
Erk
Le gouvernement représentatif n'est pas la démocratie --
http://www.le-message.org
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Fwd: module and cmake
and other libraries I'm using (hdf5, mkl, ...) share the same fate. Previously I've just hand-added all sorts of include_directories and link_directories but I'm getting fed link_directories() is surely not the solution you need. In fact, it usually creates only more problems. I know, that's why I'm working on a more elegant solution :) Use target_link_libraries() with an absolute path to the library. But to have some useful information: set CMAKE_PREFIX_PATH to the path where the libraries can be found without the lib/ or include/ suffix, CMake will add them itself. So if you have fftw2 in /opt/fftw2/lib/libfftw2.a and fftw3 in /opt/fftw-3/lib/libfftw3.so just call: cmake -D CMAKE_PREFIX_PATH=/opt/fftw2 ... I know that this might be possible. However, I'd like to get CMake to read the environment directly, as everything is defined there. This will make it easier for other users to compile the code without -D CMAKE_PREFIX_PATH=/funny/path-... for several libraries. If there is a standard environment variable for a library then just put it into the Find*.cmake in the HINTS section (again: cmake --help-command find_library). Eike signature.asc Description: This is a digitally signed message part. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Fwd: module and cmake
Hi, On Sat, May 19, 2012 at 3:44 PM, Rolf Eike Beer e...@sf-mail.de wrote: But to have some useful information: set CMAKE_PREFIX_PATH to the path where the libraries can be found without the lib/ or include/ suffix, CMake will add them itself. So if you have fftw2 in /opt/fftw2/lib/libfftw2.a and fftw3 in /opt/fftw-3/lib/libfftw3.so just call: cmake -D CMAKE_PREFIX_PATH=/opt/fftw2 ... I know that this might be possible. However, I'd like to get CMake to read the environment directly, as everything is defined there. This will make it easier for other users to compile the code without -D CMAKE_PREFIX_PATH=/funny/path-... for several libraries. If there is a standard environment variable for a library then just put it into the Find*.cmake in the HINTS section (again: cmake --help-command find_library). Luckily CMAKE_PREFIX_PATH can also be set as environment variable, not just as cmake variable. So if you can teach the module system to set CMAKE_PREFIX_PATH (or the INCLUDE/LIBRARY variants) you should be good to go. Andreas -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Fwd: module and cmake
Hello Paul, I don't know about your specific find_package file for FFTW, but we do use modules together with CMake, so I'll add my thoughts: As Eric already said, the modules command alters your environment. CMake doesn't know about shell modules, but most find_package commands provide some way to tell them about the location of a package by specifying an environment variable (normally something like PACKAGE_ROOT or PACKAGE_DIR). You'll have to look in the find-package script if this is the case with your FindFFTW.cmake script. If it's not written in the documentation on top of that file, search for $ENV. If your FindFFTW3.cmake does not examine any environment variable, you should fix it in your project (and file a bug with the original project, if it's not your project). If that's not an option, it's probably best if you just set whatever variable your findFFTW3.cmake script expects as a clue as parameter to cmake (e.g. cmake .. -DFFTW3_INCLUDE_DIR=$FFTW3_ROOT/include). So in summary, you should fix your module file to provide a suitable variable pointing to your installation root, and you should also fix poorly written FindXXX.cmake scripts circulating in the wild. Somewhat off-topic, because it's got nothing to do with cmake and your specific problem: it's probably a good idea to prepend PKG_CONFIG_PATH with the pkgconfig directory of fftw. That way, autotool-based projects will be able to find it as well. Cheers, Johannes On Saturday 19 May 2012 14:42:49 Paul Anton Letnes wrote: Hi all. I am currently trying to create a more tidy CMakeLists.txt script for a simulation code I'm working on. The target platforms are mac and linux desktops (for development) and HPC [0] servers (all linux or, in some hypothetical future, unix systems). For the desktop/laptop case, it's mostly easy enough, as one installs libraries in a standard location - /opt, /usr/local, or something of that sort. On HPC servers, it is very common to install software in modules. I am not sure how well aquainted the CMake crowd is with the module system [1] so I'll post a brief explanation. HPC servers often have a large amount of users with varying requirements. Therefore, it is common to require, say, two different versions of a library, or the same library for two different compilers. As an example, a machine I'm currently using has several modules for the FFTW library. Example: paulanto@rocks:~/Rayleigh2D $ module avail fftw fftw/2.1.5 fftw/3.2.2(default) So, there's two different fftw versions available, and I'm currently using the default one as such: paulanto@rocks:~/Rayleigh2D $ module load fftw paulanto@rocks:~/Rayleigh2D $ module list Currently Loaded Modulefiles: 1) fftw/3.2.2 There we go. Now what does it mean to load a module? Well, basically, the module command works by adjusting environment variables: paulanto@rocks:~/Rayleigh2D $ module show fftw /share/apps/modulefiles/fftw/3.2.2: prepend-path CPATH /share/apps/modulessoftware/fftw/fftw-3.2.2/include prepend-path FPATH /share/apps/modulessoftware/fftw/fftw-3.2.2/include prepend-path LD_LIBRARY_PATH /share/apps/modulessoftware/fftw/fftw-3.2.2/lib prepend-path LIBRARY_PATH /share/apps/modulessoftware/fftw/fftw-3.2.2/lib We see that the fftw module sets various paths, and that these will all be different based on the exact version I'm using on the current machine. Hence, I can't tell beforehand, in my CMakeLists.txt, where to search for, say, libfftw3.a. So why am i ranting on about this? Well, basically find_package(FFTW3) does not find the fftw library, and other libraries I'm using (hdf5, mkl, ...) share the same fate. Previously I've just hand-added all sorts of include_directories and link_directories but I'm getting fed up and want to use the more elegant find_package approach, at least where possible. So, what am I asking? 1) Is there a standard way for CMake to interact with modules? 2) Is there a standard way to tell FindXXX.cmake (where XXX=fftw, hdf5, mkl, etc) where to search for libraries? 3) Can I convince CMake to include link and include directores from the environment variables, like LIBRARY_PATH, FPATH etc? 4) How do I handle LIBRARY_PATHs with multiple entries, such as this one? paulanto@rocks:~/Rayleigh2D $ echo $LIBRARY_PATH /share/apps/modulessoftware/fftw/fftw-3.2.2/lib:/share/apps/modulessoftware /hdf5/hdf-5.1.8.5-intel/lib:/share/apps/modulessoftware/openmpi/openmpi-1.4 .3-intel/lib:/share/apps/modulessoftware/intel/compilers/11.1.059/lib/intel 64:/share/apps/modulessoftware/intel/compilers/11.1.059/ipp/em64t/lib:/shar e/apps/modulessoftware/intel/compilers/11.1.059//mkl/lib/em64t:/share/apps/ modulessoftware/intel/compilers/11.1.059/tbb/intel64/cc4.1.0_libc2.4_kernel 2.6.16.21/lib Perhaps I'm not asking the right questions, but hopefully, some can tell me what I _should_ be asking. Or even better, the answer. Cheers Paul
