On 11/09/13 16:35, David Cole wrote: >> The EP_ prefix is not needed because this is already an option of >> the ExternalProject_Add function. How about SOURCE_DIR_PRESERVE? > > To me, the name SOURCE_DIR_PRESERVE implies that the source dir, if > it exists, will not be touched by the download command or the > update command at all. > > If it will be modified or deleted at all, then there must be a more > appropriate name than "preserve".
I cannot find a better name, so I used SOURCE_DIR_PRESERVE... Perhaps SOURCE_DIR_SAFE or something similar? > In my opinion, a better strategy in this case is to require source > trees that need preserving or manual interventions to be setup > manually and simply pointed to with SOURCE_DIR, with empty download > and update commands. This makes a lot harder to setup a superbuild, or to add a new module. > My main point being: if you're actively developing code in these > source trees, then they're not really "External", are they? It depends from what you mean with the word "Project". Each researcher of my group works on different "projects", so from their point of view, each project is "external" except for the one they are working on. But perhaps from a "global" point of view they are all subprojects of a main huge project. On 11/09/13 16:46, Bill Hoffman wrote: > If you want to take advantage of ExternalProject, then you have to > make all your projects external even the main one you are > developing. That is just the nature of ExternalProject. So, for > most projects at least one of the projects in a "SuperBuild" will > be the main project they are working on. Having ExternalProject > do an initial checkout is a great way to get an initial build for a > developer. That's exactly the point, especially since there is not a "main project" that everyone is developing, the project is composed by a large number of modules, used by a large heterogeneous group of "lazy" researches (meaning that it is hard to convince them to learn how to use new software tools, and training is complicated), interested only in a few modules, used to SVN (at the moment the project is on a large SVN and people is used to updating only the subfolders they are actively developing, or update everything when there is some major update), with a limited knowledge of git and cmake, and working on different systems (linux, windows, and macos). Also some modules and dependencies can be installed on the system, and therefore should not be downloaded and built, but most of them should be built to get the latest available commit, other modules are optional and should be downloaded only if a user explicitly set an option for them, other are developed by third parties (other research groups scattered around the world) that are actually external, but not dependencies (the opposite actually). Having cmake handle the initial checkout of all the modules, all of this, instead of using git submodules or a separate tools simplifies a lot the management of the project, the user won't have to learn a new tool, and makes it easier to move from SVN since it won't require to move all the modules at once. Anyway, attached is an updated version of the patch (still working for git only and lacking of documentation) fixing the issues reported... Does it look better now? I have a weird issue with policy CMP0012 (see the patch), is there a better way to fix it? Cheers, Daniele
>From c164c1bb12414c8fe0497cc5a0f7d08d6ecb7b4f Mon Sep 17 00:00:00 2001 From: "Daniele E. Domenichelli" <[email protected]> Date: Tue, 10 Sep 2013 17:23:31 +0200 Subject: [PATCH] ExternalProject: Preserve source folder ourside the build tree If the source tree of the project is outside the build tree, do not delete it automatically. This will ensure that if the user keeps the sources outside the build repository, and has some valuable work in it, it won't be lost if he deletes the build directory. Also it should avoid multiple useless clones with different build directories. A new directory property EP_SOURCE_DIR_PRESERVE and a new parameter SOURCE_DIR_PRESERVE of the ExternalProject_Add function will allow one to explicitly set or unset this behaviour for a directory and its subdirectories or just for one project. Source preservation is always disabled when the source directory is inside the build tree. In order to keep the same behaviour, source preservation is enabled by default if CMAKE_MINIMUM_REQUIRED_VERSION is greater than 2.8.13, and disabled otherwise. --- Modules/ExternalProject.cmake | 137 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 129 insertions(+), 8 deletions(-) diff --git a/Modules/ExternalProject.cmake b/Modules/ExternalProject.cmake index 0781ea1..7370009 100644 --- a/Modules/ExternalProject.cmake +++ b/Modules/ExternalProject.cmake @@ -30,6 +30,7 @@ # [TLS_VERIFY bool] # Should certificate for https be checked # [TLS_CAINFO file] # Path to a certificate authority file # [TIMEOUT seconds] # Time allowed for file download operations +# [SOURCE_DIR_PRESERVE 1] # FIXME docs # #--Update/Patch step---------- # [UPDATE_COMMAND cmd...] # Source work-tree update command # [PATCH_COMMAND cmd...] # Command to patch downloaded source @@ -158,6 +159,8 @@ # any ExternalProject_Add calls in your CMakeLists file: # # set_property(DIRECTORY PROPERTY EP_STEP_TARGETS configure build test) +# +# FIXME: Add documentation here #============================================================================= # Copyright 2008-2012 Kitware, Inc. @@ -270,8 +273,16 @@ define_property(DIRECTORY PROPERTY "EP_STEP_TARGETS" INHERITED "ExternalProject module." ) +define_property(DIRECTORY PROPERTY "EP_SOURCE_DIR_PRESERVE" INHERITED + BRIEF_DOCS "Whether source dir stored outside the build directory should be preserved." + FULL_DOCS + "See documentation of the ExternalProject_Add() function in the " + "ExternalProject module." + ) + + -function(_ep_write_gitclone_script script_filename source_dir git_EXECUTABLE git_repository git_tag src_name work_dir gitclone_infofile gitclone_stampfile) +function(_ep_write_gitclone_script script_filename source_dir git_EXECUTABLE git_repository git_tag src_name work_dir gitclone_infofile gitclone_stampfile source_dir_preserve) file(WRITE ${script_filename} "if(\"${git_tag}\" STREQUAL \"\") message(FATAL_ERROR \"Tag for git checkout should not be empty.\") @@ -288,12 +299,70 @@ if(NOT run) return() endif() -execute_process( - COMMAND \${CMAKE_COMMAND} -E remove_directory \"${source_dir}\" - RESULT_VARIABLE error_code - ) -if(error_code) - message(FATAL_ERROR \"Failed to remove directory: '${source_dir}'\") +if(EXISTS \"${source_dir}\" AND ${source_dir_preserve}) + if(NOT IS_DIRECTORY \"${source_dir}\") + # FIXME Perhaps support symbolic links? + message(FATAL_ERROR \"\\\"${source_dir}\\\" exists and is not a git repository. Remove it and try again\") + elseif(NOT IS_DIRECTORY \"${source_dir}/.git\") + file(GLOB files \"${source_dir}/*\") + list(LENGTH files nfiles) + if(nfiles) + message(FATAL_ERROR \"\\\"${source_dir}\\\" folder exists and is not a git repository. Remove it and try again\") + endif() + else() + # Already initialized git repository: no need to clone again + execute_process( + COMMAND \"${git_EXECUTABLE}\" config --local --get remote.origin.url + WORKING_DIRECTORY \"${work_dir}/${src_name}\" + OUTPUT_VARIABLE origin_url + RESULT_VARIABLE error_code + OUTPUT_STRIP_TRAILING_WHITESPACE + ) + if(error_code) + message(FATAL_ERROR \"Failed to get origin remote url in: '${work_dir}/${src_name}'\") + endif() + if(\"\${origin_url}\" STREQUAL \"${git_repository}\") + message(STATUS \"Avoiding repeated git clone, repository already exists\") + return() + else() + string(TIMESTAMP now \"%Y%m%d%H%M%S\") + message(WARNING \"Repository URL is different. Renaming origin remote to origin.\${now}\") + execute_process( + COMMAND \"${git_EXECUTABLE}\" remote rename origin origin.\${now} + WORKING_DIRECTORY \"${work_dir}/${src_name}\" + RESULT_VARIABLE error_code + ) + if(error_code) + message(FATAL_ERROR \"Failed to rename remote in: '${work_dir}/${src_name}'\") + endif() + + execute_process( + COMMAND \"${git_EXECUTABLE}\" remote add origin \"${git_repository}\" + WORKING_DIRECTORY \"${work_dir}/${src_name}\" + RESULT_VARIABLE error_code + ) + if(error_code) + message(FATAL_ERROR \"Failed to add origin remote in: '${work_dir}/${src_name}'\") + endif() + + execute_process( + COMMAND \"${git_EXECUTABLE}\" fetch origin + WORKING_DIRECTORY \"${work_dir}/${src_name}\" + RESULT_VARIABLE error_code + ) + if(error_code) + message(FATAL_ERROR \"Failed to fetch in: '${work_dir}/${src_name}'\") + endif() + endif() + endif() +else() + execute_process( + COMMAND \${CMAKE_COMMAND} -E remove_directory \"${source_dir}\" + RESULT_VARIABLE error_code + ) + if(error_code) + message(FATAL_ERROR \"Failed to remove directory: '${source_dir}'\") + endif() endif() # try the clone 3 times incase there is an odd git clone issue @@ -1199,6 +1268,58 @@ function(_ep_add_download_command name) get_property(url TARGET ${name} PROPERTY _EP_URL) get_property(fname TARGET ${name} PROPERTY _EP_DOWNLOAD_NAME) + # Check if the external project source directory is inside the build directory + set(source_in_binary_dir 0) + get_filename_component(abs_src_dir ${source_dir} ABSOLUTE) + get_filename_component(abs_bin_dir ${CMAKE_BINARY_DIR} ABSOLUTE) + if(\"${abs_src_dir}\" STREQUAL \"${abs_bin_dir}\") + set(source_in_binary_dir 1) + else() + string(LENGTH \"${abs_src_dir}\" src_len) + string(LENGTH \"${abs_bin_dir}\" bin_len) + if(src_len GREATER bin_len) + math(EXPR substr_len \"${bin_len}-1\") + string(SUBSTRING "${abs_src_dir}" 0 ${substr_len} src_dir) + if(src_dir STREQUAL "${abs_bin_dir}/") + set(source_in_binary_dir 1) + endif() + endif() + endif() + + # If source is not in build directory it should be optionally saved (since CMake 2.8.13) + set(source_dir_preserve 0) + if(NOT source_in_binary_dir) + get_property(source_dir_preserve_set TARGET ${name} PROPERTY _EP_SOURCE_DIR_PRESERVE SET) + if(source_dir_preserve_set) + get_property(source_dir_preserve TARGET ${name} PROPERTY _EP_SOURCE_DIR_PRESERVE) + else() + get_property(ep_source_dir_preserve_set DIRECTORY PROPERTY EP_SOURCE_DIR_PRESERVE SET) + if(source_dir_preserve_set) + get_property(source_dir_preserve DIRECTORY PROPERTY EP_SOURCE_DIR_PRESERVE) + elseif(NOT CMAKE_MINIMUM_REQUIRED_VERSION VERSION_LESS 2.8.13) + set(source_dir_preserve 1) + endif() + endif() + endif() + # FIXME is there a better way to avoid this CMP0012 warning? (see line 295) + # + # given arguments: + # + # "EXISTS" "xxx" "AND" "TRUE" + # + # An argument named "TRUE" appears in a conditional statement. Policy + # CMP0012 is not set: if() recognizes numbers and boolean constants. Run + # "cmake --help-policy CMP0012" for policy details. Use the cmake_policy + # command to set the policy and suppress this warning. + cmake_policy(PUSH) + cmake_policy(SET CMP0012 NEW) + if(${source_dir_preserve}) + set(source_dir_preserve 1) + else() + set(source_dir_preserve 0) + endif() + cmake_policy(POP) + # TODO: Perhaps file:// should be copied to download dir before extraction. string(REGEX REPLACE "^file://" "" url "${url}") @@ -1314,7 +1435,7 @@ function(_ep_add_download_command name) _ep_write_gitclone_script(${tmp_dir}/${name}-gitclone.cmake ${source_dir} ${GIT_EXECUTABLE} ${git_repository} ${git_tag} ${src_name} ${work_dir} ${stamp_dir}/${name}-gitinfo.txt ${stamp_dir}/${name}-gitclone-lastrun.txt - ) + ${source_dir_preserve}) set(comment "Performing download step (git clone) for '${name}'") set(cmd ${CMAKE_COMMAND} -P ${tmp_dir}/${name}-gitclone.cmake) list(APPEND depends ${stamp_dir}/${name}-gitinfo.txt) -- 1.8.4.rc3
-- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
