Re: [cmake-developers] [CMake 0016082]: Support alternative download URL in ExternalProject_Add

2016-05-02 Thread Brad King 
On 04/29/2016 03:40 PM, A. Klitzing wrote:
> I APPEND the log of every failed download and print that only if no
> URL works now. So we reserve the detailed error log of a single
> download attempt.

Thanks.  I added the test case with the patch below and
it causes the ExternalProjectLocal test to fail.  The
download tries both alternatives "a;b" as a single URL.

Also it looks like _ep_add_download_command has a lot
of logic that inspects the URL value.  Your change will
need to account for multiple values there as well.  I think
it may be a bit more involved than the patch so far.

Thanks,
-Brad

diff --git a/Tests/ExternalProjectLocal/CMakeLists.txt 
b/Tests/ExternalProjectLocal/CMakeLists.txt
index 17f1630..9d4d98b 100644
--- a/Tests/ExternalProjectLocal/CMakeLists.txt
+++ b/Tests/ExternalProjectLocal/CMakeLists.txt
@@ -126,7 +126,8 @@ ExternalProject_Add_Step(${proj} mypatch
 #
 set(proj TutorialStep1-LocalTGZ)
 ExternalProject_Add(${proj}
-  URL "${CMAKE_CURRENT_SOURCE_DIR}/Step1.tgz"
+  URL "${CMAKE_CURRENT_SOURCE_DIR}/missing/Step1.tgz" # need alternate
+  "${CMAKE_CURRENT_SOURCE_DIR}/Step1.tgz"
   URL_MD5 38c648e817339c356f6be00eeed79bd0
   CMAKE_ARGS -DCMAKE_INSTALL_PREFIX:PATH= -G ${CMAKE_GENERATOR} 

   INSTALL_COMMAND ""

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake 0016082]: Support alternative download URL in ExternalProject_Add

2016-04-29 Thread A. Klitzing 
Hi Brad,

thanks for your advice!
I APPEND the log of every failed download and print that only if no
URL works now. So we reserve the detailed error log of a single
download attempt.

Best regards
   André



2016-04-29 19:45 GMT+02:00 Brad King :
> On 04/28/2016 03:13 PM, A. Klitzing wrote:
>> -  message(FATAL_ERROR \"error: downloading '${remote}' failed
>> +  message(WARNING \"downloading '\${url}' failed
>
> Thanks for working on this!  Please revise the logic to collect
> the list of failed URLs in a list and report an error listing
> all URLs only after they all fail.  Otherwise once the source
> moves all future builds will get warnings even though they
> succeed on one of the alternatives.
>
> Thanks,
> -Brad
>
From b830045866c42010fda85c88765314f4ab9543df Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Andr=C3=A9=20Klitzing?= 
Date: Thu, 28 Apr 2016 21:03:26 +0200
Subject: [PATCH] Add support for multiple/alternative URLs

Now it is possible to pass multiple URLs as a list
that will be tried in and foreach(). So it will
try next URL if the previous failed.

ExternalProject_Add(dummy
URL "${FIRST_URL}" "${SECOND_URL}" "${THIRD_URL}"
)
---
 Modules/ExternalProject.cmake | 25 +++--
 1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/Modules/ExternalProject.cmake b/Modules/ExternalProject.cmake
index 9cc8a20..3b0ce3a 100644
--- a/Modules/ExternalProject.cmake
+++ b/Modules/ExternalProject.cmake
@@ -69,8 +69,8 @@ Create custom targets to build projects in external trees
 URL of mercurial repo
   ``HG_TAG ``
 Mercurial branch name, commit id or tag
-  ``URL /.../src.tgz``
-Full path or URL of source
+  ``URL /.../src.tgz /other/path/src.tgz``
+Full pathes or URLs of source.
   ``URL_HASH ALGO=value``
 Hash of file at URL
   ``URL_MD5 md5``
@@ -901,8 +901,10 @@ function(_ep_write_downloadfile_script script_filename remote local timeout no_p
   endif()
 
   file(WRITE ${script_filename}
-"${hash_check}message(STATUS \"downloading...
- src='${remote}'
+"${hash_check}
+foreach(url ${remote})
+ message(STATUS \"downloading...
+ src='\${url}'
  dst='${local}'
  timeout='${timeout_msg}'\")
 
@@ -910,7 +912,7 @@ ${tls_verify_code}
 ${tls_cainfo_code}
 
 file(DOWNLOAD
-  \"${remote}\"
+  \"\${url}\"
   \"${local}\"
   ${show_progress}
   ${timeout_args}
@@ -921,13 +923,24 @@ list(GET status 0 status_code)
 list(GET status 1 status_string)
 
 if(NOT status_code EQUAL 0)
-  message(FATAL_ERROR \"error: downloading '${remote}' failed
+  string(APPEND logFailedURLs \"downloading '\${url}' failed
   status_code: \${status_code}
   status_string: \${status_string}
   log: \${log}
 \")
 endif()
 
+if(status_code EQUAL 0)
+   break()
+endif()
+endforeach()
+
+if(NOT status_code EQUAL 0)
+   message(FATAL_ERROR \"error: each download failed!
+\${logFailedURLs}
+\")
+endif()
+
 message(STATUS \"downloading... done\")
 "
 )
-- 
2.8.0

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake 0016082]: Support alternative download URL in ExternalProject_Add

2016-04-29 Thread Brad King 
On 04/28/2016 03:13 PM, A. Klitzing wrote:
> -  message(FATAL_ERROR \"error: downloading '${remote}' failed
> +  message(WARNING \"downloading '\${url}' failed

Thanks for working on this!  Please revise the logic to collect
the list of failed URLs in a list and report an error listing
all URLs only after they all fail.  Otherwise once the source
moves all future builds will get warnings even though they
succeed on one of the alternatives.

Thanks,
-Brad

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake 0016082]: Support alternative download URL in ExternalProject_Add

2016-04-28 Thread A. Klitzing 
Hi there!

I added that feature in the attached patch.

Best regards
   André Klitzing
From 856b9e78b00180a232d4b9d498a726489778a07d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Andr=C3=A9=20Klitzing?= 
Date: Thu, 28 Apr 2016 21:03:26 +0200
Subject: [PATCH] Add support for multiple/alternative URLs

Now it is possible to pass multiple URLs as a list
that will be tried in and foreach(). So it will
try next URL if the previous failed.

ExternalProject_Add(dummy
URL "${FIRST_URL}" "${SECOND_URL}" "${THIRD_URL}"
)
---
 Modules/ExternalProject.cmake | 23 +--
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/Modules/ExternalProject.cmake b/Modules/ExternalProject.cmake
index 9cc8a20..7e68b30 100644
--- a/Modules/ExternalProject.cmake
+++ b/Modules/ExternalProject.cmake
@@ -69,8 +69,8 @@ Create custom targets to build projects in external trees
 URL of mercurial repo
   ``HG_TAG ``
 Mercurial branch name, commit id or tag
-  ``URL /.../src.tgz``
-Full path or URL of source
+  ``URL /.../src.tgz /other/path/src.tgz``
+Full pathes or URLs of source.
   ``URL_HASH ALGO=value``
 Hash of file at URL
   ``URL_MD5 md5``
@@ -901,8 +901,10 @@ function(_ep_write_downloadfile_script script_filename remote local timeout no_p
   endif()
 
   file(WRITE ${script_filename}
-"${hash_check}message(STATUS \"downloading...
- src='${remote}'
+"${hash_check}
+foreach(url ${remote})
+ message(STATUS \"downloading...
+ src='\${url}'
  dst='${local}'
  timeout='${timeout_msg}'\")
 
@@ -910,7 +912,7 @@ ${tls_verify_code}
 ${tls_cainfo_code}
 
 file(DOWNLOAD
-  \"${remote}\"
+  \"\${url}\"
   \"${local}\"
   ${show_progress}
   ${timeout_args}
@@ -921,13 +923,22 @@ list(GET status 0 status_code)
 list(GET status 1 status_string)
 
 if(NOT status_code EQUAL 0)
-  message(FATAL_ERROR \"error: downloading '${remote}' failed
+  message(WARNING \"downloading '\${url}' failed
   status_code: \${status_code}
   status_string: \${status_string}
   log: \${log}
 \")
 endif()
 
+if(status_code EQUAL 0)
+   break()
+endif()
+endforeach()
+
+if(NOT status_code EQUAL 0)
+   message(FATAL_ERROR \"error: each download failed\")
+endif()
+
 message(STATUS \"downloading... done\")
 "
 )
-- 
2.8.0

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] [CMake 0016082]: Support alternative download URL in ExternalProject_Add

2016-04-26 Thread Mantis Bug Tracker 

The following issue has been SUBMITTED. 
== 
https://public.kitware.com/Bug/view.php?id=16082 
== 
Reported By:A. Klitzing
Assigned To:
== 
Project:CMake
Issue ID:   16082
Category:   CMake
Reproducibility:always
Severity:   minor
Priority:   normal
Status: new
== 
Date Submitted: 2016-04-26 09:04 EDT
Last Modified:  2016-04-26 09:04 EDT
== 
Summary:Support alternative download URL in
ExternalProject_Add
Description: 
We use a script with a lot of ExternalProject_Add to generate our dependencies.
But some project moves their files to different destination after some time. For
example: openssl

It would be very helpful if ExternalProject_Add could support alternative
download URLs as a list. So cmake could try the next URL if a connection error
occurs.

ExternalProject_Add(dummy
URL "${FIRST_URL}" "${SECOND_URL}" "${THIRD_URL}"
)


Additional Information: 
OpenSSL:

Current version:
ftp://ftp.openssl.org/source/

If a new version arrives the old one will be moved to:
ftp://ftp.openssl.org/source/old/

== 

Issue History 
Date ModifiedUsername   FieldChange   
== 
2016-04-26 09:04 A. KlitzingNew Issue
==

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers