Re: [cmake-developers] [CMake] LINK_LIBRARIES not spilled to response file

2012-07-09 Thread Claus Klein 

I have added a patch that should help. (Tested on Darwin only)

Please check it for Windows before apply.

Claus

On 09.07.2012, at 12:13, Zaheer Chothia wrote:


Done: http://public.kitware.com/Bug/view.php?id=13385

--Zaheer


--

Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers

Re: [CMake] LINK_LIBRARIES not spilled to response file

2012-07-09 Thread Brett Delle Grazie 
On Jul 8, 2012 10:59 PM, Zaheer Chothia zaheer.chot...@gmail.com wrote:

 Hello,

 I posted a mail here [1] but have yet to receive any replies.  I think my
last
 message was too long and detailed so let me summarize:
 * Command line length is limited on Windows.
 * To alleviate this, CMake places object files into a response file, but
the
   same is not done for libraries.
 * This can pose a problem when the link line becomes too long.

 I am primarily interested in a solution for the Ninja generator, although
this
 issue affects other generators too.  My previous mail contains a testcase
and
 proposed solution.  In the interim another issue [2] was posted, but that
is
 orthogonal and does not solve what is discussed here.

I'd suggest you raise a defect. Kitware are pretty good at fixing things or
suggesting temporary workarounds until the underlying issue is fixed.


 Kind regards,

 --Zaheer

 [1]: http://www.cmake.org/pipermail/cmake/2012-June/051065.html
 [2]: http://public.kitware.com/Bug/view.php?id=13366

 On Thu, Jun 28, 2012 at 3:50 PM, Zaheer Chothia zaheer.chot...@gmail.com
wrote:

 Hello,

 I encountered an issue while building a CMake project where one target
is linked
 against a large number of libraries.  Unlike object files, libraries are
not
 placed into a response file, which can lead to build commands which
exceed the
 length limits on Windows.  For reference, I am using the CMake 2.8.9-rc1
and
 Ninja generator with Microsoft compilers.

 Following this mail is a testcase generator [1] to demonstrate this issue
 (sample project attached for convenience).  The build fails with this
error (for
 readibility I replaced a long sequence of libraries with ...):

 FAILED: cmd.exe /c cd.  C:\Program Files
 (x86)\CMake\bin\cmake.exe -E vs_link_exe
 C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @hello.exe.rsp  /DWIN32
 /D_WINDOWS /W3 /Zm1000 /D_DEBUG /MDd /Zi  /Ob0 /Od /RTC1 /Fehello.exe
 /Fdhello.pdb -link /implib:hello.lib /version:0.0  /STACK:1000
 /machine:X86  /debug /INCREMENTAL /subsystem:console
 src\abcdefghijklmnopqrstuvwxyz0123456789\library1.lib
 src\abcdefghijklmnopqrstuvwxyz0123456789\library2.lib ...
 kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib
 oleaut32.lib uuid.lib comdlg32.lib advapi32.lib  cd.
 The command line is too long.

 ninja: build stopped: subcommand failed.

 Although this example may seem artificial, with the use case I refer to
(i)
 libraries are are specified by absolute paths so they are indeed
reasonably long
 and (ii) since there are third-party libraries involved I would not be
able to
 simply combine source files into one large library as is possible here.
 I
 should also mention that this issue does not affect the Visual Studio
 generators, however it is present with the following: Ninja, MinGW
 Makefiles, NMake Makefiles, MSYS Makefiles.  For Ninja I suspect that the
 indirection via cmd.exe imposes a maximum command length of 8192 KB,
whereas for
 the others this will likely be 32 KB (CreateProcess).

 I would be quite content if this is fixed for the Ninja generator.  A
simple fix
 would be to adapt the build rules by moving $LINK_LIBRARIES from
'command' to
 'rspfile_content':

 --- rules.ninja.bak 2012-06-28 15:23:35 +0100
 +++ rules.ninja 2012-06-28 15:38:09 +0100
 @@ -40,10 +40,10 @@
  # Rule for linking C executable.

  rule C_EXECUTABLE_LINKER_RSPFILE
 -  command = cmd.exe /c $PRE_LINK  C:\Program Files
 (x86)\CMake\bin\cmake.exe -E vs_link_exe
 C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
 /Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
 $LINK_FLAGS $LINK_LIBRARIES  $POST_BUILD
 +  command = cmd.exe /c $PRE_LINK  C:\Program Files
 (x86)\CMake\bin\cmake.exe -E vs_link_exe
 C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
 /Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
 $LINK_FLAGS  $POST_BUILD
description = Linking C executable $out
rspfile = $out.rsp
 -  rspfile_content = $in
 +  rspfile_content = $in $LINK_LIBRARIES

 Best,

 --Zaheer

  [1]: BEGIN: testcase.sh
---
 #!/bin/bash -e

 NUM_LIBRARIES=500
 # Use a long path to quickly exhaust the command-line length limit.
 SRC_DIR=src/abcdefghijklmnopqrstuvwxyz0123456789

 # Root directory: application and CMakeLists.txt
 echo int main() { return 0; }  hello.c

 cat  EOF  CMakeLists.txt
 cmake_minimum_required(VERSION 2.8)

 project(Hello)

 add_subdirectory($SRC_DIR)

 add_executable(hello hello.c)
 target_link_libraries(hello
 EOF

 for ((i = 1; i = $NUM_LIBRARIES; i++)); do
 echo library$i  CMakeLists.txt
 done

 echo )  CMakeLists.txt

 # Libraries: sources and CMakeLists.txt
 mkdir -p $SRC_DIR
 [[ -f $SRC_DIR/CMakeLists.txt ]]  rm $SRC_DIR/CMakeLists.txt
 for ((i = 1; i = $NUM_LIBRARIES; i++)); do
 echo int function$i() { return $i; }  $SRC_DIR/function$i.c
 echo

Re: [CMake] LINK_LIBRARIES not spilled to response file

2012-07-09 Thread Zaheer Chothia 
Done: http://public.kitware.com/Bug/view.php?id=13385

--Zaheer

On Mon, Jul 9, 2012 at 8:15 AM, Brett Delle Grazie
brett.dellegra...@gmail.com wrote:

 On Jul 8, 2012 10:59 PM, Zaheer Chothia zaheer.chot...@gmail.com wrote:

 Hello,

 I posted a mail here [1] but have yet to receive any replies.  I think my
 last
 message was too long and detailed so let me summarize:
 * Command line length is limited on Windows.
 * To alleviate this, CMake places object files into a response file, but
 the
   same is not done for libraries.
 * This can pose a problem when the link line becomes too long.

 I am primarily interested in a solution for the Ninja generator, although
 this
 issue affects other generators too.  My previous mail contains a testcase
 and
 proposed solution.  In the interim another issue [2] was posted, but that
 is
 orthogonal and does not solve what is discussed here.

 I'd suggest you raise a defect. Kitware are pretty good at fixing things or
 suggesting temporary workarounds until the underlying issue is fixed.


 Kind regards,

 --Zaheer

 [1]: http://www.cmake.org/pipermail/cmake/2012-June/051065.html
 [2]: http://public.kitware.com/Bug/view.php?id=13366

 On Thu, Jun 28, 2012 at 3:50 PM, Zaheer Chothia zaheer.chot...@gmail.com
 wrote:

 Hello,

 I encountered an issue while building a CMake project where one target is
 linked
 against a large number of libraries.  Unlike object files, libraries are
 not
 placed into a response file, which can lead to build commands which
 exceed the
 length limits on Windows.  For reference, I am using the CMake 2.8.9-rc1
 and
 Ninja generator with Microsoft compilers.

 Following this mail is a testcase generator [1] to demonstrate this issue
 (sample project attached for convenience).  The build fails with this
 error (for
 readibility I replaced a long sequence of libraries with ...):

 FAILED: cmd.exe /c cd.  C:\Program Files
 (x86)\CMake\bin\cmake.exe -E vs_link_exe
 C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @hello.exe.rsp  /DWIN32
 /D_WINDOWS /W3 /Zm1000 /D_DEBUG /MDd /Zi  /Ob0 /Od /RTC1 /Fehello.exe
 /Fdhello.pdb -link /implib:hello.lib /version:0.0  /STACK:1000
 /machine:X86  /debug /INCREMENTAL /subsystem:console
 src\abcdefghijklmnopqrstuvwxyz0123456789\library1.lib
 src\abcdefghijklmnopqrstuvwxyz0123456789\library2.lib ...
 kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib
 oleaut32.lib uuid.lib comdlg32.lib advapi32.lib  cd.
 The command line is too long.

 ninja: build stopped: subcommand failed.

 Although this example may seem artificial, with the use case I refer to
 (i)
 libraries are are specified by absolute paths so they are indeed
 reasonably long
 and (ii) since there are third-party libraries involved I would not be
 able to
 simply combine source files into one large library as is possible here.
 I
 should also mention that this issue does not affect the Visual Studio
 generators, however it is present with the following: Ninja, MinGW
 Makefiles, NMake Makefiles, MSYS Makefiles.  For Ninja I suspect that the
 indirection via cmd.exe imposes a maximum command length of 8192 KB,
 whereas for
 the others this will likely be 32 KB (CreateProcess).

 I would be quite content if this is fixed for the Ninja generator.  A
 simple fix
 would be to adapt the build rules by moving $LINK_LIBRARIES from
 'command' to
 'rspfile_content':

 --- rules.ninja.bak 2012-06-28 15:23:35 +0100
 +++ rules.ninja 2012-06-28 15:38:09 +0100
 @@ -40,10 +40,10 @@
  # Rule for linking C executable.

  rule C_EXECUTABLE_LINKER_RSPFILE
 -  command = cmd.exe /c $PRE_LINK  C:\Program Files
 (x86)\CMake\bin\cmake.exe -E vs_link_exe
 C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
 /Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
 $LINK_FLAGS $LINK_LIBRARIES  $POST_BUILD
 +  command = cmd.exe /c $PRE_LINK  C:\Program Files
 (x86)\CMake\bin\cmake.exe -E vs_link_exe
 C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
 /Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
 $LINK_FLAGS  $POST_BUILD
description = Linking C executable $out
rspfile = $out.rsp
 -  rspfile_content = $in
 +  rspfile_content = $in $LINK_LIBRARIES

 Best,

 --Zaheer

  [1]: BEGIN: testcase.sh
 ---
 #!/bin/bash -e

 NUM_LIBRARIES=500
 # Use a long path to quickly exhaust the command-line length limit.
 SRC_DIR=src/abcdefghijklmnopqrstuvwxyz0123456789

 # Root directory: application and CMakeLists.txt
 echo int main() { return 0; }  hello.c

 cat  EOF  CMakeLists.txt
 cmake_minimum_required(VERSION 2.8)

 project(Hello)

 add_subdirectory($SRC_DIR)

 add_executable(hello hello.c)
 target_link_libraries(hello
 EOF

 for ((i = 1; i = $NUM_LIBRARIES; i++)); do
 echo library$i  CMakeLists.txt
 done

 echo )  CMakeLists.txt

 # Libraries: sources and CMakeLists.txt
 mkdir -p $SRC_DIR
 [[ -f

Re: [CMake] LINK_LIBRARIES not spilled to response file

2012-07-08 Thread Zaheer Chothia 
Hello,

I posted a mail here [1] but have yet to receive any replies.  I think my
last
message was too long and detailed so let me summarize:
* Command line length is limited on Windows.
* To alleviate this, CMake places object files into a response file, but the
  same is not done for libraries.
* This can pose a problem when the link line becomes too long.

I am primarily interested in a solution for the Ninja generator, although
this
issue affects other generators too.  My previous mail contains a testcase
and
proposed solution.  In the interim another issue [2] was posted, but that is
orthogonal and does not solve what is discussed here.

Kind regards,

*--Zaheer*
*
*
[1]: http://www.cmake.org/pipermail/cmake/2012-June/051065.html
[2]: http://public.kitware.com/Bug/view.php?id=13366

On Thu, Jun 28, 2012 at 3:50 PM, Zaheer Chothia zaheer.chot...@gmail.comwrote:

 Hello,

 I encountered an issue while building a CMake project where one target is
 linked
 against a large number of libraries.  Unlike object files, libraries are
 not
 placed into a response file, which can lead to build commands which exceed
 the
 length limits on Windows.  For reference, I am using the CMake 2.8.9-rc1
 and
 Ninja generator with Microsoft compilers.

 Following this mail is a testcase generator [1] to demonstrate this issue
 (sample project attached for convenience).  The build fails with this
 error (for
 readibility I replaced a long sequence of libraries with ...):

 FAILED: cmd.exe /c cd.  C:\Program Files
 (x86)\CMake\bin\cmake.exe -E vs_link_exe
 C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @hello.exe.rsp  /DWIN32
 /D_WINDOWS /W3 /Zm1000 /D_DEBUG /MDd /Zi  /Ob0 /Od /RTC1 /Fehello.exe
 /Fdhello.pdb -link /implib:hello.lib /version:0.0  /STACK:1000
 /machine:X86  /debug /INCREMENTAL /subsystem:console
 src\abcdefghijklmnopqrstuvwxyz0123456789\library1.lib
 src\abcdefghijklmnopqrstuvwxyz0123456789\library2.lib ...
 kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib
 oleaut32.lib uuid.lib comdlg32.lib advapi32.lib  cd.
 The command line is too long.

 ninja: build stopped: subcommand failed.

 Although this example may seem artificial, with the use case I refer to (i)
 libraries are are specified by absolute paths so they are indeed
 reasonably long
 and (ii) since there are third-party libraries involved I would not be
 able to
 simply combine source files into one large library as is possible here.  I
 should also mention that this issue does not affect the Visual Studio
 generators, however it is present with the following: Ninja, MinGW
 Makefiles, NMake Makefiles, MSYS Makefiles.  For Ninja I suspect that the
 indirection via cmd.exe imposes a maximum command length of 8192 KB,
 whereas for
 the others this will likely be 32 KB (CreateProcess).

 I would be quite content if this is fixed for the Ninja generator.  A
 simple fix
 would be to adapt the build rules by moving $LINK_LIBRARIES from 'command'
 to
 'rspfile_content':

 --- rules.ninja.bak 2012-06-28 15:23:35 +0100
 +++ rules.ninja 2012-06-28 15:38:09 +0100
 @@ -40,10 +40,10 @@
  # Rule for linking C executable.

  rule C_EXECUTABLE_LINKER_RSPFILE
 -  command = cmd.exe /c $PRE_LINK  C:\Program Files
 (x86)\CMake\bin\cmake.exe -E vs_link_exe
 C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
 /Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
 $LINK_FLAGS $LINK_LIBRARIES  $POST_BUILD
 +  command = cmd.exe /c $PRE_LINK  C:\Program Files
 (x86)\CMake\bin\cmake.exe -E vs_link_exe
 C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
 /Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
 $LINK_FLAGS  $POST_BUILD
description = Linking C executable $out
rspfile = $out.rsp
 -  rspfile_content = $in
 +  rspfile_content = $in $LINK_LIBRARIES

 Best,

 --Zaheer

  [1]: BEGIN: testcase.sh
 ---
 #!/bin/bash -e

 NUM_LIBRARIES=500
 # Use a long path to quickly exhaust the command-line length limit.
 SRC_DIR=src/abcdefghijklmnopqrstuvwxyz0123456789

 # Root directory: application and CMakeLists.txt
 echo int main() { return 0; }  hello.c

 cat  EOF  CMakeLists.txt
 cmake_minimum_required(VERSION 2.8)

 project(Hello)

 add_subdirectory($SRC_DIR)

 add_executable(hello hello.c)
 target_link_libraries(hello
 EOF

 for ((i = 1; i = $NUM_LIBRARIES; i++)); do
 echo library$i  CMakeLists.txt
 done

 echo )  CMakeLists.txt

 # Libraries: sources and CMakeLists.txt
 mkdir -p $SRC_DIR
 [[ -f $SRC_DIR/CMakeLists.txt ]]  rm $SRC_DIR/CMakeLists.txt
 for ((i = 1; i = $NUM_LIBRARIES; i++)); do
 echo int function$i() { return $i; }  $SRC_DIR/function$i.c
 echo add_library(library$i function$i.c)  $SRC_DIR/CMakeLists.txt
 done

 echo Testcase has been setup: now build with CMake and Ninja generator.
  [1]: END: testcase.sh
 ---

--

[CMake] LINK_LIBRARIES not spilled to response file

2012-06-28 Thread Zaheer Chothia 
Hello,

I encountered an issue while building a CMake project where one target is linked
against a large number of libraries.  Unlike object files, libraries are not
placed into a response file, which can lead to build commands which exceed the
length limits on Windows.  For reference, I am using the CMake 2.8.9-rc1 and
Ninja generator with Microsoft compilers.

Following this mail is a testcase generator [1] to demonstrate this issue
(sample project attached for convenience).  The build fails with this error (for
readibility I replaced a long sequence of libraries with ...):

FAILED: cmd.exe /c cd.  C:\Program Files
(x86)\CMake\bin\cmake.exe -E vs_link_exe
C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @hello.exe.rsp  /DWIN32
/D_WINDOWS /W3 /Zm1000 /D_DEBUG /MDd /Zi  /Ob0 /Od /RTC1 /Fehello.exe
/Fdhello.pdb -link /implib:hello.lib /version:0.0  /STACK:1000
/machine:X86  /debug /INCREMENTAL /subsystem:console
src\abcdefghijklmnopqrstuvwxyz0123456789\library1.lib
src\abcdefghijklmnopqrstuvwxyz0123456789\library2.lib ...
kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib
oleaut32.lib uuid.lib comdlg32.lib advapi32.lib  cd.
The command line is too long.

ninja: build stopped: subcommand failed.

Although this example may seem artificial, with the use case I refer to (i)
libraries are are specified by absolute paths so they are indeed reasonably long
and (ii) since there are third-party libraries involved I would not be able to
simply combine source files into one large library as is possible here.  I
should also mention that this issue does not affect the Visual Studio
generators, however it is present with the following: Ninja, MinGW
Makefiles, NMake Makefiles, MSYS Makefiles.  For Ninja I suspect that the
indirection via cmd.exe imposes a maximum command length of 8192 KB, whereas for
the others this will likely be 32 KB (CreateProcess).

I would be quite content if this is fixed for the Ninja generator.  A simple fix
would be to adapt the build rules by moving $LINK_LIBRARIES from 'command' to
'rspfile_content':

--- rules.ninja.bak 2012-06-28 15:23:35 +0100
+++ rules.ninja 2012-06-28 15:38:09 +0100
@@ -40,10 +40,10 @@
 # Rule for linking C executable.

 rule C_EXECUTABLE_LINKER_RSPFILE
-  command = cmd.exe /c $PRE_LINK  C:\Program Files
(x86)\CMake\bin\cmake.exe -E vs_link_exe
C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
/Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
$LINK_FLAGS $LINK_LIBRARIES  $POST_BUILD
+  command = cmd.exe /c $PRE_LINK  C:\Program Files
(x86)\CMake\bin\cmake.exe -E vs_link_exe
C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe  /nologo @$out.rsp  $FLAGS
/Fe$out /Fd$TARGET_PDB -link /implib:$TARGET_IMPLIB /version:0.0
$LINK_FLAGS  $POST_BUILD
   description = Linking C executable $out
   rspfile = $out.rsp
-  rspfile_content = $in
+  rspfile_content = $in $LINK_LIBRARIES

Best,

--Zaheer

 [1]: BEGIN: testcase.sh ---
#!/bin/bash -e

NUM_LIBRARIES=500
# Use a long path to quickly exhaust the command-line length limit.
SRC_DIR=src/abcdefghijklmnopqrstuvwxyz0123456789

# Root directory: application and CMakeLists.txt
echo int main() { return 0; }  hello.c

cat  EOF  CMakeLists.txt
cmake_minimum_required(VERSION 2.8)

project(Hello)

add_subdirectory($SRC_DIR)

add_executable(hello hello.c)
target_link_libraries(hello
EOF

for ((i = 1; i = $NUM_LIBRARIES; i++)); do
echo library$i  CMakeLists.txt
done

echo )  CMakeLists.txt

# Libraries: sources and CMakeLists.txt
mkdir -p $SRC_DIR
[[ -f $SRC_DIR/CMakeLists.txt ]]  rm $SRC_DIR/CMakeLists.txt
for ((i = 1; i = $NUM_LIBRARIES; i++)); do
echo int function$i() { return $i; }  $SRC_DIR/function$i.c
echo add_library(library$i function$i.c)  $SRC_DIR/CMakeLists.txt
done

echo Testcase has been setup: now build with CMake and Ninja generator.
 [1]: END: testcase.sh ---


cmake_testcase_many_libraries_rspfile.tar.bz2
Description: BZip2 compressed data
--

Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake